Ai Runtime Engineer Jobs

About Syllo Syllo is on a mission to transform litigation. Our product is a unified litigation platform that enables lawyers and paralegals to safely harness the power of language models and agentic AI throughout the litigation life cycle. Since going to market, we have gained a diverse group of enterprise customers, including some of the biggest law firms and corporations in the country, and we are quickly expanding. By reducing the expense of litigation industry-wide, we aim to improve access to high-quality representation and promote the alignment of legal outcomes with merit. About the Role As we continue to scale our AI platform, we're investing in our own inference stack to deliver best-in-class performance, reliability, cost efficiency, and flexibility across the latest generation of open-source language models. We're looking for a Staff Software Engineer to spearhead this effort. You'll define the architecture, evaluate emerging technologies, and build the systems that power model serving. You'll partner closely with machine learning, infrastructure, and product engineering to establish the foundation for how AI models are deployed, optimized, monitored, and operated in production. This is a highly hands-on technical role. You'll spend the majority of your time designing, building, and optimizing production systems while helping shape our long-term AI infrastructure strategy. Responsibilities Lead the design and development of our production inference platform. Define the technical roadmap for inference infrastructure, model serving, and runtime optimization. Build and operate scalable, cost-effective systems for serving large language models in production. Evaluate and integrate modern inference technologies, frameworks, and serving runtimes. Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost. Develop systems for model deployment, traffic routing, autoscaling, scheduling, observabil

NVIDIA is at the forefront of the generative AI revolution, building the software and systems that power the world’s most advanced large language model workloads. We are looking for a Software Engineer focused on bring-up, triage, benchmarking, analysis, and optimization of distributed training and inference workloads across NVIDIA GPU platforms at the largest scales we run. In this role you will help bring up, benchmark, and debug distributed LLM workloads on multi-GPU and multi-node deployments, and own the design and implementation of the benchmarking tooling, automation, and debugging workflows that support them. This is a hands-on role for an engineer who enjoys deep technical problems across deep learning systems, GPU performance, distributed computing, and large-scale operations. What you’ll be doing: Bring up, validate, and debug large-scale AI clusters, infrastructure, and end-to-end workloads. Bring up, tune, and benchmark AI pre-training, post-training, and inference workloads using PyTorch, NeMo / Megatron, TensorRT-LLM, and adjacent NVIDIA AI software stacks. Perform root-cause analysis of failures in large distributed environments Contribute to the resilience and failure-attribution tooling that detects, triages, and attributes node, fabric, and workload failures across the cluster. Build and maintain repeatable benchmark suites, automation, acceptance criteria, and qualification workflows on new platforms. Tune runtime settings, communication parameters, and deployment configurations in close partnership with framework, systems, and platform teams. Deliver actionable, data-driven recommendations based on profiling, benchmark results, and cluster characterization. What we ne

DoorDash is building the world's most reliable on-demand, logistics engine for delivery! We're looking for talented engineers to help us develop a 24x7, global infrastructure system that powers DoorDash's three-sided marketplace of consumers, merchants, and dashers. About the Team Data Platform’s Data User Experience Engineering builds the products every DoorDash employee uses to answer questions with data. That portfolio includes our AI first reporting and authoring surface,Conversational analytics with AI, Semantic and Knowledge layer along with MCP and skills layer plus the full stack infrastructure that lets users and agents reach all of it. Our users span analysts, engineers, operators, and executives across DoorDash, Wolt, and Deliveroo, on a warehouse and OLAP footprint that includes Snowflake,Spark and Clickhouse. The team is a mix of engineers who have built AI & Data platforms and products at scale. About the Role You will own the full path from a question to an answer an operator can put in a deck without checking it twice. The BI reporting surfaces are converging, and deciding what that convergence looks like while someone is authoring, building a report, writing a query, asking why a number moved, or asking for the chart they were about to build by hand. You’re excited about this opportunity because you will… Ship agentic analytics features to thousands of users where a wrong number is a real business decision made badly, and accuracy is the product Build a data platform for authoring, dashboard runtime, drill-down and cross-filter semantics, scheduling and alerting, content certification and permissions, and the embedding path that puts the same components in front of external users Streaming large result sets to the browser, rendering dozens of interdependent tiles. Build the AI experience inside that platform: agent-assisted query authoring, generated charts and widgets held to the same governance as hand-built ones Evolve the semantic laye

Backblaze is the object storage leader in the open cloud movement, fueling customer success with cloud storage built purposefully to unlock budgets, unburden administrators, and unleash innovators. Together with our partners, we’re helping customers break free from the restrictive, overpriced legacy solutions that hold them back, and blaze forward with the full power of the open cloud in their hands. Founded in 2007, we scaled the business with less than $3 million in outside funding until 2021, when we did a traditional IPO on the Nasdaq stock exchange. Today, Backblaze generates over $100m in revenue and is the leading specialized storage cloud - managing over three billion gigabytes of data storage for 500K+ customers in 175+ countries, including businesses, developers, IT professionals, and individuals. But while there is a lot to celebrate in our past, there is almost as much opportunity ahead of us. We are seeking a Sr. AI Security Engineer to join our team! About The Role Backblaze is seeking a Senior AI Security Engineer to design and implement safeguards for internal AI usage , with a focus on agentic systems, developer protection, and runtime security . This is a hands-on role for a practitioner who has built and deployed security controls , not just defined policy. You will enable teams to safely use AI by creating enforcement layers, identity controls, and detection capabilities that constrain and monitor AI-driven activity. What You’ll Do: Agentic AI Safeguards Architect and implement guardrails for tool-using AI systems , including: Tool access controls and allowlists Context and memory isolation Step-level validation of agent actions Apply mitigations aligned to the OWASP Agentic AI Top 10 (e.g., prompt injection, unsafe tool use, data leakage, excessive autonomy) Runtime Security Controls Build enforcement mechanisms that govern AI behavior at execution time: Interceptors, proxies, or middleware for tool/API calls Policy decision and enforcement laye

EnCharge AI is a leader in advanced AI hardware and software systems for edge-to-cloud computing. EnCharge’s robust and scalable next-generation in-memory computing technology provides orders-of-magnitude higher compute efficiency and density compared to today’s best-in-class solutions. The high-performance architecture is coupled with seamless software integration and will enable the immense potential of AI to be accessible in power, energy, and space constrained applications. EnCharge AI launched in 2022 and is led by veteran technologists with backgrounds in semiconductor design and AI systems. About the Role EnCharge AI is looking for an Embedded SW Engineer to develop the firmware for our Edge AI processors. The candidate must possess an excellent understanding of computer architecture and operating system concepts including, but not limited to, memory management, virtualization and PCIe address space. The role includes designing and developing the core Firmware for various parts of the SOC. The candidate must possess strong communication skills to interface with Runtime, Architecture and H/W teams. Responsibilities Develop the critical pieces of EAI Firmware used to deploy inference jobs on EAI processors Validate different IP blocks on the SOC Evaluate and integrate third-party device drivers to interface with EnCharge’s SW stack Work closely with the Runtime, Hardware and Architecture teams define the driver architecture Qualifications Bachelors in EE/CS Advanced programming skills in C/C++ for operating system kernel & systems development Understanding of RISC-V architecture is a plus Exposure to PCIe BAR and IOMMU architecture Exposure to virtualization and hypervisor technologies Deep understanding of operating systems concepts, data structures, x86-64 and accelerator architectures Experience with low-level debug tools as well as emulators and simulators Experience running, analyzing, and tuning system performance benchmarks Excellent verb

About the Role We're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency. The core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered. A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship. You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production. Responsibilities Build the provisioning state machine: design and implement

About the Role Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale. If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI! Responsibilities Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale. Develop and optimize runtime inference services for large-scale AI applications. Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world. Conduct design and code reviews to ensure high standards of quality. Create services, tools, and developer documentation to support the inference engine. Implement robust and fault-tolerant systems for data ingestion and processing. Requirements 3+ years of experience writing high-performance, well-tested, production-quality code. Proficiency with Python and PyTorch. Demonstrated experience in building high performance libraries and tooling. Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale. Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum Preferred: Knowledge of AI inference techniques such as speculative decoding. Preferred: Knowledge of CUDA/Triton programming. Nice to have: Knowledge of Rust, Cython and compilers. About Together AI Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference th

About the Role REMOTE IN INDIA We're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency. The core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered. A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship. You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production. Responsibilities Build the provisioning state machine

About the Role We're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency. The core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered. A product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship. You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production. Responsibilities Build the provisioning state machine: design and implement

About Us: AI needs a new infrastructure layer. We're building it at Modal. Every era of computing brought new workloads that previous infrastructure couldn't support: mainframes, databases, and the cloud. Each time, the company that rebuilt the layer underneath defined the decade. AI is no different, except it touches everything instead of one slice, and the window to build the layer underneath it is open right now. Our customers include category-defining companies like Ramp , Cognition, DoorDash, and Suno. They rely on Modal for instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. We've passed $1B in annualized run rate, up 15x from a year ago. In October 2026 we raised a $750M Series D led by Accel at a $15.75B valuation. Our team includes creators of popular open-source projects (e.g., Seaborn , Luigi ), academic researchers, international olympiad medalists, and experienced engineering and product leaders with decades of experience. The Role: Modal’s Inference Runtime team owns the container runtime stack used to run inference and training workloads across our fleet. We work at the boundary of Linux, containers, filesystems, storage, GPU drivers, and distributed systems.Our goal is to make demanding ML workloads start quickly, run efficiently, and remain securely isolated — whether they use a single GPU, hundreds of gigabytes of memory, or multiple GPUs connected with RDMA.We’re looking for a systems engineer who enjoys working deep in the stack. You’ll build production runtime infrastructure in Rust and Go, diagnose difficult Linux and performance problems, and help determine the architecture of Modal’s container platform. You’ll also work closely with maintainers of gVisor and contribute to the runtime itself when the right fix belongs upstream. The Work: Make container startup, checkpoint, and restore dramatically faster for large i

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role The Computer Use team teaches Claude to see and operate computer interfaces, and builds the agent harness and end-user products that turn that capability into real tools. The team sits inside Anthropic's research organization and closes the loop between product and model. As a Product Engineer on this team, you'll own end-to-end delivery of our computer-use and browser-control product surfaces. You'll build across the full stack, from the user interface to the agent runtime to the backend services behind it. You'll work directly alongside researchers, with no layers between you and the model or the user. This is a dynamic role in which priorities evolve frequently. Success depends on a high tolerance for ambiguity, the adaptability to shift focus as needs change, and the agility and discernment to continuously prioritize the highest impact work. Key responsibilities Own end-to-end delivery of computer-use and browser-control product surfaces: scope, build, ship, measure, and iterate Diagnose and resolve reliability and robustness issues in the computer-use agent harness that block real-world usage Partner with computer-use researchers Partner with the Claude Cowork team on shared surfaces, integrations, and knowledge-worker workflows Instrument products and use usage data to drive prioritization and measure progress Translate fuzzy user pain points into concrete, shippable features for knowledge workers Minimum qualifications Experience building and shipping a product from zero to one with end-to-end ownership, as a founding or early engineer at a startup or with equivalent ownership inside a larger company

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Every time Claude runs code, uses a computer, or works as an agent, it does so inside a sandbox. Our Sandboxing team builds the isolation layer behind that work. The same systems run the code Claude writes for users, the environments we train and evaluate models in, and the agents our customers deploy. These workloads are untrusted by design, and there are a lot of them. Sandboxes need to start in milliseconds, hold up under adversarial code, and scale to a very large number of concurrent sessions. You will work at the boundary between the Linux kernel, virtualization, and the container ecosystem. You will partner with research, product, and security teams to make sure that safe and fast do not trade off against each other. Key responsibilities Design, build, and operate the sandboxing infrastructure that runs untrusted, model-generated code and agent workloads at scale Improve sandbox startup time, density, and throughput, including snapshot and restore, checkpointing, and warm pools Strengthen isolation across compute, filesystem, and network, including egress controls and per-session credential handling Investigate and fix performance, reliability, and security issues at the kernel, hypervisor, and container runtime level Build the monitoring, tracing, and debugging tools that show where time and resources go inside a sandbox Work with researchers to support reinforcement learning environments and evaluations that need fast, reproducible, isolated execution Work with product and platform engineers to expose sandboxes as a dependable, well-designed capability for Claude products and customers Partner with security tea

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Anthropic's Environments organization builds and maintains the infrastructure that improves Claude’s capabilities through reinforcement learning. That includes the frameworks researchers use to build environments and the infrastructure responsible for running them. The team's mission is to productionize research. You'll embed with research teams, get up to speed on how they work, and design the frameworks and APIs that let them move faster, building systems the team can understand, own, and maintain themselves. Scope also includes keeping production RL runs healthy, maintainable, monitored, and easy to triage. You'll be a strong fit if you have deep expertise in Python, a refined sense of taste for API and framework design, and good intuition for how complex systems fail, especially silently. It's a bonus if you've built and operated a stateful distributed system, such as a workflow engine, actor framework, or durable-execution runtime, where correctness depends on getting shared state and recovery right. You should be comfortable diving into messy research code, finding the abstractions that matter, and improving them incrementally while researchers continue to build on your work. You should also be comfortable using AI tools to accelerate your own development, but have an impulse towards deep verification. Key responsibilities Design widely used APIs, frameworks, and abstractions that other engineers and researchers build on, making correct usage the default and ruling out entire classes of errors structurally Own the platform layers that sit beneath every environment, including the agent runtime Build the

Tenstorrent is leading the industry on cutting-edge AI technology, revolutionizing performance expectations, ease of use, and cost efficiency. With AI redefining the computing paradigm, solutions must evolve to unify innovations in software models, compilers, platforms, networking, and semiconductors. Our diverse team of technologists have developed a high performance RISC-V CPU from scratch, and share a passion for AI and a deep desire to build the best AI platform possible. We value collaboration, curiosity, and a commitment to solving hard problems. We are growing our team and looking for contributors of all seniorities. As a Low-Level Software Engineer on Tenstorrent’s Low-Level Kernel (LLK) team, you’ll work at the intersection of software and AI hardware to build the primitives that power our workloads. LLKs sit directly on top of the ISA of Tenstorrent’s AI hardware and form the foundation for the software stack above them. You’ll tackle challenging problems in low-level programming, hardware-aware optimization, and performance engineering, with the opportunity to influence how our hardware is programmed and how efficiently AI workloads run. Improvements at this layer can have an outsized impact across the entire stack. You’ll take ownership of complex technical problems from design through implementation and optimization, while working closely with engineers across LLK, hardware, runtime, and operations. This role is hybrid, based out of Toronto, Canada. We welcome candidates at various experience levels for this role. During the interview process, candidates will be assessed for the appropriate level, and offers will align with that level, which may differ from the one in this posting. Who You Are Strong C/C++ engineer with significant experience building low-level, performance-critical software. Deep understanding of computer architecture and how software maps to hardware. Strong problem solver who can investigate complex performance and syst

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. About the Role We are looking for an AI Agent Security Architect to function as the primary technical authority for Replit’s autonomous and AI agent security blueprint. In this critical role, you will design, implement, and maintain the runtime defense systems, guardrail frameworks, and sandboxing architectures that govern AI agents executing code, invoking tools, and reasoning across our platform. You will be a key technical contributor—leading high-impact AI security initiatives and bridging the gap between non-deterministic AI behavior and rigorous cybersecurity controls for both engineering and executive leadership. What You'll Do AI Agent Security Strategy & Technical Execution AI Agent Security Blueprint: Define the long-term vision and architectural patterns for securing autonomous agent workflows, Model Context Protocol (MCP) integrations, multi-turn reasoning loops, and multi-agent coordination. Runtime Guardrails & Policy Enforcement: Architect and deploy dynamic input/output guardrail systems, semantic firewalls, and real-time intent verification filters to prevent goal hijacking, system prompt leaks, and indirect prompt injections. Agent Execution & Tool Sandboxing: Partner with Infrastructure and AppSec teams to design secure, short-lived, micro-isolated environments (e.g., microVMs, WebAssembly, container sandboxes) where agents can dynamically execute code, run shell commands, and interact with host operating systems safely. Agentic Threat Modeling & Red Teaming: Conduct specialized threat modeling against non-deterministic systems. Lead automated and manual AI red-teaming initiatives to uncover vulnerabilities in RAG context pipelines, vector stores, and tool-calling interfaces. Identity