Inference Technical Lead Jobs

About the Team OpenAI's Industrial Compute organization is building the world's most advanced AI infrastructure ecosystem. Through a combination of strategic partnerships and self-built campuses, we are scaling the compute, storage, and networking platforms that power frontier AI training and inference. The Scaling Analytics team builds the data and software systems that help Industrial Compute understand, plan, and operate infrastructure at global scale. We work across capacity, hardware, storage, infrastructure software, and operational systems to connect fragmented sources of infrastructure data and make that information reliable and usable for engineering and planning. As OpenAI's infrastructure footprint grows, CPU and storage data increasingly spans internal platforms, vendor systems, APIs, databases, object storage, capacity management systems, and operational tooling. Building reliable connections across these environments is critical to understanding available capacity, utilization, fleet state, and infrastructure growth. About the Role We are seeking a Data Engineer to build the data systems and integrations that connect OpenAI's CPU, storage, and supporting infrastructure platforms. This role sits at the intersection of data engineering and backend software engineering. Rather than focusing primarily on traditional analytical pipelines, you will build the software and integrations required to collect, normalize, and make infrastructure data available across a heterogeneous set of systems. CPU and storage data may originate from internal infrastructure platforms, vendor APIs, databases, object storage, capacity systems, and operational services. You will determine how to reliably connect these systems and where those integrations should live—whether within an existing infrastructure service, an orchestration framework, a scheduled workload, or a purpose-built application. You will work closely with Infrastructure Engineering, Capacity Engineering, Storage,

About the Team OpenAI’s Infrastructure Operations team is responsible for the availability, reliability, and operational excellence of one of the world’s largest AI infrastructure networks. The team owns day-to-day operations of production AI networks across Industrial Compute's data centers, working with colocation providers, deployment teams, and hardware vendors to deliver highly available GPU infrastructure for AI training and inference workloads. About the Role We are seeking an Infrastructure Operations Engineer to operate and improve the large-scale Ethernet fabrics that support GPU clusters, storage systems, and management infrastructure. This role combines hands-on production operations with automation, observability, and incident response across a global AI network. The ideal candidate has experience operating high-availability data center, cloud, AI, or HPC networks and can move comfortably from physical-layer troubleshooting to routing and fabric behavior, change execution, and root-cause analysis. You will partner closely with network architecture, systems engineering, GPU engineering, storage engineering, security, deployment, site operations, service providers, colocation partners, and hardware vendors to raise reliability and reduce operational toil. Key Responsibilities Own the operational health, availability, and reliability of production AI network infrastructure across Industrial Compute's data centers. Monitor, troubleshoot, and resolve network incidents while meeting service-level objectives (SLOs), reducing Mean Time to Detect (MTTD), and minimizing Mean Time to Recovery (MTTR). Operate and maintain large-scale Ethernet fabrics supporting GPU compute, storage, and management networks. Execute production network changes, maintenance windows, and capacity expansions with minimal customer impact. Manage the hardware lifecycle, including switch and optics replacements, RMA coordination, software upgrades, and preventive maintenance. Support new A

About the team The Applied team safely brings OpenAI's technology to the world. We released ChatGPT; Plugins; DALL·E; and the APIs for GPT-5, embeddings, and fine-tuning. We also operate inference infrastructure at scale. There's a lot more on the immediate horizon. Our customers build fast-growing businesses around our APIs, which power product features that were never before possible. ChatGPT is a prime example of what is currently possible. We simultaneously ensure that our powerful tools are used responsibly. Safe deployment is more important to us than unfettered growth. The Fraud Engineering team works within our Applied Engineering organization identifying and responding to fraudsters on our platform. We are looking for a software engineer with anti fraud & abuse experience to help architect and build our next-generation anti-fraud systems. About the role The Scaled Abuse team protects OpenAI’s products and customers by detecting, preventing, and responding to fraudulent and abusive behavior at scale. We build and operate the backend and data systems that power real-time detection, investigation workflows, and enforcement — balancing strong protections with a great user experience as the platform grows. Our work sits at the intersection of engineering and abuse expertise: we partner closely with Trust & Safety, Security, and Product to understand emerging attack patterns, translate messy signals into clear system behavior, and continuously harden our defenses. The problems are dynamic and ambiguous by default, so we value engineers who can quickly dive into an unfamiliar codebase, develop strong intuition about how it works end-to-end, and propose pragmatic improvements that make the entire stack more resilient. In this role, you will: Design and build systems for fraud detection and remediation while balancing fraud loss, cost of implementation, and customer experience Work closely with finance, security, product, research, and trust & safety ope

About the Team The Private Computing team works across product, engineering, security, and safety to build advanced privacy products and infrastructure at OpenAI. Our mission is to provide world-class security features to users so their private data remains private, even from OpenAI. We use technologies like confidential computing, trusted execution environments, and end-to-end encryption to ship product features across ChatGPT, the API, and our future consumer devices. About the Role We’re looking for software engineers to design, build, and scale novel privacy features and infrastructure across ChatGPT, API, and future consumer devices. In this role, you will: Ship fast while balancing difficult trade-offs in complex domains Build core abstractions for trusted execution environments and end-to-end-encryption Build product features for private inference and storage across ChatGPT, API, and future consumer devices Update build systems to increase trust and verifiability Integrate with safety and integrity infrastructure Operate systems at scale with high reliability, including an on-call rotation Collaborate with a diverse set of cross-functional teams across product, engineering, security, safety, policy, and legal You might thrive in this role if you: Care deeply about user privacy and security Have 5+ years of experience in professional software engineering Have experience building and scaling confidential computing or encryption technologies in production environments Have experience with Kubernetes and cloud orchestration systems Take pride in building and operating scalable, reliable, secure systems Can collaborate well and drive alignment in the face of difficult trade-offs Are comfortable with ambiguity and rapid change Workplace & Location This role is based in San Francisco, CA. We follow a hybrid model with 4 days a week in the office and offer relocation assistance to new employees. About OpenAI OpenAI is an AI research and deployment company dedicat

About the Team OpenAI's Training team is responsible for producing the large language models that power our research, our products, and ultimately bring us closer to AGI. Achieving this goal requires combining deep research into improving our current architecture and optimization techniques, alongside long-term bets aimed at improving the efficiency and capability of future generations of models. We are responsible for integrating these techniques and producing model artifacts used by the rest of the company, and ensuring that these models are world-class in every respect. About the Role As a member of the training team, you will push the frontier of LLM development for OpenAI's flagship models, enhancing intelligence, efficiency, and adding new capabilities. Relevant interests may include areas such as architecture design, long-context and efficient attention, optimization and the science of scaling. Ideal candidates have a deep understanding of LLM architectures, a sophisticated understanding of model inference, and a hands-on empirical approach. A good fit for this role will be equally happy coming up with a creative breakthrough, investing in strengthening a baseline, designing an eval, debugging a thorny regression, or tracking down a bottleneck. This role is based in London. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Design, prototype and scale up new architectures to improve model intelligence Execute and analyze experiments autonomously and collaboratively Study, debug, and optimize both model performance and computational performance Contribute to training and inference infrastructure You might thrive in this role if you: Have experience landing contributions to major LLM training runs Can thoroughly evaluate and improve deep learning architectures in a self-directed fashion Are motivated by safely deploying LLMs in the real world Are well-versed in the state of the a

About the team The Applied team at OpenAI safely brings cutting-edge technology to the world. We have released groundbreaking products such as ChatGPT, Plugins, DALL·E, and APIs for GPT-4, GPT-3, embeddings, and fine-tuning. Our team also manages large-scale inference infrastructure. With much more on the horizon, our impact continues to grow. Our customers create fast-growing businesses using our APIs, enabling product features previously unimaginable. ChatGPT exemplifies the current scope of possibilities. We prioritize the responsible use of our powerful tools, valuing safe deployment over unchecked expansion. Within Applied Engineering, the Financial Engineering team ensures that our products are monetized effectively to accommodate customers' varying needs and scales. Collaborating closely with the GTM and Finance teams, we strive to tailor our billing stack to our evolving internal requirements. We seek an experienced engineer to architect and refine our billing systems, enhancing their functionality to meet the demands of our increasingly complex and expansive product offerings. In this role, you will: Architect and build the next generation of billing and monetization systems at OpenAI. Develop across the stack to create comprehensive billing integrations for our range of ChatGPT and API users. Design a versatile billing platform suitable for both subscription and usage-based offerings, ensuring scalability and enterprise readiness/flexibility. Construct and integrate tools that empower internal teams to seamlessly incorporate billing data into their workflows. Collaborate closely with a wide array of stakeholders, including the Product, Data, Finance, and Go-To-Market teams, as well as fellow engineers. You might thrive in this role if you: Possess a minimum of 5 years of professional software engineering experience, with added experience in payments, billing, or monetization seen as a bonus. Enjoy engaging with various partners, particularly those outside

About the Team The Statsig team at OpenAI builds and operates the experimentation platform that powers product development, measurement, and decision-making across the company. We partner closely with product, engineering, and infrastructure teams to ensure experiments are trustworthy, statistically rigorous, and scalable to the needs of frontier AI products. Our mission is to help teams make better decisions through reliable experimentation. We care deeply about statistical correctness, pragmatic solutions, and building systems that researchers and engineers can trust at massive scale. The team operates at the intersection of experimentation methodology, data infrastructure, causal inference, and product analytics. We are looking for experienced experimentation experts who want to shape the future of experimentation in the AI era. About the role: We're seeking a Data Engineer to take the lead in building our data pipelines and core tables for OpenAI. These pipelines are crucial for powering analyses, safety systems that guide business decisions, product growth, and prevent bad actors. If you're passionate about working with data and are eager to create solutions with significant impact, we'd love to hear from you. This role also provides the opportunity to collaborate closely with the researchers behind ChatGPT and help them train new models to deliver to users. As we continue our rapid growth, we value data-driven insights, and your contributions will play a pivotal role in our trajectory. Join us in shaping the future of OpenAI! In this role, you will: Design, build and manage our data pipelines, ensuring all user event data is seamlessly integrated into our data warehouse. Develop canonical datasets to track key product metrics including user growth, engagement, and revenue. Work collaboratively with various teams, including, Infrastructure, Data Science, Product, Marketing, Finance, and Research to understand their data needs and provide solutions. Implement ro

About the Team The Hardware Health and Observability team owns the end-to-end health lifecycle of OpenAI’s global compute fleet. Our mission is to maximize healthy, usable compute across accelerator vendors, generations, cloud providers, and regions through reliable health signals, automated remediation, and scalable operational tooling. We build the systems that observe, detect, remediate, and verify hardware issues across GPUs, CPUs, networking, and platform infrastructure, enabling frontier model training and inference workloads to run reliably at hyperscale. We are the last line of defense for the success of OAI’s production and research workloads. About the Role On the Hardware Health and Observability team, you’ll build critical infrastructure that keeps OpenAI’s largest compute clusters healthy and operational at scale. Even small numbers of unhealthy systems can impact large-scale training and inference workloads. This team focuses on minimizing downtime, improving fleet efficiency, and ensuring compute resources remain continuously available to researchers and product teams. Engineers on this team own problems end-to-end, from defining health signals and debugging failures to building automated remediation systems that operate across millions of GPUs globally. In this role, you will: Define and maintain health signals across GPUs, CPUs, networking, and platform infrastructure. Build and evolve health checks that detect, remediate, and verify failures at scale. Ensure critical health checks execute with minimal latency to maximize workload uptime. Investigate hardware failures and system-level issues across large-scale compute environments. Own node lifecycle workflows including drain, quarantine, repair, RMA, and return-to-service processes. Build automation and tooling that enables global cluster management with minimal manual intervention. Partner with workload, reliability, and provider teams to integrate health signals into training and inference system

About the Team OpenAI’s Hardware organization develops AI-native silicon and system-level solutions for the unique demands of advanced AI workloads. Building on efforts like Jalapeño, the team is developing future generations of AI-native silicon and tightly integrated systems to power the next generation of frontier models. By co-designing chips, systems, tools, and methodologies, the team helps deliver faster, more efficient, and production-ready hardware for OpenAI’s supercomputing platform. About the Role On the Accelerators team, you will help OpenAI evaluate and bring up new compute platforms that can support large-scale AI training and inference. Your work will range from prototyping system software on new accelerators to enabling performance optimizations across our AI workloads. You’ll work across the stack, collaborating with both hardware and software aspects - working on kernels, sharding strategies, scaling across distributed systems, and performance modeling. You'll help adapt OpenAI's software stack to non-traditional hardware and drive efficiency improvements in core AI workloads. This is not a compiler-focused role, rather bridging ML algorithms with system performance - especially at scale. In this role, you will: Prototype and enable OpenAI's AI software stack on new, exploratory accelerator platforms. Optimize large-scale model performance (LLMs, recommender systems, distributed AI workloads) for diverse hardware environments. Develop kernels, sharding mechanisms, and system scaling strategies tailored to emerging accelerators. Collaborate on optimizations at the model code level (e.g. PyTorch) and below to enhance performance on non-traditional hardware. Perform system-level performance modeling, debug bottlenecks, and drive end-to-end optimization. Work with hardware teams and vendors to evaluate alternatives to existing platforms and adapt the software stack to their architectures. Contribute to runtime improvements, compute/communication over

About OpenAI OpenAI is dedicated to ensuring that artificial general intelligence (AGI) benefits all of humanity. Our mission requires building not only world-class AI models, but also the infrastructure that enables those models to be deployed reliably, efficiently, and at global scale. As demand for AI continues to grow, we are expanding the ways OpenAI can bring high-performance inference capacity online across a diverse hardware ecosystem. About the Team The GPT Infrastructure team builds software that turns advanced inference and optimization research into production products. One focus is enabling strategic infrastructure partners and accelerator vendors to qualify and onboard new compute without a bespoke porting and optimization effort for every hardware platform. We build the control planes, APIs, secure partner-side execution environments, evaluation systems, artifact pipelines, and operational tooling that make these workflows repeatable and trustworthy. The work sits at the intersection of distributed systems, AI inference, compilers and runtimes, performance engineering, security, and external partnerships. About the Role We are seeking an experienced systems generalist who can work comfortably across the stack to help build an automated inference optimization platform. Given a workload, target hardware profile, compiler and runtime context, and a trusted verifier, the system runs durable optimization campaigns that generate, compile, execute, grade, and improve candidate kernels, runtime configurations, and serving-stack changes. You will design both the OpenAI-hosted control plane and the partner-side software that evaluates candidates on real accelerator hardware. The product must keep long-running workflows reliable, make performance results reproducible, and maintain clear trust boundaries around sensitive model and hardware information. This is a deeply cross-stack role, combining strong software engineering fundamentals with systems thinking and

About the Team The Applied team at OpenAI safely brings cutting-edge technology to the world. We have released widely used products such as ChatGPT, Sora, and the OpenAI API, powering models including GPT-5 and a growing set of multimodal capabilities across text, image, audio, and video. Our team also manages large-scale inference and platform infrastructure that supports these experiences at global scale. With much more on the horizon, our impact continues to grow. Our customers build fast-growing businesses using our APIs, unlocking product capabilities that were previously unimaginable. ChatGPT and Sora exemplify the breadth of what’s now possible across text, image, audio, and video experiences. As these capabilities expand, we prioritize the responsible use of our technology, emphasizing safe and thoughtful deployment over unchecked growth. Within Applied Engineering, the Ads Monetization team in Financial Engineering builds the core systems dealing with all the money flows for ChatGPT Ads. These systems are a combination of low-latency, high scale, high reliability, while being built in a financially correct, accurate, auditable and explainable way. This role sits at the intersection of ads delivery, data engineering, and financial systems. In this role, you will: Architect and build the core monetization systems for ChatGPT Ads. Build and operate the core services and pipelines that power ads monetization end-to-end, from event capture and validation through aggregation, pricing, metering, and ultimately producing billable outputs. Define and implement the source of truth for ads monetization data, including schemas, data models, and invariants that ensure outputs are consistent, explainable, and auditable. Own correctness and reconciliation: align production outputs with downstream invoicing/finance requirements, build controls/monitors, and close gaps through investigations and backfills. Develop across the stack to create comprehensive billing integration

About the team The Applied team at OpenAI safely brings cutting-edge technology to the world. We have released widely used products such as ChatGPT, Sora, and the OpenAI API, powering models including GPT-5 and a growing set of multimodal capabilities across text, image, audio, and video. Our team also manages large-scale inference and platform infrastructure that supports these experiences at global scale. With much more on the horizon, our impact continues to grow. Our customers build fast-growing businesses using our APIs, unlocking product capabilities that were previously unimaginable. ChatGPT and Sora exemplify the breadth of what’s now possible across text, image, audio, and video experiences. As these capabilities expand, we prioritize the responsible use of our technology, emphasizing safe and thoughtful deployment over unchecked growth. Within Applied Engineering, the Financial Engineering team builds the frontend platforms and workflows that power how OpenAI’s products are priced, purchased, and managed at scale. We work closely with Go-To-Market and Finance partners to translate complex commercial requirements into clear, reusable billing experiences. We’re looking for a frontend engineer to help define and evolve these billing platforms, shaping the primitives, interfaces, and patterns that product teams rely on as our offerings and customer needs continue to grow in complexity. In this role, you will: Create scalable, reusable UI components and patterns that support a wide range of billing use cases across products. Partner closely with backend engineers to deliver end-to-end billing capabilities spanning subscriptions, usage-based pricing, and enterprise offerings. Build tools and surfaces that allow teams like Finance, Sales, Support, and GTM to manage and reason about billing data efficiently. Create AI powered features for billing workflows and platform tooling to automate repetitive tasks, surface insights, and improve decision-making. Help define

About the Team OpenAI’s Hardware organization develops silicon and system-level solutions designed for the unique demands of advanced AI workloads. The team is responsible for building the next generation of AI-native silicon while working closely with software and research partners to co-design hardware tightly integrated with AI models. In addition to delivering production-grade silicon for OpenAI’s supercomputing infrastructure, the team also creates custom design tools and methodologies that accelerate innovation and enable hardware optimized specifically for AI. About the Role As an Engineer on our hardware optimization and co-design team, you will co-design future hardware from different vendors for programmability and performance. You will work with our kernel, compiler and machine learning engineers to understand their unique needs related to ML techniques, algorithms, numerical approximations, programming expressivity, and compiler optimizations. You will evangelize these constraints with various vendors to develop and influence future hardware architectures towards efficient training and inference on our models. If you are excited about efficiently distributing a large language model across devices, dealing with and optimizing system-wide/rack-wide networking bottlenecks and eventually tailoring the compute pipe and memory hierarchy of the hardware platform, simulating workloads at different abstractions and working closely with our partners, this is the perfect opportunity! This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. Key Responsibilities Co-design future hardware for programmability and performance with our hardware vendors Assist hardware vendors in developing optimal kernels and add support for it in our compiler Develop performance estimates for critical kernels for different hardware configurations and drive decisions on compute core and memory h

About the Team The Scaling team is responsible for the architectural and engineering backbone of OpenAI’s infrastructure. We design and deliver advanced systems that support the deployment and operation of cutting-edge AI models. Our work spans system software, networking, platform architecture, fleet-level monitoring, and performance optimization. About the Role We’re hiring an SW Engineer to enable production workloads and end-to-end testing on new platforms. This role will include creating new test harnesses and platform stress benchmarks, porting existing inference and training workloads to new, sometimes early-access, systems/hardware, analyzing performance and bottlenecks, and characterizing the end-to-end behavior of new systems (compute, comms, storage, control plane, and failure modes). Key Responsibilities Port and validate key inference and training workloads on new platforms/SKUs as they arrive; drive correctness, performance, and stability to an internal readiness bar. Build a suite of benchmarks and stress tests that capture real E2E behavior of our workloads by exercising all aspects of a system, including CPU, GPU, memory subsystem, frontend, scale-up, and scale-out networking (including WAN traffic, NVlink and RDMA collectives), storage, thermals, and any other relevant parts. Deep-dive performance on distributed training/inference: Collective performance and tuning (across NCCL/RCCL and internal libraries) Overlap of compute/communication, kernel-level bottlenecks, memory bandwidth and scheduling effects Create repeatable test harnesses that run in CI / lab environments and produce actionable outputs (pass/fail, performance score, regression detection). Partner with systems + fleet bring-up engineers to ensure the platform is not only stable and performant, but also operationally usable and scalable (containerization, K8s integration, telemetry hooks, failure triage loops). Work cross-functionally with vendors and internal stakeholders by producing

About the Team OpenAI’s Industrial Compute team is responsible for building and scaling large-scale compute capacity across first-party data centers, strategic partners, and industrial infrastructure environments. We focus on converting power, land, hardware, and operational execution into reliable compute capacity that can support frontier AI training and inference workloads. This team operates at the intersection of infrastructure delivery, hardware systems, utilities, supply chain, and capacity strategy—ensuring OpenAI can scale compute faster than traditional models allow. About the Role We are seeking a Tokens-as-a-Service (TaaS) Lead to drive the end-to-end conversion of industrial-scale infrastructure investments into usable token capacity for OpenAI workloads. In this role, you will own execution across complex compute programs where raw infrastructure capacity must be transformed into operational GPU throughput. You will coordinate across data center delivery, power, networking, hardware deployment, workload enablement, finance, and external partners to ensure capacity becomes productive tokens as quickly and efficiently as possible. This role is ideal for someone who can bridge physical infrastructure delivery with compute utilization outcomes. Success requires strong systems thinking, elite program leadership, and the ability to drive accountability across internal teams and strategic partners. In this role, you will Lead Tokens-as-a-Service programs across industrial compute environments, including first-party and partner-owned capacity. Convert delivered power, space, and hardware capacity into production-ready token throughput. Build integrated execution plans spanning construction, power energization, rack deployment, networking, cluster readiness, and workload onboarding. Partner with infrastructure engineering, hardware, networking, finance, supply chain, and operations teams. Drive external providers, EPCs, OEMs, utilities, and strategic partners t