Staff Infrastructure Engineer, Cluster Infrastructure

We are looking for an AI Infra engineer to join our growing team. We work with Kubernetes, Slurm, Python, C++, PyTorch, and primarily on AWS. As an AI Infrastructure Engineer, you will be partnering closely with our Inference and Research teams to build, deploy, and optimize our large-scale AI training and inference clusters. Responsibilities Design, deploy, and maintain scalable Kubernetes clusters for AI model inference and training workloads Manage and optimize Slurm-based HPC environments for distributed training of large language models Develop robust APIs and orchestration systems for both training pipelines and inference services Implement resource scheduling and job management systems across heterogeneous compute environments Benchmark system performance, diagnose bottlenecks, and implement improvements across both training and inference infrastructure Build monitoring, alerting, and observability solutions tailored to ML workloads running on Kubernetes and Slurm Respond swiftly to system outages and collaborate across teams to maintain high uptime for critical training runs and inference services Optimize cluster utilization and implement autoscaling strategies for dynamic workload demands Qualifications Strong expertise in Kubernetes administration, including custom resource definitions, operators, and cluster management Hands-on experience with Slurm workload management, including job scheduling, resource allocation, and cluster optimization Experience with deploying and managing distributed training systems at scale Deep understanding of container orchestration and distributed systems architecture High level familiarity with LLM architecture and training processes (Multi-Head Attention, Multi/Grouped-Query, distributed training strategies) Experience managing GPU clusters and optimizing compute resource utilization Required Skills Expert-level Kubernetes administration and YAML configuration management Proficiency with Slurm job scheduling, resource mana

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Anthropic is seeking talented and experienced Software Engineers to join our Continuous Integration (CI) team within the Developer Productivity organization. The CI team is responsible for the infrastructure that enables hundreds of engineers to ship code safely and efficiently to production. By joining this team, you'll have a direct impact on the velocity and reliability of Anthropic's research and product development. The CI team manages Anthropic's continuous integration system, which includes: CI infrastructure for automated testing and code quality assurance across our monorepo, designed to scale with rapid growth Test infrastructure that runs on Kubernetes clusters across multiple cloud providers, handling intelligent test selection, execution, and reporting for a large and complex test suite Merge queue management and complex branching strategies that ensure code quality at scale CI tooling and automation to improve developer productivity and reduce operational overhead Responsibilities Design and build highly reliable, scalable CI infrastructure that supports thousands of daily builds across multiple cloud providers Develop intelligent test selection systems that reduce CI time while maintaining code quality Build and improve incident response automation, including cluster load shedding, automatic recovery, and observability tooling Improve test infrastructure reliability through flake detection, quarantine systems, and test state management You may be a good fit if you Have 10+ years of relevant industry experience building and operating large-scale CI/CD systems Have deep experience with CI orchestration tool

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role We run one of the industry's largest AI compute fleets, spanning multiple cloud providers and datacenters, to train, research, and serve frontier AI models. Those fleets run on Kubernetes, and the Kubernetes Platform team owns the control plane that makes them work. We are operating at a scale where the defaults stop working. We own the scheduler and extend it to place topology-sensitive ML workloads across thousands of accelerators at once. We scale the control plane itself — apiserver, etcd, controllers — so it stays responsive as object counts and node counts grow by orders of magnitude. And we build the core cluster services every workload depends on, like service discovery, so they hold up under the same pressure. We make sure the control plane is fast, correct, and always available. Your work will directly determine whether Anthropic can keep reliably and safely training frontier models as our compute footprint continues to grow. Key responsibilities Own, operate, and extend the Kubernetes scheduler for Anthropic's accelerator fleets, including custom scheduling plugins and policies for gang scheduling, topology awareness, and preemption Scale the Kubernetes control plane (apiserver, etcd, controller-manager) to support clusters far beyond typical limits, and find the next bottleneck before it finds us Design, build, and operate core cluster services such as service discovery that every workload in the fleet depends on Build and maintain custom controllers, operators, and CRDs Partner with research, training, and inference to understand workload shapes and turn their requirements into platform capabilities Collabo

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role The Security Engineering team's mission is to safeguard our AI systems and maintain the trust of our users and society at large. Whether we're developing critical security infrastructure, building secure development practices, or partnering with our research and product teams, we are committed to operating as a world-class security organization and keeping the safety and trust of our users at the forefront of everything we do. Responsibilities: Build security for large-scale AI clusters across multiple clouds and bare-metal data centers, including IAM, network segmentation and encryption controls Design and implement network security controls: default-deny ingress and egress, segmentation between research, training and production environments, private connectivity across clouds and data centers, cloud firewall policy and mTLS with service identity Ship these controls as secure defaults using infrastructure as code and GitOps workflows, so new clusters, VPCs and data center links inherit the right boundaries Build visibility into network traffic and boundary drift, with detection for exfiltration paths and automated remediation where it is safe to do so Design secure-by-design development workflows and CI/CD pipelines across our services, with expertise in Kubernetes security, container orchestration and identity management Threat model and assess risk for complex multi cloud and networked environments, and join infrastructure networking, cluster and data center teams at design time so topology, routing and firewall decisions carry our security requirements from day one Mentor engineers and contribute to hiring and growt

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Anthropic's Infrastructure organization builds and operates the distributed systems that train, serve, and secure our AI models. That includes data pipelines, some of the largest Kubernetes clusters in the world, and the databases, observability, and developer tooling that every other team at Anthropic depends on. As a Staff Software Engineer on the Infrastructure team, you'll scope and lead complex, multi-month infrastructure projects, make the architectural decisions other engineers build on, and work with research and product teams to build systems that keep pace as their needs change. Team placement happens after the interview process, based on your interests and experience alongside organizational needs. This helps us match you with the team where you'll have the most impact. Key responsibilities Independently scope and lead complex, multi-month infrastructure projects, from an ambiguous starting point through to a production system Make architectural decisions that shape the foundation other engineers and teams build on Drive alignment on technical direction across teams, working through ambiguous problem spaces Partner with research and product teams to understand their infrastructure and compute needs, and turn them into technical designs Own the reliability, scalability, and security of the systems you build as usage and complexity grow Set technical strategy and standards for your team's infrastructure Build and improve operational processes such as incident response, postmortems, and on-call rotations, so the team learns from every incident Mentor other engineers and help raise the technical bar for the team

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role Anthropic's Infrastructure organization is foundational to our mission of developing AI systems that are reliable, interpretable, and steerable. The systems we build determine how quickly we can train new models, how reliably we can run safety experiments, and how effectively we can scale Claude to millions of users — demonstrating that safe, reliable infrastructure and frontier capabilities can go hand in hand. Node Infra owns the full lifecycle of accelerator capacity at Anthropic. We ingest and provision compute from all major cloud providers and from datacenters custom-built for Anthropic, stand up and scale the clusters behind one of the industry's largest AI compute fleets, and build the health, diagnostics and repair automation that keep every GPU, TPU and Trainium node in the fleet usable and ready to power Anthropic's frontier AI research. Key responsibilities Own the technical strategy and roadmap for node lifecycle management - ingestion, bring-up, health checking, and automated repair Drive cross-team initiatives to build and scale AI clusters across multiple clouds and accelerator families Design and operate the systems that detect, isolate, and remediate unhealthy hardware automatically, driving up fleet MTBI and minimizing stranded capacity Define infrastructure architecture, ensuring the hardest problems get solved - whether by you directly or by working through others Work closely with cloud providers and internal research/inference/product teams to shape long-term compute, data, and infrastructure strategy Establish and evolve operational excellence practices (incident response, postmortem culture, on-