Lead Software Engineer Data Infrastructure Jobs

Overview: The Data Acquisition team within the Foundations organization at OpenAI is responsible for all aspects of data collection to support our model training operations. Our team manages web crawling and GPTBot services and works closely with Data Processing, Architecture, and Scaling teams. We are looking for a skilled Software Engineer to join our Data Acquisition team. Responsibilities: Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search. Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability. Work closely with the legal team to handle any compliance or data privacy-related matters. Develop and deploy highly scalable distributed systems capable of handling petabytes of data. Architect and implement algorithms for data indexing and search capabilities. Build and maintain backend services for data storage, including work with key-value databases and synchronization. Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks. Conduct and analyze experiments on data to provide insights into system performance. Qualifications: BS/MS/PhD in Computer Science or a related field. 4+ years of industry experience in software development. Experience with large web crawlers a plus Strong expertise in large stateful distributed systems and data processing. Proficiency in Kubernetes, and Infrastructure-as-Code concepts. Willingness and enthusiasm for trying new approaches and technologies. Ability to handle multiple tasks and adapt to changing priorities. Strong communication skills, both written and verbal. About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an

About the Team Security is at the foundation of OpenAI’s mission to ensure that artificial general intelligence benefits all of humanity. The Security team protects OpenAI’s technology, people, and products. We are technical in what we build but operational in how we execute, and we support every product and research effort at OpenAI. Our tenets include prioritizing for impact, enabling researchers and developers, preparing for future transformative technologies, and fostering a strong, collaborative security culture. About the Role OpenAI is seeking a Principal Software Engineer to join the Infrastructure Security (InfraSec) team. InfraSec safeguards the core of OpenAI’s research and production environments: GPU supercomputing clusters, multi-cloud infrastructure, datacenters, networking, storage, and the critical services that power our frontier AI models. Our charter spans everything from bare-metal hardware and firmware to Kubernetes clusters, service meshes, and the data pathways that carry highly sensitive model weights and user data. As a Principal Software Engineer, you will set technical direction and drive execution of critical foundational services, such as authentication systems, egress/ingress proxies, access brokers, and key management platforms, that demand high standards of reliability, scalability, and software craftsmanship. These systems form the security backbone of OpenAI’s customer and supercomputing environment and must remain robust under intense scale and adversarial pressure. In this role, you will: Own the architecture and roadmap for one or more core security services (e.g., authN/Z, policy enforcement, secure proxies, key management), taking them from design to rollout to long-term operation. Design and implement planet-scale security systems that provide strong guarantees across hardware, operating systems, Kubernetes, networks, and CI/CD: balancing security, reliability, latency, and developer ergonomics. Lead cross-functional launches

About the Team We’re hiring Software Engineers to join our broader Infrastructure organization, which supports multiple high-impact teams. Depending on your interests and experience, you could work on one of several focus areas—including Core Distributed Systems, Reliability Engineering, Observability, Developer Productivity or Cloud Infrastructure. About the Role All teams are deeply collaborative, work on mission-critical services, and are responsible for building distributed, scalable infrastructure to bring OpenAI’s technology to the world through products like ChatGPT and the OpenAI API. You’ll work closely with stakeholders to understand infrastructure, data and compute needs, setting the technical strategy that supports cutting-edge research and product development. This is a critical role for someone who is passionate about solving complex engineering problems at scale, ensuring their performance, scalability and reliability Team Focus Areas Distributed Systems: Owning and building important, highly scalable, available, performant, and reliable distributed systems (and their building blocks) to power the entire stack at OpenAI Systems Engineering: Work across layers of the stack—debugging system bottlenecks, evolving core infrastructure, and solving novel problems in performance and scalability. Reliability Engineering: Build scalable, fault-tolerant systems and lead efforts around service health, incident response, and resilience. Observability: Design and maintain observability tooling (metrics, logs, tracing) to give teams visibility into production systems at scale. Developer Productivity: Create tools, environments, and workflows that help engineers ship high-quality software faster and more safely. Cloud Infrastructure: Own the cloud-native infrastructure (compute, networking, storage) that underpins all services and research workloads. Databases: Building high performance, distributed database systems that power all of OpenAI's product stack. In this

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. We are hiring a Principal Engineer II to architect the core data processing engine of the Snowflake Data & AI Cloud. At Snowflake, we believe that high-performance, unified compute fabrics are the indispensable building blocks for Agentic AI. Autonomous agents require more than just models; they require a high-fidelity, low-latency state layer to reason, act, and persist context. This role is not about building traditional data processing pipelines or legacy ETL/ELT workflows; it is about building the core distributed systems and atomic primitives that make those agentic workflows possible. In this role, you will be a lead architect of the Snowflake Data Transformation Engine. You will design and implement the fundamental transformations infrastructure—Stateful Stream Processing Engines, Incremental View Maintenance Engine, Materialization Internals, and the Distributed Orchestration Fabric. Our solid foundation supporting the seamless transition for enterprises between batch and streaming through Dynamic Tables, Streams & Tasks, and DBT Projects is the starting point. Your architectural work will extend the reach of the core engine to accelerate and support the massive scale of the Snowpark and Spark ecosystems. You are building the systems that allow both data eng

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. About the Role: This is a Principal Product Engineering role focused on Money Infrastructure at Replit. You’ll work on the financial backbone that powers how Replit earns money, how builders earn money, and how Agents transact. This role sits at the intersection of engineering, product, and the business. The systems you build directly impact revenue, trust, and some of the most critical user journeys on the platform. Getting them right enables growth, experimentation, and global scale. Getting them wrong creates broken payments, confusing pricing, and lost trust. We’re looking for engineers who can design and scale reliable financial systems while translating complex monetary logic into intuitive, user-friendly experiences for both Replit customers and builders on the platform. We love folks who have a passion for monetizing innovation and being a part of the greater pricing story. You will: Lead the design, architecture, and implementation of Replit’s core money infrastructure, spanning pricing, billing, payments, and monetization. Own and scale the global order-to-cash foundation supporting credit-based subscriptions, usage-based billing, marketplaces, in-app payments, and commerce for Agents. Enable rapid pricing and packaging experimentation across the company by building flexible abstractions and APIs for new SKUs, plans, and monetization models. Build high-converting, localized payment experiences across geographies — thinking globally while enabling users to pay locally. Power builder monetization by creating payment rails for apps, Agents, subscriptions, and new monetization primitives. Partner closely on data specifications with finance, accounting, and data teams to produce accurate, auditable, and reliable f

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: The Unified Data Store (UDS) team is the architect of Airbnb’s global system-of-record. We design, build, and operate the mission-critical storage platform that powers every user profile, listing, reservation, and financial transaction on the platform. Supporting over 150 million users worldwide, our work is the bedrock of Airbnb’s reliability and efficiency. As a Staff Engineer in our Brazil Engineering Hub , you will join a high-impact group of technical leaders who value craft and operational excellence. You won't just be managing data; you will be building a modern distributed infrastructure service that enables hundreds of product teams to ship features with total confidence. The Difference You Will Make: We are looking for a hands-on technical leader who thrives on solving deep architectural challenges and leading through ambiguity. As a Staff Engineer, you will serve as the Technical North Star for the UDS Client Stack, ensuring our data access layer is seamless, high-performance, and future-proof. A Typical Day: Define Technical Strategy: Lead the multi-year roadmap and long-term architecture for the UDS client stack, balancing immediate execution with systemic platform evolution. Architect for Scale: Design and operate a high-performance data access layer that abstracts complexities like indexing, replication, and global consistency models. Drive Engineering Excellence: Lead deep-dive design reviews and establish best practices for building fault-tolerant distributed systems across the organization. Empower Developers: Act as a bridge between infrastructure and

About the Team The Platform Analytics team builds the systems OpenAI researchers use to understand the quality and behavior of the models we train including what models are doing, why they behave in a particular way, and how that behavior changes across experiments. Neptune is a core part of this work. It ingests, stores, queries, and visualizes large volumes of metrics from pretraining, post-training, and reinforcement learning. Hundreds of researchers depend on these systems in their daily work to compare experiments, debug unexpected behavior, and decide what to try next. Our scope is broader than metrics. We also build platforms that help researchers analyze samples, traces, evaluation results, and other structured or unstructured data through dashboards, APIs, and increasingly agent-driven workflows. These systems need to remain fast, reliable, and understandable as the scale and complexity of research change quickly. We are not trying to become a consulting team that builds a separate solution for every research project. We work directly with researchers to understand recurring problems, then turn them into reusable infrastructure and platform capabilities that many teams can build on. About the Role We’re looking for a hands-on experienced software engineer who can take ownership of a critical system and drive it from problem definition through production adoption. This person should be able to own a platform such as CacheHouse end to end: define its technical direction, design its data model and storage architecture, integrate it with several research dashboards and workflows, guide one or two engineers, and ensure the system works reliably for its users. The right candidate should already bring the technical judgment, ownership, and execution expected at this level. The primary learning curve should be OpenAI’s stack and research problem space, not learning how to lead a complex engineering effort or deliver a production system. You will work directly with

About the Team Security is at the foundation of OpenAI’s mission to ensure that artificial general intelligence benefits all of humanity. The Identity Infrastructure Engineering team sits at the core of this effort, designing and building the identity and access management solutions that protect our model weights, customer data, and critical systems across multiple cloud environments. We partner with teams across OpenAI—Applied Engineering, Research, IT, and Security—to provide a secure and scalable platform for permissioning, orchestration, and innovative AI research. About the Role We’re looking for a Staff+ Software Engineer to help build and evolve the identity infrastructure that supports OpenAI’s research, engineering, and internal platforms. This role sits at the intersection of cloud infrastructure, identity systems, and software engineering. You’ll work across production systems, infrastructure-as-code, cloud control planes, identity providers, and operational infrastructure to build secure, scalable, and reliable systems used broadly across the company. The ideal candidate has experience building and operating large-scale, mission-critical systems with strong reliability and security requirements, and is comfortable writing production code, designing distributed systems, and driving ambiguous projects from 0 to 1 while building the operational rigor needed to run critical infrastructure over time. In this role, you will: Lead the architecture, development, and operation of identity infrastructure that spans cloud platforms, internal systems, and critical engineering services. Design and evolve systems for authentication, authorization, access governance, auditability, and policy enforcement with a strong focus on reliability, scalability, and secure-by-default design. Build foundational infrastructure and platform capabilities that are broadly used across engineering, research, and security teams. Improve the reliability, observability, performance, and op

Who we are About Stripe Stripe is a financial infrastructure platform for businesses. Millions of companies—from the world’s largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyone’s reach while doing the most important work of your career. About the team Product and Risk Data Engineering is Stripe's single source of truth data engineering layer for Payments, Risk, and Product — we enable Stripe to confidently run, measure, and grow the business by making accurate information easy to access. We curate and maintain high-quality data warehouses and pipelines that serve as the authoritative foundation for product and financial activity across Stripe, powering analytics, ML capabilities, agentic workflows, and merchant-facing data interfaces. Beyond building data, we act as the internal experts in data technologies and partner with Data Platform to deliver high-quality, low-friction data processing frameworks. We also serve as the bridge between data producers and data consumers — championing best-in-class data engineering practices and guiding product teams on event-driven data API modeling — so that every team at Stripe can build, decide, and grow from a trusted, well-engineered data foundation. What you’ll do We're looking for a person who could contribute to the team by solving high-impact, cutting-edge data problems. The ideal candidate will be someone that has built data pipelines for large scale volume, is deeply knowledgeable of key tools including Airflow/Spark/Kafka/Flink, is empathetic, excels at building strong relationships, and collaborates effectively with other Stripe teams to understand their use cases and unlock new capabilities. Responsibilities Lead the technical outcomes for

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . The Storage Services team operates Pinterest's scalable online structured data storage platform—supporting both SQL-based table models and complex graph data structures—managing 100TB+ datasets and serving over 1.5M queries per second across many of Pinterest's most important products. We're looking for an exceptional Staff Software Engineer to lead the technical strategy and execution of our storage infrastructure initiatives, defining how these systems are designed, built, and operated. You'll drive innovation across distributed SQL, high-throughput/low-latency query processing, graph workloads, and the developer experience for storage clients. What you’ll do: Provide technical guidance and direction to a high-performing team building reliable, performant, and cost-efficient storage systems that operate at massive scale and power busines

The worldwide data management software market is massive. At MongoDB we are transforming industries and empowering developers to build amazing apps that people use every day. We are the leading modern data platform and the first database provider to IPO in over 20 years. Join our team and be at the forefront of innovation and creativity. MongoDB is seeking a Software Engineer 3 to join the Atlas Clusters Platform team. The team is responsible for building MongoDB Atlas, our database as a service offering and fastest growing product. Atlas allows users to deploy fault-tolerant, secure, globally distributed MongoDB clusters in just minutes. The Atlas Clusters Platform team develops the foundational orchestration platform behind MongoDB Atlas. Our systems drive cluster planning and execution, evolve the Atlas control plane toward service-oriented architecture, and provide critical infrastructure that help Atlas run safely and efficiently across cloud environments. We are looking to speak to candidates who are based in New York City for our hybrid working model. What you’ll do Build and design new features for MongoDB Atlas Contribute to and lead complex technical projects Work closely with product and design teams, considering the user’s perspective while building technical solutions Work with customers and support engineers to fix issues Collaborate with team members to develop our codebase, best practices, and design principles Learn from and mentor other team members We’re looking for someone who Has at least 3 years of professional software development experience Is skilled at writing large-scale, distributed backend systems in a compiled language (Java, C#, Go, etc.) Is comfortable working across the stack of a modern web application (e.g. React, TypeScript, Kubernetes) Has experience with at least one major cloud provider technology (AWS, Azure, GCP) Has led the launch of a new feature and maintained it in production Is eager to solve tough problems Has excellent

We are a global team of innovators and pioneers dedicated to shaping the future of observability. At New Relic, we build an intelligent platform that empowers companies to thrive in an AI-first world by giving them unparalleled insight into their complex systems. As we continue to expand our global footprint, we're looking for passionate people to join our mission. If you're ready to help the world's best companies optimize their digital applications, we invite you to explore a career with us! Your Opportunity As a Senior Software Engineer within the Container Fabric (CF) organization, you will be a key driver in evolving New Relic’s global internal platform. We are looking for an operations-heavy engineer with 5–8 years of relevant experience who can leverage open-source and custom tooling to orchestrate and maintain large-scale Kubernetes environments. You will play a "Captain" role—leading critical deliverables and mentoring junior engineers while maintaining the reliability of our global fleet. What You'll Do Architectural Leadership: Drive the design and implementation of internal tools, specifically focusing on Kubernetes Operators and Controllers to automate resource management. Platform Orchestration: Lead complex, large-scale infrastructure shifts. Operational Excellence: Take ownership of incident response, author comprehensive retrospectives, and implement systemic hardening to prevent recurrence using advanced overcommit strategies. This Role Requires Experience: 5–8 years in a DevOps, Site Reliability, or Infrastructure Engineering role. Kubernetes Mastery: Deep internals knowledge of Kubernetes and hands-on experience writing custom operators. Tooling Proficiency: Strong experience building production-grade tools and services, specifically for infrastructure automation. Operations-Heavy Mindset: A proven track record of Day 1/Day 2 operations for a large-scale Kubernetes fleet, handling high-severity incidents, and improving SLA compliance through auto

Who We Are Nuro is a self-driving technology company on a mission to make autonomy accessible to all. Founded in 2016, Nuro is building the world’s most scalable driver, combining cutting-edge AI with automotive-grade hardware. Nuro licenses its core technology, the Nuro Driver™, to support a wide range of applications, from robotaxis and commercial fleets to personally owned vehicles. With technology proven over years of self-driving deployments, Nuro gives the automakers and mobility platforms a clear path to AVs at commercial scale, empowering a safer, richer, and more connected future. About the Team A rider taps "request ride" and within seconds, an autonomous vehicle is matched, dispatched, and on its way. The On-Road Experience team owns the systems that make that moment happen. The On-road Experience team builds the real-time distributed system behind Nuro's ride-hailing product. This is foundational infrastructure where correctness and performance have consequences on a real road. About the Role This is the perfect role if you love architecting performant, reliable distributed systems, processing real-time vehicle data, and building the foundational APIs that enable seamless product experiences. The position demands technical excellence, a deep understanding of system design, and a knack for solving tough problems. Come join us in defining the future! What you'll own Backend services for ride planning, vehicle matching, and real-time trip execution Partner API integrations (Uber and others) routing ride requests to Nuro vehicles Telemetry pipelines surfacing live vehicle state to riders and operations About You 4+ years of backend engineering experience with strong Go proficiency Experience designing and operating distributed systems where reliability isn't optional End-to-end product ownership — from requirements through launch and iteration Solid hands-on experience with GCP (CloudSQL, BigQuery, Redis, or equivalent) A track record of technical lead

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . Pinterest is seeking a Staff Software Engineer, Capacity Engineering. The team is responsible for efficiently managing one of the largest-scale cloud-native infrastructures in the world. This role is highly impactful, as efficiency is an ongoing strategic priority for Pinterest. The role has direct visibility across Pinterest Engineering and with Engineering and company leadership. The team is looking for a candidate with a strong background in implementing performance and efficiency projects on large scale distributed systems. In this individual-contributor role you will own and drive performance and efficiency for a core area of Capacity Engineering, partnering with the company-wide efficiency lead and collaborating with performance and efficiency leaders across the organization. What you’ll do: Drive efficiency in large-scale shared environme

Join us in building the future of finance. Our mission is to democratize finance for all. An estimated $124 trillion of assets will be inherited by younger generations in the next two decades. The largest transfer of wealth in human history. If you’re ready to be at the epicenter of this historic cultural and financial shift, keep reading. About the team + role We are building an elite team, applying frontier technologies to the world’s biggest financial problems. We’re looking for bold thinkers. Sharp problem-solvers. Builders who are wired to make an impact. Robinhood isn’t a place for complacency, it’s where ambitious people do the best work of their careers. We’re a high-performing, fast-moving team with ethics at the center of everything we do. Expectations are high, and so are the rewards. The Storage Platform team builds and operates the platform that powers database access across Robinhood. We own relational (Postgres/Aurora), key-value (DynamoDB), and caching systems, along with the SDKs, control plane automation, and data plane services that enable safe and reliable access at scale. Our mission is to standardize and strengthen how services connect to storage, improve reliability and performance, and reduce operational overhead through automation. We manage thousands of databases and hundreds of caching clusters supporting millions of users and critical brokerage workloads. Availability is our highest priority — our systems are designed to meet strict uptime targets, including no downtime during market hours. As a Staff Software Engineer , you will design and evolve the core infrastructure that underpins Robinhood’s storage systems. You’ll lead complex distributed systems initiatives such as horizontal sharding, proxy-based query routing, connection pooling, and cross-shard transactions. You’ll work on improving database reliability, performance, and cost efficiency across multi-region deployments. This role has a direct impact on system availability, laten