Senior Software Engineer - Distributed Data Systems

P-1428 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Training and customizing state-of-the-art AI models is one of the most demanding workloads in computing, and it sits at the heart of Databricks' Mosaic AI mission. AI Runtime (AIR) is our managed platform for large-scale GPU training and fine-tuning. It gives customers on-demand access to fleets of the latest accelerators and a serverless experience that hides the complexity of provisioning, scheduling, and orchestrating multi-node jobs, with the resilience to keep training running for days or weeks across thousands of GPUs. AIR powers the full spectrum of custom training, from fine-tuning open models to pre-training frontier-scale foundation models, for some of the most sophisticated AI teams in the world. As a Senior Software Engineer for AI Runtime, you will play a critical role in building and scaling the systems that make large-scale training fast, reliable, and effortless. You will drive the architecture and evolution of the managed GPU training stack, spanning scheduling and capacity, distributed training performance, fault tolerance, and the developer experience of launching and operating jobs at scale. Beyond hands-on contributions to core systems, you will help shape the technical direction for AIR, mentor other engineers, partner across product, research, and platform teams, and contribute to the initiatives that expand the technical and business impact of custom training at Databricks. Check out our recent blog post: Introducing AI Runtime: Scalable, serverless NVIDIA GPUs on Databricks for training and finetuning The impact you will have: Drive the architecture and evolu

Our mission at Databricks is to radically simplify the whole data lifecycle from ingestion to ETL, BI, and all the way up to ML/AI with a unified platform. To achieve this goal, we believe the data warehouse architecture as we know it today will be replaced by a new architectural pattern, Lakehouse ( CIDR 2021 paper ), open platforms that unify data warehousing and advanced analytics. The new architecture will help address several major challenges, including data staleness, reliability, total cost of ownership, data lock-in, and limited use-case support. A critical part of realizing this vision is the next generation (decoupled) query engine and structured storage system that can outperform specialized data warehouses in relational query performance, yet retain the expressiveness and of general purpose systems such as Apache Spark™ to support diverse workloads ranging from ETL to data science. As part of this team, you will be working in one or more of the following areas to design and implement these next gen systems that leapfrog state-of-the-art: Query compilation and optimization Distributed query execution and scheduling Vectorized execution engine Data security Resource management Transaction coordination Efficient storage structures (encodings, indexes) Automatic physical data optimization What we look for: A passion for database systems, storage systems, distributed systems, language design, or performance optimization Experience working towards a multi-year vision with incremental deliverables Motivated by delivering customer value and impact 5+ years of experience working in a related system (preferred) Optional: PhD in databases or distributed systems Pay Range Transparency Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles. Actual compensation packages are based

P-78 At Databricks, we are passionate about enabling data teams to solve the world’s toughest problems, from advancing AI research to powering next-generation applications. We do this by building and operating the world’s best data and AI infrastructure platform. Founded by engineers and driven by customer obsession, we embrace the hardest technical challenges, whether it’s scaling distributed systems across multiple clouds or delivering reliable, low-latency communication between thousands of services. And we’re only getting started. As a Senior Software Engineer on the Application Traffic team , you will design and build the systems that power Databricks’ service-to-service communication across thousands of clusters in a multi-cloud environment. You will also help create abstractions that hide networking complexity from product teams, making connectivity, discovery, and reliability seamless by default. The impact you’ll have: You’ll work across three key areas that define Databricks’ networking stack: Ingress Control Plane : Build the control plane for Databricks’ global ingress layer. Enable programming of API gateways with static and dynamic endpoints, simplify service onboarding, and make it easy to expose APIs securely across clouds. Service-to-Service Communication : Design scalable mechanisms for service discovery and load balancing across thousands of clusters. Provide networking abstractions so product teams don’t need to worry about underlying connectivity details. Overload Protection : Build intelligent rate limiting and admission control systems to protect critical services under high load. Ensure reliability and predictable performance for both customer-facing and internal workloads. What we look for: BS (or higher) in Computer Science or related field 5+ years of experience designing and building large-scale distributed systems Strong proficiency in one or more languages such as Java, Scala, Go, or C++ Experience with service-oriented architectures an

P-78 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers — and customer obsessed — we leap at every opportunity to solve technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started. As a software engineer with a backend focus, you will work with your team to build infrastructure and products for the Databricks platform at scale. The impact you'll have: Our backend teams span many domains, from our core compute fabric resource management infrastructure, to service platforms, to machine learning infrastructure. For instance, you might work on challenges such as: Resource management infrastructure powering the big data and machine learning workloads on the Databricks platform in a scalable, secure, and cloud-agnostic way Develop reliable, scalable services and client libraries that work with massive amounts of data on the cloud, across geographic regions and Cloud providers Build tools to allow Databricks engineers to operate their services across different clouds and environments Build services, products and infrastructure at the intersection of machine learning and distributed systems. What we look for: BS (or higher) in Computer Science, or a related field 5+ years of production level experience in one of: Java, Scala, C++, or similar language. Experience developing large-scale distributed systems. Experience working on a SaaS platform or with Service-Oriented Architectures. Experience with cloud technologies, e.g. AWS, Azure, GCP, Docker, or Kubernetes. Experience with security and systems that handle sen

Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the world around us, that’s why we’re building a universal autonomy platform: self-driving for all roads and all rides. Founded in 2016, Nuro is a physical AI company developing Level 4 autonomous driving technology for a wide range of vehicles, use cases, and markets. Powered by the Nuro Driver™, our universal autonomy platform enables the global mobility ecosystem to deploy autonomy at scale, from robotaxis and logistics fleets to personal vehicles. With years of real-world deployment experience and a flexible, partner-led business model, Nuro is working toward a future where millions of autonomous vehicles powered by our technology help make everyday life safer, easier, and more connected. Nuro has raised over $2B in capital from Uber, NVIDIA, Google, Softbank, Fidelity, T. Rowe Price, and other leading investors. About the Team A rider taps "request ride" and within seconds, an autonomous vehicle is matched, dispatched, and on its way. The On-Road Experience team owns the systems that make that moment happen. The On-road Experience team builds the real-time distributed system behind Nuro's ride-hailing product. This is foundational infrastructure where correctness and performance have consequences on a real road. About the Role This is the perfect role if you love architecting performant, reliable distributed systems, processing real-time vehicle data, and building the foundational APIs that enable seamless product experiences. The position demands technical excellence, a deep understanding of system design, and a knack for solving tough problems. Come join us in defining the future! What you'll own Backend services for ride planning, vehicle matching, and real-time trip execution Partner API integrations (Uber and others) routing ride reque

P-1126 Summary At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high-value challenges that are central to their missions. Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data and AI technology, while simultaneously operating with the resilience, security, and scale that is critical to making customers successful on our platform. Customers trust Databricks with their most valuable data and Trust & Safety has the mission to build the most trusted data analytics and ML platform in the world. Security Engineering is an integral part of Trust & Safety and has a critical role to play in keeping customer data from bad actors. We are looking for senior leaders such as yourselves to create the vision and define the strategy for this space. The impact you will have: Make Databricks safer for our customers by identifying and plugging key gaps in our infrastructure and services Attract top talent from across the industry. Represent the security engineering discipline throughout the organization, having a powerful voice to make us more data-driven Represent Databricks at academic and industry conferences & events What we look for: 9+ years of experience in Data Security or related areas and expertise in two or more of the following--Cryptography, Kubernetes Security, Web Security, Governance, Privacy, Trust, Safety, Authentication, Identity Management, Access Control, Key Management, Inter-Service Authentication, Secure Application Frameworks, Detection & Response. Experience building systems at large scale internet companies is a huge plus. 15+ years of experience building large scale distributed systems with high availability Le