Staff Software Engineer, Lakeflow Pipelines DR

Job Title: Senior Staff Software Engineer - Enzyme Location: Mountain View, California --------------------------------- P-1183 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. The Lakeflow Engineering team here at Databricks is responsible for the entirety of the ETL product line. This includes products such as Materialized Views, Structured Streaming, and Delta Live Tables. We run one of the world's biggest (if not the biggest) data engineering platforms - responsible for processing exabytes of data daily for tens of thousands of customers. We're seeking a dedicated technical leader to spearhead the Materialized Views engineering team. The team is responsible for building next generation Materialized View features for both ETL workloads and for query acceleration. As part of this team, you will be working in one or more of the following areas to design and implement these next gen systems that leapfrog state-of-the-art: Incrementally maintaining materialized views Query optimization Resource management Efficient storage structures Automatic physical data optimization The Impact you will have: Solve real business needs at large scale by applying your software engineering. Deliver a highly scalable, available, and fault-tolerant architecture for materialized views Low level systems debugging, performance measurement & optimization on large production clusters. Build architecture design, influence product roadmap, and take ownership and responsibility over new projects. Introduce tools to allow greater automation and operability of services. Use your deep experience to help prevent and investigate production issues.

P-1930 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Training and customizing state-of-the-art AI models is one of the most demanding workloads in computing, and it sits at the heart of Databricks' Mosaic AI mission. AI Runtime (AIR) is our managed platform for large-scale GPU training and fine-tuning. It gives customers on-demand access to fleets of the latest accelerators and a serverless experience that hides the complexity of provisioning, scheduling, and orchestrating multi-node jobs, with the resilience to keep training running for days or weeks across thousands of GPUs. AIR powers the full spectrum of custom training, from fine-tuning open models to pre-training frontier-scale foundation models, for some of the most sophisticated AI teams in the world. As a Staff Software Engineer for AI Runtime, you will play a critical role in building and scaling the systems that make large-scale training fast, reliable, and effortless. You will drive the architecture and evolution of the managed GPU training stack, spanning scheduling and capacity, distributed training performance, fault tolerance, and the developer experience of launching and operating jobs at scale. Beyond hands-on contributions to core systems, you will help define the long-term technical vision for AIR, mentor senior engineers, partner across product, research, and platform teams, and lead the initiatives that expand the technical and business impact of custom training at Databricks. Check out our recent blog posts: How we keep GPUs reliable across Databricks AI Fast, fault-tolerant PyTorch training on AI Runtime The impact you will have: Drive the architecture and evolution of AIR'

RDQ427R351 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Search plays a foundational role in this mission. Whether through keyword-based retrieval, semantic similarity via vector embeddings, or hybrid approaches that combine both, our Search technologies help customers find, discover, and understand information across massive, complex datasets. These capabilities power everything from Retrieval Augmented Generation (RAG), AI assistants, and recommendation systems to enterprise knowledge management, in-product search, and data exploration. As a Staff Engineer for Search, you’ll play a critical role in building and scaling our next-generation Search product — spanning Vector Search, Keyword Search, and Search Quality. You’ll drive the design and evolution of a highly-performant, cost-efficient, and developer-friendly Search stack that enables customers to build AI applications with accuracy, speed, and scale. Beyond hands-on contributions, you’ll help define the long-term vision, mentor senior engineers, collaborate across teams, and lead strategic efforts that expand the technical and business impact of Search at Databricks. Check out our recent blog post: Decoupled by Design: Billion-Scale AI Search The impact you will have: Drive the architecture and evolution of the Search product, delivering scalable, low-latency, and high-quality retrieval across massive multimodal datasets. Identify and solve challenges in indexing, storage, query execution, and ranking — blending innovations in vector search, keyword search, and hybrid retrieval techniques. Partner with infrastructure, product, and research teams to shape APIs and developer experiences that

P-1126 Summary At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high-value challenges that are central to their missions. Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data and AI technology, while simultaneously operating with the resilience, security, and scale that is critical to making customers successful on our platform. Customers trust Databricks with their most valuable data and Trust & Safety has the mission to build the most trusted data analytics and ML platform in the world. Security Engineering is an integral part of Trust & Safety and has a critical role to play in keeping customer data from bad actors. We are looking for senior leaders such as yourselves to create the vision and define the strategy for this space. The impact you will have: Make Databricks safer for our customers by identifying and plugging key gaps in our infrastructure and services Attract top talent from across the industry. Represent the security engineering discipline throughout the organization, having a powerful voice to make us more data-driven Represent Databricks at academic and industry conferences & events What we look for: 7+ years of experience in Data Security or related areas and expertise in two or more of the following--Cryptography, Kubernetes Security, Web Security, Governance, Privacy, Trust, Safety, Authentication, Identity Management, Access Control, Key Management, Inter-Service Authentication, Secure Application Frameworks, Detection & Response. Experience building systems at large scale internet companies is a huge plus. 10+ years of experience building large scale distributed systems with high availability Leadersh

P-1124 Summary At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high-value challenges that are central to their missions. Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data and AI technology, while simultaneously operating with the resilience, security, and scale that is critical to making customers successful on our platform. Customers trust Databricks with their most valuable data and Trust & Safety has the mission to build the most trusted data analytics and ML platform in the world. Security Engineering is an integral part of Trust & Safety and has a critical role to play in keeping customer data from bad actors. We are looking for senior leaders such as yourselves to create the vision and define the strategy for this space. The impact you will have: Make Databricks safer for our customers by identifying and plugging key gaps in our infrastructure and services Attract top talent from across the industry. Represent the security engineering discipline throughout the organization, having a powerful voice to make us more data-driven Represent Databricks at academic and industry conferences & events What we look for: 7+ years of experience in Data Security or related areas and expertise in two or more of the following--Cryptography, Kubernetes Security, Web Security, Governance, Privacy, Trust, Safety, Authentication, Identity Management, Access Control, Key Management, Inter-Service Authentication, Secure Application Frameworks, Detection & Response. Experience building systems at large scale internet companies is a huge plus. 10+ years of experience building large scale distributed systems with high availability Leadersh

P-150 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Search plays a foundational role in this mission. Whether through keyword-based retrieval, semantic similarity via vector embeddings, or hybrid approaches that combine both, our Search technologies help customers find, discover, and understand information across massive, complex datasets. These capabilities power everything from Retrieval Augmented Generation (RAG), AI assistants, and recommendation systems to enterprise knowledge management, in-product search, and data exploration. As a Staff Software Engineer for Search Quality, you will drive the technical direction of ranking, relevance, evaluation, and quality initiatives across Databricks’ next-generation Search product. You’ll design and build the systems, models, and evaluation frameworks that ensure our Search stack delivers accurate, high-quality results across diverse multimodal datasets and query patterns. You’ll work across research, product, and infra to push the frontier of retrieval quality for enterprise AI applications — blending traditional information retrieval techniques, representation learning, and neural ranking. Beyond hands-on contributions, you will help define our long-term vision for relevance and quality, mentor senior engineers, and lead strategic efforts that raise the accuracy, reliability, and product impact of Search across Databricks. Check out our recent blog post: Reranking in Databricks AI Search for Faster, Smarter Retrieval in RAG Agents The impact you will have: Lead the technical vision for Search Quality, shaping the ranking architecture, relevance modeling stack, and evaluation systems that power Datab