Lead Data Engineer Jobs

Role: DataOps Solution Architect — Azure Data Platform (Fabric & Databricks) Owns end-to-end architecture of the Azure data platform (Fabric, Databricks, Azure services) and sets the governance, security, and DataOps standards engineering teams implement. Design authority and primary architecture advisor to the client, not the day-to-day builder. Experience Required 8-10+ years overall in data engineering / cloud architecture, including 3+ years as a Solution Architect or in a comparable design-authority role. Microsoft Fabric: 2+ years hands-on. Azure Databricks: 4+ years hands-on. Proven track record designing and delivering enterprise-scale Azure data platforms end-to-end. Azure is mandatory ; experience across multiple client engagements or a consulting/SI background is a plus. Core Technical Expertise Azure Platform Fabric & Databricks architecture. ADF. ADLS. Entra ID. Key Vault. Monitor/Log Analytics. Microsoft Fabric Workspace & environment strategy. Lakehouse/Warehouse. OneLake. Pipelines/Notebooks. REST APIs/CLI. Governance. Azure Databricks Workspace strategy. Unity Catalog. RBAC. Jobs/Workflows. Delta Lake. Asset Bundles. ML platform integration. DevOps & IaC Azure DevOps/GitHub CI/CD. Terraform & Bicep. Branching/release strategy. Reusable IaC modules. Security & Networking Entra ID. RBAC architecture. Private Endpoints/DNS. Key Vault integration. DataOps Standards CI/CD & test gates for pipelines. Data-quality frameworks. Observability/alerting architecture. SLAs/SLOs. Incident-management practices. FinOps guardrails. Key Responsibilities Own end-to-end architecture; translate requirements into scalable, secure, cost-effective designs. Design enterprise-scale Fabric/Databricks architecture including workspace, networking, security, and governance. Set standards for CI/CD, IaC, DataOps, and environment management; ensure enterprise security and regulatory compliance. Evaluate technology options; identify architectural risks a

You will lead a small, hands-on engineering team building the secure, scalable Core Analytics Data Access Platform that accelerates Datadog’s Applied AI and analytics capabilities. The team owns the Data Access Platform — a unified interface that lets AI and analytics teams discover and self-serve production-ready datasets while abstracting underlying systems and embedding required legal and compliance guardrails. In this role you’ll own technical direction, contribute to design and code, and partner closely with Applied AI, Product Analytics, and internal platform teams to provide reliable datasets and APIs for model training and analysis. This role balances day-to-day engineering leadership with long-term platform planning. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Lead a Hands-On Engineering Team: Manage, mentor, and grow a small team of 2–4 data engineers (mix of senior and junior) across Paris and NYC, fostering technical excellence and career development. Own Technical Direction and Delivery: Define architecture, engineering priorities, and the team roadmap for the Data Access Platform, driving implementation of scalable, secure data pipelines and platform services. Contribute to Design and Code: Spend substantial time coding, reviewing, and shipping critical platform components to ensure performance, reliability, and operational excellence. Partner with Internal Stakeholders: Work closely with Applied AI, Internal Product Analytics, product managers, and platform teams to define data contracts, APIs, SLAs, observability, and curated analytical datasets. Ensure Data Security, Governance, and Reliability: Implement access controls, lineage, monitoring, and compliance guardrails to support safe model training and repeatable analytics workflows.

Principal Data Privacy Architect Description - Job Summary - Role Purpose • Lead and oversee complex, cross-functional privacy and data protection programs from strategy through implementation, ensuring alignment across business, technical, legal, and compliance stakeholders. • This role will design and implement scalable, AI-ready data privacy architecture across enterprise data environments, applications, and AI-enabled workflows. • The Principal Data Privacy Architect will serve as a hands-on subject matter expert responsible for embedding privacy-by-design, consent enforcement, data sovereignty, data loss prevention, and compliance controls into large, complex global data environments. • The architect will partner closely with Data Engineering, Cybersecurity, Legal, Privacy, AI Governance, Product, and Enterprise Architecture teams to ensure customer, employee, partner, and sensitive enterprise data is accessed, processed, shared, retained, and protected in a compliant, secure, and trustworthy manner. - Why This Role Matters • Architect for Trust & Scale: Build reusable privacy architecture patterns that enable secure, compliant, and scalable data usage across platforms, products, and regions. • Enable Responsible AI: Design privacy guardrails for AI agents, generative AI, RAG pipelines, model inputs and outputs, embeddings, vector stores, and automated data workflows. • Reduce Risk While Enabling Innovation: Translate privacy, consent, regulatory, and data sovereignty obligations into practical engineering controls that accelerate business outcomes. Responsibilities - Think Customer First • Embed customer trust, transparency, and privacy-by-design principles into enterprise data platforms and customer-facing applications. • Design consent-aware data access and usage p

Become a part of our caring community Humana is seeking a Lead Cloud Architect – NoSQL Databases to provide strategic leadership, architecture direction, and engineering oversight for enterprise NoSQL database platforms across Humana’s cloud environments. This role will focus on the design, implementation, modernization, and governance of NoSQL database solutions, including MongoDB, Azure Cosmos DB, Neo4j, and vector database technologies. The successful candidate will help define and advance Humana's enterprise NoSQL strategy, support platform rationalization initiatives, and ensure database solutions are secure, scalable, resilient, automated, and aligned with enterprise architecture standards. This role requires hands-on technical depth, strong cloud architecture experience, and the ability to collaborate across security, engineering, quality, application, and business teams. Key Responsibilities Develop, document, and maintain enterprise-wide NoSQL database standards, reference architectures, design patterns, and best practices. Lead architecture and engineering efforts for NoSQL platforms including MongoDB, Azure Cosmos DB, Neo4j, and vector databases. Support NoSQL platform rationalization and modernization initiatives, including migration planning and execution from Cosmos DB to MongoDB where appropriate. Architect secure, highly available, scalable, and performant NoSQL database solutions across cloud environments, including Azure and/or Google Cloud Platform. Define database architecture patterns for document databases, graph databases, key-value workloads, and vector search use cases. Guide application teams on NoSQL data modeling, partitioning, indexing, query patterns, performance optimization, and operational readiness. Oversee automation of NoSQL provisioning, configuration, monit

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Senior Security Engineer on the Detection and Response (D&R) team at Roblox, you’ll protect our user community alongside the underlying platform infrastructure. You’ll design high-fidelity detections, engineer security data platforms, and respond alongside the team during incidents. This is a hybrid in-office role in San Mateo. You Will: Deliver robust D&R capabilities: Engineer high-fidelity detections end-to-end. Lead partners through threat modeling and logging, to deploying actionable alerts, while keeping false positives low. Build security data pipelines: Develop security data pipelines and actively contribute to internal software and data platforms, collaborating across engineering teams. Ensure service reliability: Participate in an on-call rotation to keep detection and response services healthy. Embody security culture: Serve as a trusted security partner across Roblox, helping protect our community and enterprise while fostering a culture grounded in trust, ownership, and shared responsibility. You Have: 3+ years of experience in Security Data Engineering: You have built services that are efficient, reliable, and scalable using programming languages like Golang or Py

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. Engineering Manager, Home Infrastructure The Home Infrastructure team builds the mission-critical backend and data systems that power Roblox’s Homepage and Experience Details Page, two of the highest-traffic surfaces on Roblox. These surfaces reach the vast majority of Roblox’s daily active users and are core drivers of discovery, engagement, retention, and platform growth. We are a full-stack product infrastructure team responsible for content distribution across Roblox. Our systems support multiple modes of user interaction, including exploratory browsing, directed discovery, and personalized content recommendations across the many types of content that make up the Roblox ecosystem. This team sits at the intersection of large-scale distributed systems, machine learning-powered personalization, data infrastructure, and product experimentation. We partner closely with Machine Learning, Data Science, Product, Design, Frontend, Ads, Marketplace, Virtual Economy, and other teams across Roblox to build the platforms that help users find the most relevant and engaging content. As Engineering Manager for Home Infrastructure, you will lead a team of Backend and Data Engineers responsible for the e

Opportunity Overview: This is a unique opportunity to join a high-caliber software engineering team that is experiencing rapid growth. You’ll play a key role in building impactful healthcare technology on a modern technology stack, with a focus on our core data and AI platforms. Your work will focus on enhancing the platform's key features while also balancing scalability, reusability, and performance. As a Staff Engineer on the Application Engineering team, you’ll serve as a senior technical leader - responsible for designing and delivering high-quality, scalable software systems that power Cohere Health’s core platform. You’ll act as a multiplier, elevating the technical bar for the team, mentoring engineers, and partnering with product, data, design , clinical and payment teams to deliver solutions that meet compliance, quality, and performance standards. This role is ideal for engineers who thrive on solving complex problems in healthcare, have deep expertise in building distributed systems including data solutions, and want to influence architectural decisions for security and scale, drive cross-collaborations for alignment, establish technical standards for consistency and evolve both application and data engineering best practices at scale. What you’ll do: Technical Leadership & Architecture Define and drive the architecture of large-scale, distributed application systems across the Cohere platform. Ensure solutions are secure, performant, maintainable, and compliant with NCQA, CMS, and payer requirements. Champion platform engineering best practices in CI/CD, testing, release management, and observability. Hands-On Engineering Write clean, maintainable, and well-tested code, primarily in modern frameworks (e.g., Python, TypeScript/React, Java/Kotlin). Lead the development of core features and APIs that directly impact providers, payers, and patients. Partner with DevOps and Data teams to ensure seamless integration, scalability, and operati

Opportunity Overview: We are seeking a Technical, Hands-on Manager to lead a team in building AI-driven healthcare enterprise applications . In this role, you will combine strong people leadership with deep technical expertise to guide the development of scalable, data-intensive solutions that drive meaningful business impact. As a leader who is still technically involved , you will mentor and manage data scientists and analysts while having the ability to guide the team through evaluating, selecting, and implementing the right models , paired with an understanding of end to end workflow and ensure the delivery of scalable AI solutions . This position requires excellent communication and collaboration skills as you partner closely with internal stakeholders and cross‑functional engineering, product, and clinical teams . In our fast-paced environment, adaptability is key—your ability to reprioritize quickly and lead your team through evolving business needs will ensure maximum impact What you’ll do: Lead, mentor, and develop a high‑performing team of Data Scientists and ML Engineers, ensuring strong execution and continuous skill advancement. Contribute to event‑driven architecture design and implementation, enabling asynchronous processing and large‑scale system integration. Work seamlessly across functions—partnering with Data Scientists on model tuning, experimentation, and prompt design; collaborating with Product and Software Engineering to embed AI/ML into user-facing applications; engaging with DevOps/Platform Engineering on environment setup, CI/CD, monitoring, and reliability; and working with Data Engineering on pipeline design and ingestion strategies. Provide technical leadership in the effective use of AWS services such as Lambda, EC2, EMR, S3, Athena, Batch, Textract, Comprehend, Bedrock. Drive the implementation of project scope definition, effort estimation, and planning in close coordination with cross-functional teams. Conduct code reviews, pr

We are investing in agentic AI and need a Senior AI Engineer to lead the design and delivery of these systems. This is a foundational hire: you will own both the agent-facing workstreams — pipelines, orchestration, conversational interfaces — and the underlying context layer that makes them reliable, including memory management, knowledge graph integration, and retrieval infrastructure. You will work closely with data engineers, project leads, and client stakeholders, and play a key role in shaping how Lynx builds and ships AI solutions at scale. What This Involves: Lead the architecture and delivery of agentic AI systems end-to-end: agents, orchestration, tool use, and multi-step reasoning workflows. Own the context layer: design and implement memory architectures (episodic, semantic, working memory) and integrate GraphRAG and knowledge graph retrieval into agentic pipelines. Build robust RAG systems — including vector retrieval, graph traversal, and hybrid search — and ensure retrieval quality through evaluation frameworks. Translate client requirements into technical designs, presenting approaches and trade-offs to both technical and non-technical stakeholders. Define standards and reusable patterns for agentic AI development that other engineers at Lynx can build on. Set up observability, evaluation, and monitoring pipelines to ensure AI systems perform correctly in production. Requirements: 5–8 years of software or ML engineering experience, with at least 2–3 years building LLM-based or agentic AI systems in production. Deep hands-on experience with agentic frameworks (LangChain, LlamaIndex, AutoGen, CrewAI, or similar) and LLM APIs (OpenAI, Anthropic, etc.). Strong understanding of agent design patterns: ReAct, planning loops, tool use, multi-agent coordination, and memory architectures. Practical experience with GraphRAG or knowledge graph-based retrieval (e.g., Neo4j, Microsoft GraphRAG) and vector databases (Pinecone, Weaviate, Qdrant, etc.). Proficiency in

We’re looking for a Senior Engineering Manager who is ready to lead through ambiguity and improve how software gets built at MongoDB. This role leads teams focused on developer productivity, with an emphasis on measurable improvements to the software development lifecycle. This role can be based remotely in the United States. The Team The AXIS team (AI, X-functional tools, Insights, and Signals) sits within Developer Productivity and is responsible for overseeing the metrics and observability infrastructure of our expansive developer environment to help build a strong data-driven culture. You’ll also be a key partner in building the agentic ecosystem for AI-driven development across engineering. Candidate Profile We’re looking for an experienced leader with a passion for solving the big challenge of measuring developer productivity and providing the actionable signals that help teams improve their performance. They should be comfortable working collaboratively with other leaders and partners across our Engineering and Data teams in maximizing the use of data for insights and AI enablement. The right candidate for this role will have 4+ years of experience managing software engineers, including hiring, performance management, growth planning, and compensation; required for external candidates and preferred for internal candidates 8+ years of hands-on software engineering experience building and operating production systems; experience in developer tooling, platform engineering, observability, or data engineering is a strong plus Demonstrated the ability to lead through ambiguity, work across team boundaries, and deliver outcomes without close supervision Strong customer orientation and sound judgment in finding practical, high-leverage solutions Experience working with systems involving analytics, data pipelines, and metrics platforms Experience with AI tools development and enablement efforts Strong technical judgment, including the ability to evaluate t

We’re looking for a Senior Engineering Manager who is ready to lead through ambiguity and improve how software gets built at MongoDB. This role leads teams focused on developer productivity, with an emphasis on measurable improvements to the software development lifecycle. This role can be based remotely in Canada. The Team The AXIS team (AI, X-functional tools, Insights, and Signals) sits within Developer Productivity and is responsible for overseeing the metrics and observability infrastructure of our expansive developer environment to help build a strong data-driven culture. You’ll also be a key partner in building the agentic ecosystem for AI-driven development across engineering. Candidate Profile We’re looking for an experienced leader with a passion for solving the big challenge of measuring developer productivity and providing the actionable signals that help teams improve their performance. They should be comfortable working collaboratively with other leaders and partners across our Engineering and Data teams in maximizing the use of data for insights and AI enablement. The right candidate for this role will have 4+ years of experience managing software engineers, including hiring, performance management, growth planning, and compensation; required for external candidates and preferred for internal candidates 8+ years of hands-on software engineering experience building and operating production systems; experience in developer tooling, platform engineering, observability, or data engineering is a strong plus Demonstrated the ability to lead through ambiguity, work across team boundaries, and deliver outcomes without close supervision Strong customer orientation and sound judgment in finding practical, high-leverage solutions Experience working with systems involving analytics, data pipelines, and metrics platforms Experience with AI tools development and enablement efforts Strong technical judgment, including the ability to evaluate tradeoffs, i

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. About Dynamic Tables Dynamic Tables (DTs) are Snowflake's declarative streaming transformation primitive. Customers define a SQL query and a freshness target; Snowflake handles the rest: orchestrating refreshes, maintaining snapshot consistency across a DAG of dependencies, and automatically incrementalizing the computation so that cost scales with what changed. Dynamic Tables is one of the fastest growing products at Snowflake and is a core part of Snowflake’s Data Engineering strategy. The Dynamic Tables performance team is responsible for making incremental refresh fast, predictable, and cost-efficient across increasingly complex query shapes. As a Staff Engineer on this team, you will own the technical direction for critical performance initiatives and be a force multiplier for the engineers around you. What You'll Do Lead the design and implementation of performance improvements to the incremental view maintenance engine, including multi-join incrementalization, novel incrementalization semantics, incremental window functions, and stacked operations. Help define the roadmap for the incremental view maintenance engine, identifying key performance, scalability, and correctness milestones, prioritizing high-impact enhancements, and aligning technical investments with prod

About the Role OpenAI's ads platform is experiencing rapid global scale. As the Lead Data Scientist for SMB Ads Growth, you will architect the analytics function end-to-end—driving strategy across targeting, funnel optimization, and performance forecasting. You will work directly with the SMB Ads Marketing team and your insights will be the primary catalyst for high-stakes decisions across marketing, product, and sales engineering. What You'll Do Full-Funnel Analytics Establish the foundational growth metrics and North Star KPIs for the SMB Ads ecosystem, optimizing the journey from lead acquisition to long-term retention. Diagnose funnel friction points through advanced behavioral analysis and quantify the incremental revenue impact of proposed optimizations. Partner cross-functionally to transform complex data findings into actionable, high-priority roadmaps for product and marketing stakeholders. Targeting, Segmentation & Propensity Modeling Engineer sophisticated propensity models and look-alike frameworks to identify and capture high-LTV SMB advertisers. Own the lifecycle of target list construction, including advanced data enrichment, multi-dimensional prioritization, and granular performance tracking. Develop robust segmentation architectures that power hyper-personalized outreach across paid, partnership, and outsourced (BPO) channels. Synthesize market signals to refine our value proposition, ensuring OpenAI remains a key platform for SMB business growth. Campaign Analytics & Measurement Design and implement rigorous multi-touch attribution and incrementality frameworks to evaluate channel efficacy. Lead the experimental roadmap: formulate hypotheses, execute A/B and multivariate tests, and communicate results to executive leadership. Automate business-critical reporting and dashboards to provide real-time visibility during weekly operating reviews. Forecasting & Planning Build high-fidelity revenue and advertiser growth models to project perfor

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. The Mission: Building the Data Foundations for AI We are the Snowflake Interoperable Foundations organization - the foundational layer that powers Snowflake’s AI, Analytics and Data Engineering capabilities. We lead innovations across open table formats such as Apache Iceberg, helping customers build peta-byte scale multi-cloud data lakes on Snowflake. We deliver core Metadata capabilities that power Snowflake’s industry-leading performance, AI, governance and platform features. We are embarking on a 0->1 redesign of our core systems across Interoperable Foundations. While we already manage exabyte-scale data supporting Snowflake’s AI capabilities, the next frontier is providing the foundational data layer that accelerates agentic innovation in an open, multi-format data world, You will be setting the technical vision across our investments in metadata platforms, Apache Iceberg and AI-ready storage. Your Impact: From Redesign to Reality 0->1 Architectural Leadership: Lead the ground-up redesign of our core Metadata systems, influencing the transaction frameworks that power query, DML, and AI-driven data interactions in addition to extending our lead on platform capabilities such as Zero Copy Cloning and Cross-Region / Cross-Cloud Replication. Iceberg Innovation: Drive

We’re building a world of health around every individual — shaping a more connected, convenient and compassionate health experience. At CVS Health®, you’ll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger – helping to simplify health care one person, one family and one community at a time. Position Summary The Senior Software Engineer / Technical Lead (AI & Automation) will provide technical leadership for the design, development, modernization, and support of critical applications supporting Prior Authorization Operations (PAOps) under PBM line of business. This role will be responsible for building and maintaining scalable, cloud-native solutions that enable intelligent workflow automation, AI-driven decisioning, document processing, and business process optimization. The ideal candidate is a hands-on technical leader with strong software engineering and cloud architecture expertise, coupled with practical experience implementing Generative AI, Agentic AI, and Large Language Model (LLM) solutions in production environments. This individual will collaborate closely with Data Engineering, Data Science, Product, and Business teams to deliver highly available, secure, and scalable applications while driving innovation through AI-powered solutions. Key areas of focus include: Application architecture, development, and production support Cloud-native engineering and platform modernization Microservices and distributed systems AI/GenAI, Agentic AI, and LLM-based solutions Event-driven and streaming architectures Engineering best practices, mentoring, and technical leadership R