Pipeline Excellence Director Jobs

The Data Engineering team is responsible for building ETL pipelines that populate the Internal Data Platform, which drives analytics that help the company run more efficiently. Our team builds highly performant and scalable processes that extract massive datasets and makes those datasets available for querying in an optimal way. We are looking to speak to candidates who are based in Gurgaon for our hybrid working model. What you’ll do Guide the Data Engineering team on building highly performance ETL pipelines using Spark and other Big Data technologies Help design the architecture of our Internal Data Platform to support the implementation of a robust medallion architecture Provide thought leadership on ways to achieve infrastructure cost savings on Cloud hyperscalers Design and build AI agents that can help automate many of the common development and support tasks that the team performs Work with Security and Compliance teams to ensure that datasets have appropriate permissions and regulations in place Work with our Data Platform, and Governance sibling teams to make data scalable, consumable, and discoverable We’re looking for someone with 10+ years experience working on enterprise data lakes/warehouses 5+ years of Spark and Python experience 5+ years of direct hands-on experience working with AWS or GCP Thorough AI knowledge, particularly with codegen tools and agentic frameworks Hive, Iceberg, Glue, or other technologies that expose big data as tables Familiarity with different big data file types such as Parquet, Avro, and JSON Exposure to real-time or streaming data technologies is a plus Success Measures In 3 months, you'll have a thorough understanding of the architecture of MongoDB’s internal Data and AI ecosystem In 6 months, you'll have owned the delivery of a large project from start (scoping, design) to finish (delivery) In 12 months, you'll have designed new features, led development work, and become a go-to expert on parts of the system About MongoDB

We're transforming the grocery industry At Instacart, we invite the world to share love through food because we believe everyone should have access to the food they love and more time to enjoy it together. Where others see a simple need for grocery delivery, we see exciting complexity and endless opportunity to serve the varied needs of our community. We work to deliver an essential service that customers rely on to get their groceries and household goods, while also offering safe and flexible earnings opportunities to Instacart Personal Shoppers. Instacart has become a lifeline for millions of people, and we’re building the team to help push our shopping cart forward. If you’re ready to do the best work of your life, come join our table. Instacart is a Flex First team There’s no one-size fits all approach to how we do our best work. Our employees have the flexibility to choose where they do their best work—whether it’s from home, an office, or your favorite coffee shop—while staying connected and building community through regular in-person events. Learn more about our flexible approach to where we work. Overview Instacart’s Catalog Operations team powers the data that makes shopping effortless. Catalog data represents the vast majority of what customers see when they browse, search, and buy. Our quality, coverage, and accuracy directly shape the customer experience. We are an AI-first operations team that builds LLM-driven pipelines and scalable quality programs to create, enrich, and govern product data across the Instacart ecosystem. We’re looking for a Catalog Operations Analyst II to accelerate our Health and Meals pillar within Online Grocery, one of our top strategic priorities. In this role, you will drive end-to-end data quality and enrichment for health, nutrition, and meal-related attributes, partnering closely with Catalog Engineering, Data Science, Product, and ML, as well as external data providers and BPOs. You’ll join a tight-knit team of 8 who roll

About the Team OpenAI, in close collaboration with our capital partners, is building the world's most advanced AI infrastructure ecosystem. The Scaling Analytics team serves as the data backbone for this effort, enabling leaders and operators to make informed decisions across infrastructure deployment, hardware operations, supply chain, capacity planning, and site execution. As OpenAI’s Industrial Compute expands across an increasing number of global data center campuses, the complexity of managing infrastructure capacity, hardware health, supply flows, and operational performance continues to grow. Scaling Analytics develops the data models, pipelines, metrics, and reporting systems that transform fragmented operational data into actionable insights, helping OpenAI operate infrastructure at unprecedented scale. About the Role We are seeking a Data Engineer to help build and scale the analytical foundations that power OpenAI's infrastructure organization. This individual will partner closely with Hardware Operations, Capacity Planning, Supply Chain, Infrastructure Delivery, Finance, and Engineering teams to create reliable data products that support critical operational and strategic decisions. Today, much of the team's expertise is concentrated within several highly specialized domains including hardware health, GPU attribution, and supply analytics. As Stargate grows and new sites come online, the demand for analytics support continues to expand across both existing and emerging problem spaces. This role will increase the team's ability to move quickly, reduce operational bottlenecks, and provide additional depth across critical infrastructure analytics functions. The ideal candidate combines strong data engineering fundamentals with an ability to navigate ambiguous operational environments, translating complex infrastructure problems into scalable data solutions that improve visibility, decision-making, and execution. Key Responsibilities Design, build, and maint

We’re building a world of health around every individual — shaping a more connected, convenient and compassionate health experience. At CVS Health®, you’ll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger – helping to simplify health care one person, one family and one community at a time. Position Summary: We are seeking a highly skilled backend-focused Senior Software Engineer to join our modernization team focused on transforming the legacy CVS pharmacy systems into a cutting-edge, cloud-native platform. As part of the team, you will play a crucial role in designing and developing microservices that drive the modernization of critical patient and drug domains. You will work on integrating cloud-native solutions, enhancing performance, and ensuring seamless operation within a highly scalable and secure environment. The ideal candidate will have extensive experience in backend development, system design, and a strong understanding of cloud-native software engineering principles. Responsibilities: · Design, build, and maintain scalable data pipelines to support analytics, ML, and operational reporting · Develop robust data ingestion, transformation, and integration workflows using Python, SQL, and modern data engineering frameworks · Build and maintain batch and streaming data pipelines leveraging technologies such as Kafka (or similar pub/sub tools) · Work with Google Cloud Platform (GCP) services, including Cloud Storage, Dataflow, Pub/Sub, BigQuery, Cloud Spanner and Cloud Functions · Develop and manage data APIs and interfaces (REST and GraphQL) to enable high-performance data access across microservices · Implement CI/CD automation fo

Everpure (NYSE: P) has evolved from storage pioneer to data platform, closing fiscal 2026 with $3.7 billion in revenue, its first billion-dollar quarter, and accelerating growth into FY27. Our strategic agenda spans the companies defining the next era of technology - hyperscalers, AI labs, the AI hardware supply chain, data platform providers, and the broader AI ecosystem. This type of work—work that changes the world—is what the tech industry was founded on. So, if you're ready to seize the endless opportunities and leave your mark, come join us. THE ROLE Everpure is moving from an application-centric world to a data-centric one — where AI depends on trusted, well-governed, and contextualized data across the business. As part of this shift, we're looking for a Staff Analyst, AI & Automation to lead the technical build of AI and applied ML capability inside the analytics function. This is an individual contributor role — you will not manage people. Your leverage comes from what you build and ship: AI agents, applied ML models, and automated analytics workflows that change how insight gets produced, working closely with cross-functional teams to bring these to life. AI is changing by the week, not by the year. We're not looking for someone who adopts AI once it's already mainstream — we want someone who is naturally ahead of the curve: scanning what's new, testing it quickly, and bringing it into the team before it's obvious to everyone else. This role has real freedom to experiment, rethink existing approaches, and change course as the field moves. WHAT YOU'LL DO Own the technical build of AI-powered analytics tools — including RAG pipelines, LLM-assisted querying, agent workflows, and applied ML models. Our current stack includes Glean, Gemini, Claude Code and Snowflake Cortex, and we expect that to change; we care more that you pick up new tools quickly than that you already know ours. Apply data science methods (forecasting, segmentation, anomaly detection, c

Must be based in Vancouver The role We're hiring a dedicated data engineer to own the production data platform that our delivery, product, and engineering teams run on; designing integrated, governed data pipelines and delivering automated reporting, AI-assisted workflows, and predictive signals on top of them. You'll write production code, design systems, own CI/CD, and be accountable for the correctness of data that leaders make decisions on. What you'll do Design and operate our cloud data platform: ingestion, transformation, orchestration and serving. Integrate data from across the business (delivery tooling, CRM, product telemetry, finance, support and customer feedback systems) with shared identifiers, data contracts and lineage. Build automated and continuously refreshed reporting so teams manage by exception rather than chasing status. Connect approved AI agents to governed data with structured outputs, provenance, guardrails and human approval in the loop. Build feature pipelines and the MLOps controls behind predictive use cases: tests, versioning, promotion gates and drift monitoring. Own the engineering standards for data: testing, observability, environment promotion, PII classification and access control. What you'll bring Strong software engineering fundamentals: production-quality code, API and interface design, testing discipline, systems design. Real experience building and operating production data platforms on a cloud warehouse or lakehouse (Snowflake and AWS preferred) with dbt and a modern orchestrator. Practical AI tooling experience: something shipped, not prototyped. LLM-backed classification, extraction or structured-output pipelines; agent and tool-calling workflows; retrieval; evals. You can reaso

SonicWall is a cybersecurity forerunner with more than 30 years of expertise and is recognized as a leading partner-first company, ensuring our partners and their customers are never alone in the fight against cybercrime. With the ability to build, scale and manage security across the cloud, hybrid and traditional environments in real-time, SonicWall provides relentless security against the most evasive cyberattacks across endless exposure points for increasingly remote, mobile and cloud-enabled users. With its own threat research center, SonicWall can quickly and economically provide purpose-built security solutions to enable any organization—enterprise, government agencies and SMBs—around the world. For more information, visit www.sonicwall.com or follow us on Twitter , LinkedIn , Facebook and Instagram . Role Overview As our lead AI/ML Engineer , you will design, build, and scale the intelligence layer powering our next-generation security products. You will turn complex datasets—including configurations, security alerts, and raw network logs—into production-ready AI capabilities that drive automated analysis, reasoning, and intelligent recommendations. Key Responsibilities Architect AI Systems: Design and deploy robust Retrieval-Augmented Generation (RAG) pipelines, agentic workflows, and LLM applications tailored to parse and reason over security telemetry and unstructured logs. Model Development & Tuning : Train, fine-tune, and evaluate ML/DL models for pattern matching, anomaly detection, and classification across massive, heterogeneous security datasets. Ensure AI Trust & Alignment : Implement strict guardrails, evaluation frameworks, and safety alignment techniques to ensure all model outputs and recommendations are deterministic, safe, and highly accurate. Establish MLOps: Build and maintain scalable ML pipelines (from data preprocessing to model monitoring in production), collaborati

To provide technical expertise and hands-on support in the operations and maintenance of PNG pipelines, ensuring the reliable and efficient delivery of services. The Operations and Maintenance Engineer plays a critical role in executing maintenance activities, troubleshooting technical issues, and ensuring compliance with safety and regulatory standards, all while contributing to the continuous improvement of operational processes and the overall performance of the system. Source: Adani Group | Job ID: 54301

Data Engineer-6336 itD is seeking an experienced Data Engineer to build and modify high-complexity data pipelines, monitoring, and alerting systems supporting a legal-focused infrastructure team. The ideal candidate will bring deep expertise in SQL, Python, ETL/data pipelines, and data warehousing , along with a proven track record of leading complex, cross-functional data engineering initiatives from planning through delivery. Location: Remote, with preference for candidates based in California and ideally the Bay Area. Candidates open to occasional onsite work in San Francisco should indicate this in their skills matrix. Contract: 3Months Pay Rate: depending on experience and market rate. We provide comprehensive medical benefits, a 401k plan, paid holidays, and more. Please note that we are only considering direct W2 candidates at this time, as we are unable to offer sponsorship. Responsibilities Build and modify high-complexity ETL and data pipelines to efficiently extract, transform, and process data from large-scale datasets. Develop and maintain monitoring and alerting systems that proactively identify data, infrastructure, and workflow issues. Design data warehousing solutions and optimize queries and pipelines for large-scale data processing. Collaborate with attorneys, data scientists, engineers, and other technical and non-technical stakeholders to understand data requirements, sources, dependencies, and timelines. Lead end-to-end data engineering initiatives across cross-functional teams, including planning, documentation, stakeholder coordination, system design, and execution. Identify potential data and system issues early and implement solutions that improve reliability, efficiency, and operational visibility. Communicate technical concepts and project status clearly and concisely to both technical and non-technical audiences. Internal Responsibilities Attend regular internal practice community meetings. Collaborate with your itD practice team on indu

WPP is the trusted growth partner for the world’s leading brands. We unite cutting-edge media intelligence and data solutions, world-class creativity, next-generation production, transformative enterprise solutions and expert strategic counsel in a single company – powered by exceptional talent and our agentic marketing platform, WPP Open, to help our clients navigate change, capture opportunity and deliver transformational growth. We work with the world's most valuable brands and have global reach across 100+ markets, with deep local expertise. Our people are the key to our success. We're committed to fostering a culture of creativity, belonging and continuous learning, attracting and developing the brightest talent, and providing exciting career opportunities that help our people grow. For more information, visit WPP.com. Why we're hiring: As a Data Engineer in the WPP Enterprise Data Group, you will be responsible for the design and implementation of scalable data solutions providing enterprise-scale data transformation across a broad range of projects. Your role will focus on delivering solutions that utilize large-scale data ingestion, processing, storage/querying, streaming, and batch analytics using Databricks . As part of a team, you will implement world-class solutions designed by our data architects. Your responsibilities will include estimating, designing, coding, testing, deploying, and ensuring scalability and performance on Azure using key technologies like Databricks. As a hands-on technologist with an extensive data engineering background using Databricks , you will be joining a group of Data Engineers who are passionate about building the best possible solutions for our business and endorse a culture of life-long learning and collaboration. What you'll be doing: Design, build, test and maintain data pipelines (ELT), according to business and technical requirements. Implement secure pla

WPP is the trusted growth partner for the world’s leading brands. We unite cutting-edge media intelligence and data solutions, world-class creativity, next-generation production, transformative enterprise solutions and expert strategic counsel in a single company – powered by exceptional talent and our agentic marketing platform, WPP Open, to help our clients navigate change, capture opportunity and deliver transformational growth. We work with the world's most valuable brands and have global reach across 100+ markets, with deep local expertise. Our people are the key to our success. We're committed to fostering a culture of creativity, belonging and continuous learning, attracting and developing the brightest talent, and providing exciting career opportunities that help our people grow. For more information, visit WPP.com. Why we're hiring: As a Data Engineer in the WPP Enterprise Data Group, you will be responsible for the design and implementation of scalable data solutions providing enterprise-scale data transformation across a broad range of projects. Your role will focus on delivering solutions that utilize large-scale data ingestion, processing, storage/querying, streaming, and batch analytics using Databricks . As part of a team, you will implement world-class solutions designed by our data architects. Your responsibilities will include estimating, designing, coding, testing, deploying, and ensuring scalability and performance on Azure using key technologies like Databricks. As a hands-on technologist with an extensive data engineering background using Databricks , you will be joining a group of Data Engineers who are passionate about building the best possible solutions for our business and endorse a culture of life-long learning and collaboration. What you'll be doing: Design, build, test and maintain data pipelines (ELT), according to business and technical requirements. Implement secure platforms

The Team + The Role Our Emerging Team is focused on building AI Products for our product experience (PX) platform. We build from the ground up to explore, prototype, and ship AI-native experiences that change how software teams understand and serve their users. This is not an AI layer added to existing product; it is a deliberate bet on what product intelligence looks like next. The team operates with high autonomy, moves quickly, and builds products without clear precedents. As a Staff Software Engineer (AI), you will sit at the intersection of deep technical capability and strong product judgment. You will design and build production-grade AI systems, including RAG pipelines, agentic workflows, and LLM-powered features, while making clear tradeoffs across prompting, fine-tuning, architecture, evaluation, and deployment. You will also partner closely with product, design, and engineering stakeholders to frame the right problems and communicate technical decisions clearly. This role is based in our New York office. What this looks like day-to-day Applied AI systems: Design and build AI-native systems, including RAG pipelines, agentic workflows, and LLM-powered product features. You will take ideas from prototype through production and ensure they can support real users. Model strategy: Make principled decisions about when to prompt, when to fine-tune, and when to use a different technical approach entirely. You will explain those tradeoffs clearly to engineers and non-engineers. Evaluation and guardrails: Instrument and evaluate model outputs rigorously by defining evaluation frameworks and identifying hallucinations early. You will implement guardrails that hold up under real-world usage and load. Productionize AI ownership: Own model deployment, monitoring, latency optimization, cost management, and reliability at scale. You will ensure AI systems are observable, performant, and production-ready. Full-stack delivery: Contribute across the stack when needed to get

The Team + The Role Our Emerging Team is focused on building AI Products for our product experience (PX) platform. We build from the ground up to explore, prototype, and ship AI-native experiences that change how software teams understand and serve their users. This is not an AI layer added to existing product; it is a deliberate bet on what product intelligence looks like next. The team operates with high autonomy, moves quickly, and builds products without clear precedents. As a Sr. Software Engineer (AI), you will sit at the intersection of deep technical capability and strong product judgment. You will design and ship applied AI systems, including RAG pipelines, agentic workflows, and LLM-powered features, from prototype through production. You will make principled technical decisions, evaluate model behavior rigorously, and communicate tradeoffs clearly to engineers and non-engineers alike. This role is based in our New York office. What this looks like day-to-day Applied AI systems: Design and build AI systems including RAG pipelines, agentic workflows, and LLM-powered features. You will take work from prototype through production and ensure it can hold up in real customer environments. Technical decision-making: Make principled decisions on when to prompt, when to fine-tune, and when to use a different tool entirely. You will explain these tradeoffs clearly so the team can move quickly without sacrificing quality. Model evaluation: Instrument and evaluate model outputs rigorously by defining evaluation frameworks and catching hallucinations early. You will implement guardrails that can withstand real-world load and production use. Productionize AI ownership: Own model deployment, monitoring, latency optimization, cost management, and reliability at scale. You will help ensure AI systems are observable, efficient, and dependable in production. Full-stack product shipping: Contribute across the stack when needed because this team ships products, not just models

Who we are About Stripe Stripe is a financial infrastructure platform for businesses. Millions of companies—from the world's largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyone's reach while doing the most important work of your career. About the team Web Presence and Platform is organized into two pillars, each of which is grouped into pods that focus on the central tenets of the Stripe public mission. The Presence pillar creates industry-leading designs for Stripe front door surfaces, educating customers about the power of our platform, sharing ideas and expertise with the public, and driving adoption. The Platform pillar builds the internal machinery that powers these surfaces, and is responsible for making our websites fast, stable, and easy to update. Together we design and build stripe.com and other sites that amount to what is, for many, their first impression of Stripe. As such, WPP offers exciting opportunities to have a major impact on Stripe success. We want to make every pixel count, we want it to be enthralling, and we want to help other Stripes seamlessly benefit from our systematic work. While the team's mission anchors to web surfaces, this role lives deeper in the stack. You'll be the engine behind the experiences rather than the pixel-level presentation—APIs, data pipelines, model orchestration, and service reliability that make LLM-driven user experiences possible at scale. What you'll do The Expansion pod brings creativity and executional rigor to attracting new prospects and driving conversion, focusing on user journeys, interactive and highly polished tools for users, and building targeted experiences for our global users. As a full stack engineer, you'll architect a

About the Team The Release Engineer team is responsible for building and maintaining the systems that power software delivery—from CI/CD pipelines and artifact management to release automation and fleet telemetry. We ensure software across bootloaders, firmware, operating systems, and cloud services is built reproducibly, validated rigorously, and released safely at scale. About the Role As a Release Engineer, you’ll design, build, and operate release infrastructure that enables reliable, secure, and traceable software delivery across complex multi-component systems. You’ll partner closely with embedded, cloud, and QA teams to ensure that every build—from development to OTA deployment—is fast, verifiable, and production-ready. We’re looking for engineers who take pride in automation, build reproducibility, and system reliability—and who enjoy building the connective tissue that allows hardware and software to ship together seamlessly. This role is based in San Francisco, CA. We use a hybrid work model of four days in the office per week and offer relocation assistance to new employees. In this role, you will: Design and operate CI/CD pipelines for multi-component builds (bootloader, firmware, OS images, backend, companion apps) using hermetic toolchains. Define versioning and branching strategies; automate promotions, changelogs, and artifact retention. Integrate unit, integration, and hardware-in-the-loop (HIL) test results; quarantine flaky tests, auto-bisect failures, and block unsafe promotions. Build A/B OTA update flows with verity and health checks; run staged rollouts and canaries; implement safe rollback and roll-forward strategies. Implement code signing for binaries and firmware, generate SBOMs, run vulnerability scanning, and attach build attestations and provenance. Manage dashboards and alerts for build health, promotion latency, failure rates, and fleet update telemetry. You might thrive in this role if you: Have experience building and operating buil