Senior Data Engineering Technical Lead Jobs

Data Engineering – Technical Lead About Us: Paytm is India’s leading digital payments and financial services company, which is focused on driving consumers and merchants to its platform by offering them a variety of payment use cases. To merchants, Paytm offers acquiring devices like Soundbox, EDC, QR and Payment Gateway where payment aggregation is done through PPI and also other banks’ financial instruments. To further enhance merchants’ business, Paytm offers merchants commerce services through advertising and Paytm Mini app store. Operating on this platform leverage, the company then offers credit services such as merchant loans, personal loans and BNPL, sourced by its financial partners. About the Role: This position requires someone to work on complex technical projects and closely work with peers in an innovative and fast-paced environment. For this role, we require someone with a strong product design sense & specialized in Hadoop and Spark technologies. Requirements: 4 to 8 years of experience in Big Data technologies. The position Grow our analytics capabilities with faster, more reliable tools, handling petabytes of data every day. Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability. Make changes to our diagnosing any problems across the entire technical stack. Design and develop a real-time events pipeline for Data ingestion for real-time dash- boarding.Develop complex and efficient functions to transform raw data sources into powerful, reliable components of our data lake. Design & implement new components and various emerging technologies in Hadoop Eco- System, and successful execution of various projects. Be a brand ambassador for Paytm – Stay Hungry, Stay Humble, Stay Relevant! Skills that will help you succeed in this role: Fluent with Strong hands-on experience with Hadoop, MapReduce, Hive, Spark, PySpark etc.Excellent progr

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: The BizTech team at Airbnb is crucial to the company's operations, handling critical data related to compliance with Tax, Payments, and Legal regulations. We also manage application data for tools such as CRM, Jira, and Workday, which are essential for Airbnb’s business. Joining this team means working with cross-functional stakeholders, designing scalable solutions, and contributing to a world-class data engineering environment with an emphasis on quality, scalability, and robust engineering practices. The Difference You Will Make: We are looking for a hands-on expert to provide technical leadership in addressing BizTech’s diverse data engineering needs and driving long-term strategies and best practices. This key leadership role requires strong collaboration and influence across teams. You'll play a crucial role in understanding business needs, identifying the right data sources, designing efficient data models, and building reliable, scalable data pipelines. As technology continues to evolve, you'll help shape and maintain significant parts of BizTech’s critical data ecosystem. Your contributions will not only address complex business challenges but also help refine and advance Airbnb’s Data Engineering Paved Path, benefiting the entire data community at Airbnb. We believe in solving problems and contributing back to our data community to continuously improve. A Typical Day /Responsibilities: Lead the design, implementation, and testing of data systems, from architecture to production. Build batch and real-time data systems that support business needs and critical products. Ensur

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: The BizTech team at Airbnb is crucial to the company's operations, handling critical data related to compliance with Tax, Payments, and Legal regulations. We also manage application data for tools such as CRM, Jira, and Workday, which are essential for Airbnb’s business. Joining this team means working with cross-functional stakeholders, designing scalable solutions, and contributing to a world-class data engineering environment with an emphasis on quality, scalability, and robust engineering practices. The Difference You Will Make: We are looking for a hands-on expert to provide technical leadership in addressing BizTech’s diverse data engineering needs and driving long-term strategies and best practices. This key leadership role requires strong collaboration and influence across teams. You'll play a crucial role in understanding business needs, identifying the right data sources, designing efficient data models, and building reliable, scalable data pipelines. As technology continues to evolve, you'll help shape and maintain significant parts of BizTech’s critical data ecosystem. Your contributions will not only address complex business challenges but also help refine and advance Airbnb’s Data Engineering Paved Path, benefiting the entire data community at Airbnb. We believe in solving problems and contributing back to our data community to continuously improve. A Typical Day /Responsibilities: Lead the design, implementation, and testing of data systems, from architecture to production. Build batch and real-time data systems that support business needs and critical products. Ensur

Work Flexibility: Remote As a Senior Lead, Data Engineering, you will serve as a technical leader who helps shape the future of enterprise data solutions. In this role, you will drive complex data initiatives, influence technical strategy, and partner with teams across the organization to build scalable, high-impact data products. This is an opportunity to solve challenging business problems while mentoring fellow engineers and elevating data engineering best practices. What You Will Do Lead the architecture, development, and modernization of scalable enterprise data platforms that support global procurement analytics and business transformation. Define and help execute a multi-year data engineering strategy focused on platform scalability, reliability, automation, technical debt reduction, and long-term maintainability. Design, build, and optimize Azure-based data solutions using technologies such as Databricks, Delta Lake, Azure Data Factory, Azure DevOps, CI/CD pipelines, and infrastructure automation. Integrate and harmonize data across multiple ERP systems by standardizing supplier, purchasing, and master data into common enterprise data models. Partner with procurement analysts, architects, engineers, and business stakeholders to translate complex business needs into reusable, scalable data products and engineering solutions. Establish engineering standards, conduct architecture reviews, improve documentation, and mentor engineers to raise the overall technical capability of the team. Identify and implement AI-enabled approaches that accelerate development, improve data quality, automate documentation, support testing, and enhance analyst productivity. Evaluate and recommend tools, frameworks, patterns, and platform investments that improve performance, reliability, security, governance, and operational ef

Discord has a highly engaged community of millions of daily active users who use the platform for many different reasons, but there’s one thing that nearly everyone does: play video games. Discord plays a uniquely important role in the future of gaming, and we are focused on making it easier and more fun for people to hang out before, during, and after playing games. Discord seeks a seasoned Data Engineer, focusing on our advertising product data. In this role, you will drive technical vision and strategy for ads data engineering in support of our ML initiatives while building and maintaining sophisticated data pipelines, datasets, and analytical tools. You will lead cross-functional initiatives to transform our advertising products through data-driven insights and mentor fellow engineers to deliver exceptional results. This role works heavily with Data Science, Machine Learning, and product. If leading technical innovation, architecting scalable solutions, and empowering teams through data excites you, we encourage you to make a move! What You'll Be Doing Design and own core ads data models : fact/dim tables, canonical datasets, and aggregation layers that power delivery, measurement, targeting, attribution, and ML use cases. Build and maintain the ML data that enables ads ranking, delivery, and targeting - including feature development, label generation workflows, intra-day training, and ML input observability to catch data quality issues before they degrade model performance. Build conversion measurement pipelines and integrate third-party attribution data - including Conversion Attribution and Mobile Measurement Partner (MMP) integrations (Adjust, AppsFlyer, Singular) - ensuring attribution accuracy and data parity across measurement surfaces. Build batch and near real-time pipeline infrastructure across the ads ecosystem - pushing toward lower-latency data for ML and reporting use cases on our BigQuery + dbt + Dagster stack. Partnering with Data Platform on lau

About Sigmoid Sigmoid Analytics is a leading Data solutions company backed by Sequoia Capital. We offer best-in-class end-to-end data value chain capabilities spanning Data Science, Data Engineering, and Data Ops. With data and technology at the core of our solutions, we are solving some of the toughest problems out there. Our culture is modelled around expertise and mutual respect with a team-first mindset. You'll work with teams that push the boundaries of what-is-possible and build solutions that energize and inspire. Offices: New York | Dallas | San Francisco | Lima | Bengaluru The below role is for our Bengaluru office. Role: Associate Director - Delivery About the Role You will lead delivery for large-scale Data Engineering programs, working closely with clients to architect and drive execution of modern data platforms - data pipelines, cloud migrations, lakehouse implementations, and real-time data processing solutions. You will own end-to-end delivery governance across multiple engagements, act as a senior escalation point for clients, and provide technical and delivery leadership to teams building enterprise-grade data engineering solutions. If you enjoy operating at the intersection of technical depth in Data Engineering and delivery leadership, we would like to talk to you. Required Skills & Competencies 15-18 years of experience in Project Delivery, Client Engagement, or Data Engineering Delivery Management, with a significant portion focused on Data Engineering programs. Strong hands-on exposure to Data Engineering tools and technologies, including cloud data platforms Proven experience in Analytics/Consulting/IT Services, managing complex Data Engineering project execution, architecture reviews, and governance. Exposure to US/Global clients, with experience handling client escalations. Ability to manage multiple Data Engineering project teams and pods, mentor Delivery/Engineering Managers and technical leads, and drive engineering process improveme

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. Enterprises are modernising data platforms and processes at a growing rate to meet the demands of their customers. A majority of this journey requires not just technical expertise but also the ability to drive predictability, and manage complexity. Snowflake’s Professional Services Organisation offers our customers a market-leading set of technical capabilities as well as best practices for modernisation based on experienced leadership. Our portfolio of modernisation solutions spans data migrations, validation, application development, data sharing, lakehouse implementations, data science, and AI. As a Solutions Architect in our team, you will be responsible for delivering exceptional outcomes for our teams and customers during our modernisation projects, with a focus on modern data architectures including lakehouses and streaming data. You will engage with customers to migrate from legacy data platforms, including those based on Cloudera, and build modern data pipelines and lakehouse solutions on Snowflake where customers can then leverage downstream AI and Machine Learning solutions. You will act as the technical lead and expert for our customers throughout this process, guiding customers through decision paths to achieve impactful outcomes and thoughtfully use AI to augm

About the Team The Applied team brings OpenAI’s technology to the world through products used by hundreds of millions of people and by developers and businesses building on our APIs. We work across research, engineering, product, policy, safety, and operations to deploy frontier AI systems responsibly and safely. The Trust & Safety Data Engineering team builds the data foundations that help OpenAI understand, detect, investigate, and mitigate abuse and safety risks across our products. We partner with Integrity, Investigations, Safety Systems, Product Policy, Privacy, Data Science, Engineering, and Data Platform to create reliable, privacy-safe datasets and pipelines for fraud and abuse detection, enforcement workflows, safety measurement, ML feature generation, launch readiness, and transparency reporting. About the Role We are hiring a Technical Lead Manager to lead and grow the Trust & Safety Data Engineering team. This is a hands-on leadership role for someone who can set strategy, shape data architecture, align senior stakeholders, coach engineers, and drive execution on high-impact data systems. You will help turn fragmented launch and incident support into durable, reusable, privacy-safe data foundations that Trust & Safety teams can rely on. The systems your team builds will help OpenAI detect risk, investigate abuse, power operational workflows, develop and evaluate safety models, measure interventions, support product launches, and report accurately on platform integrity. In This Role, You Will Lead and grow a high-performing Trust & Safety Data Engineering team. Define the roadmap and technical strategy for Trust & Safety data systems. Build canonical, privacy-safe datasets and pipelines for abuse detection, fraud detection, risk signals, enforcement, scaled review, transparency reporting, and safety monitoring. Create reusable foundations for Trust & Safety model development, including features, labels, training data, backtesting,

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Data Engineering team at Roblox plays a fundamental role in enabling the company's success by developing and maintaining highly leveraged Core Data Sets, frameworks, and tooling to support the growing demand for analytics. As a Senior Data Engineer, you will work to define the data ontology for all of Roblox, establish standard methodologies for data operations and lifecycle management, design and build analytics tooling and frameworks, and influence event instrumentation. Additionally, this role is highly multi-functional, requiring close collaboration with Data Science, Experimentation, and Machine Learning teams to understand customer requirements and analytics applications, as well as with Data Infrastructure and Storage teams to develop integrated solutions. Join us and be a part of a dynamic team driving innovation and growth at Roblox. This role will report to our Engineering Manager on the Data Engineering team. You Will: Partner with Data Science, Product, and Engineering to collect requirements to define the data ontology for all of Roblox Lead and mentor a growing team of Data Engineers to support Roblox's ever-evolving data needs Design, build, and maintain efficient an

At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done. Snowflake is about empowering enterprises to achieve their full potential — and people too. With a culture that’s all in on impact, innovation, and collaboration, Snowflake is the sweet spot for building big, moving fast, and taking technology — and careers — to the next level. Our Solution Engineering organization is seeking a Senior Data Platform Architect (Specialist) to join our Applied Field Engineering team who can provide technical leadership in working with both technical and business executives in the design and architecture of the Snowflake Cloud Data Platform as a critical component of their enterprise data architecture and overall ecosystem. In this role you will work directly with the sales team to understand the needs of our customers, strategize on how to navigate winning sales cycles, provide compelling value-based demonstrations, support enterprise Proof of Concepts, and ultimately close business. You will leverage your expertise, best practices and reference architectures highlighting Snowflake’s Data Platform capabilities across data ingestion, transformation, and lakehouse workloads. You are equally comfortable in both a business and technical context, interacting with executives and talking shop with technical audiences. IN THIS ROLE YOU WILL GET TO: Ap

NVIDIA is a global leader in high-speed computer vision, artificial intelligence (AI), and deep learning. Our team develops data engineering solutions that empower AI developers in autonomous vehicle (AV) domains to innovate quickly and effectively at scale. Are you ready to take on a senior technical role in building high-performance AI data pipelines? We seek an exceptional individual to design and optimize microservices and data pipelines to process massive volumes of AV data and enable seamless data mining and AI training. The ideal candidate will bring expertise in big data processing and distributed computing to create efficient solutions and overarching architectures for challenges such as video data curation, behavioral search, and AI dataset management. What you'll be doing: Scope and build tools, microservices, workflows, and distributed applications to accelerate data mining and AI training. Design and implement solutions for streaming, resilience, logging, security, authentication, workflow orchestration, and data management. Deploy AI models. Design and develop Retrieval-Augmented Generation (RAG) workflows enabling hybrid and agentic patterns. Analyze and operationalize complex distributed systems for speed-of-light performance. What we need to see: Experience developing high-performance, scalable software systems. MS with 6+ years, or BS (or equivalent experience) with 8+ years of relevant experience in Computer Science, Computer Engineering, or a related technical field. Strong programming skills in Python or Golang Proficiency in key technologies like Kubernetes, Helm, Hive, Parquet, SQL, vector databases, e.g., Milvus. Strong architectural skills with a proactive, problem-solving mentality. Experience in data mi

WPP is the trusted growth partner for the world’s leading brands. We unite cutting-edge media intelligence and data solutions, world-class creativity, next-generation production, transformative enterprise solutions and expert strategic counsel in a single company – powered by exceptional talent and our agentic marketing platform, WPP Open, to help our clients navigate change, capture opportunity and deliver transformational growth. We work with the world's most valuable brands and have global reach across 100+ markets, with deep local expertise. Our people are the key to our success. We're committed to fostering a culture of creativity, belonging and continuous learning, attracting and developing the brightest talent, and providing exciting career opportunities that help our people grow. For more information, visit WPP.com. Why we’re hiring We’re growing the data engineering capability behind ad-tech solutions that process large-scale, real-time audience data. In this role, you’ll take deep technical ownership of the data model that supports targeting and measurement decisions for advertisers at scale. You’ll help evolve a complex, interconnected data platform while working with modern distributed processing technologies and a highly collaborative engineering team. Who you’ll be working with You’ll work closely with data engineers and colleagues across Quality Assurance, Analytics and connector teams. We value teamwork over ego, collaboration over competition, open and honest communication, strong opinions held with curiosity, and continuous improvement through kaizen. Our technology environment Scala-based backend services Google Cloud Platform (GCP) Apache Spark for large-scale distributed data processing Parquet and BigQuery Tyda, an open-source, automation-driven software development lifecycle process What you’ll do Own and evolve a core data model spanning input and output models, field types and partitioning strate

Employee Applicant Privacy Notice Who we are: Shape a brighter financial future with us. Together with our members, we’re changing the way people think about and interact with personal finance. We’re a next-generation financial services company and national bank using innovative, mobile-first technology to help our millions of members reach their goals. The industry is going through an unprecedented transformation, and we’re at the forefront. We’re proud to come to work every day knowing that what we do has a direct impact on people’s lives, with our core values guiding us every step of the way. Join us to invest in yourself, your career, and the financial world. Role Summary: This role will lead the data engineering function supporting People Analytics, including AI-assisted workforce analytics on Snowflake. This is a player-coach role requiring hands-on technical leadership plus people leadership, with strong business partnership and the ability to balance speed, quality, governance, and innovation. What you’ll do: Manage and develop data engineers Manage, coach, and grow a team of data engineers. Set expectations for quality, collaboration, delivery, and technical ownership. Create a strong engineering culture where people solve hard problems, move quickly, and enjoy the work. Collaborate with cross-functional teams, such as other data engineers, people analysts, data scientists, and business stakeholders, to translate requirements into production-ready deliverables, and communicate technical trade-offs to non-technical partners. Stay hands on Write and review production code. Lead design reviews, code reviews, and technical problem solving. Step into critical pipelines, models, or AI workflows when needed. Build scalable People data foundations Design and maintain sustainable data models, pipelines, semantic layers, and testing frameworks. Establish team practices for documentation, lineage, data quality, and observability. Own engineering standards Set standard

NVIDIA’s DGX Cloud organization is seeking a Senior Data Engineer to become part of its data team! We develop the reliable data foundation that supports fleet health, capacity, utilization, cost, reliability, and operational decision-making throughout DGX Cloud. Our platform supports engineering, operations, finance, and product teams managing and expanding large GPU fleets across cloud service providers and NVIDIA Cloud Partners. We are looking for a practical engineer and technical lead to take charge of a key part of the Navigator data platform. We develop the systems that transform distributed infrastructure telemetry and operational data into dependable, managed data products that support fleet health, capacity, utilization, cost, and operational decisions. We are seeking a hands-on, platform-minded engineer to build and evolve the systems that turn distributed infrastructure telemetry and operational data into reliable, governed data products. You will work across ingestion, transformation, data quality, platform architecture, security, observability, and self-service consumption to help make Navigator and the DGXC data platform a dependable source of truth. We do expect strong engineering fundamentals, experience operating production systems, and the ability to learn new platforms and domains quickly. What you'll be doing: Own systems end to end. For example, work from ambiguous customer and operational needs through architecture, implementation, deployment, observability, incident response, and ongoing support. Construct data pipelines and products. Such as designing and maintain batch and streaming ingestion, transformation, reconciliation, and serving paths for fleet, capacity, utilization, cost, scheduling, and operational telemetry. Build shared libraries, workflow and DAG or equivalent experience abstractions to evolve the data platform. Develop deployment tooling, data

About Datadog: We're on a mission to build the best platform in the world for engineers to understand and scale their systems, applications, and teams. We operate at high scale—trillions of data points per day—allowing for seamless collaboration and problem-solving among Dev, Ops and Security teams globally for tens of thousands of companies. Our engineering culture values pragmatism, honesty, and simplicity to solve hard problems the right way. The Team: The Revenue Data Engineering Teams designs, builds and runs the data pipelines and helper systems to accurately and in a timely manner quantify our customers’ usage across all Datadog products. This team is at the leading edge of any new product we release. The Revenue Data Processing team builds and operates the data pipelines that does billing, and cost attribution for all Datadog products. We process terabytes of data daily to power revenue-critical systems and are at the center of every new product launch at Datadog. As a Senior Software Engineer, you will own meaningful parts of a large-scale, mission-critical processing platform — driving architectural improvements, building new billing capabilities, and maintaining the high reliability bar our downstream consumers depend on. You Will: Design and build high-throughput data pipelines for billing and cost attribution Drive platform improvements — latency reduction, Spark optimization, sharding, and cross-datacenter reliability Own root-cause investigations on billing accuracy issues in collaboration with Finance and Product teams Contribute to new billing features Work across Python and Scala, with technologies including Spark, Airflow, Trino, and Apache Iceberg Participate in on-call rotation and maintain a high reliability bar for production systems Contribute to engineering standards and help grow the technical culture of the team You Are: You have significant experience building and operating production data pipelines at scale using Spark and Airflow