Senior Data Engineer Jobs

Work Flexibility: Remote As a Senior Lead, Data Engineering, you will serve as a technical leader who helps shape the future of enterprise data solutions. In this role, you will drive complex data initiatives, influence technical strategy, and partner with teams across the organization to build scalable, high-impact data products. This is an opportunity to solve challenging business problems while mentoring fellow engineers and elevating data engineering best practices. What You Will Do Lead the architecture, development, and modernization of scalable enterprise data platforms that support global procurement analytics and business transformation. Define and help execute a multi-year data engineering strategy focused on platform scalability, reliability, automation, technical debt reduction, and long-term maintainability. Design, build, and optimize Azure-based data solutions using technologies such as Databricks, Delta Lake, Azure Data Factory, Azure DevOps, CI/CD pipelines, and infrastructure automation. Integrate and harmonize data across multiple ERP systems by standardizing supplier, purchasing, and master data into common enterprise data models. Partner with procurement analysts, architects, engineers, and business stakeholders to translate complex business needs into reusable, scalable data products and engineering solutions. Establish engineering standards, conduct architecture reviews, improve documentation, and mentor engineers to raise the overall technical capability of the team. Identify and implement AI-enabled approaches that accelerate development, improve data quality, automate documentation, support testing, and enhance analyst productivity. Evaluate and recommend tools, frameworks, patterns, and platform investments that improve performance, reliability, security, governance, and operational ef

SPECIFIC JOB RESPONSIBILITIES Pipeline Management: Maintain high-throughput streaming pipelines to ingest logs from various sources (Firewalls, Cloud, Endpoints) to a central destination. Log Normalization: Write parsers to convert raw, messy logs into standard schemas (e.g., OCSF or ECS) for consistent querying. Cost Optimization: Implement routing logic to send "high-value" data to the SIEM and "bulk" data to low-cost Object Storage (Data Lake). Data Preparation: Clean and structure data to enable AI/ML detection models and advanced analytics. EXPERIENCE REQUIRED Data Engineering: Proficiency in Python (for ETL) and SQL (for complex querying). Streaming Tech: Experience with Message Queues (e.g., Kafka, Pub/Sub) and stream processing concepts. Log Handling: Mastery of Regex and log parsing strategies for standard formats (Syslog, CEF, JSON). Storage Architecture: Understanding of Data Lake principles (Parquet/Avro formats) vs. Data Warehouses. QUALIFICATIONS, SKILLS, & KNOWLEDGE Experience with Vector Databases for storing embeddings. Knowledge of Log Observability/Routing tools (middleware that routes logs). Familiarity with Big Data frameworks (e.g., Spark, Flink). PROFESSIONAL DEVELOPMENT EXPECTATIONS Ability to embrace Clearwater's CLEAR core values (Commitment to Client Success, Lead with Accountability, Integrity & Collaboration, Excellence in All That We Do, Advance Colleague Success, Respect & Transparency) and culture. The base salary range for this role is 35,000- 45,000]. Base salary is part of our total rewards package which also includes the opportunity for merit-based salary increases, eligibility for our 401(k) plan, medical, dental, vision, life and disability insurances and leaves provided in line with your work state. Our robust time-off policy includes flexible paid time off, 11 paid holidays, and paid sick time. Total compensation, including base salary to be offered, will depend on elem

Biohub is the first large-scale initiative bringing frontier AI models, massive compute, and frontier experimental capabilities under one roof. We're building a general-purpose system to accelerate scientific discovery, integrating frontier AI models, biological foundation models, and lab capabilities, with the ultimate goal of curing disease. Our technology powers scientists around the world, translating AI capabilities into tools that accelerate research everywhere. The Team Biohub is a 501(c)(3) biomedical research organization building the first large-scale scientific initiative combining frontier AI with frontier biology to solve disease. We build the technology to help scientists around the world use AI-powered biology to study how cells operate, organize, and work as part of systems to understand why disease happens and how to correct it. With our compute capacity, AI research and engineering, and state-of-the-art technology for measuring, imaging, and programming biology, we are enabling scientists worldwide to use AI-powered biology to advance our understanding of human health. The Opportunity The role is part of the Data Engineering team, which focuses on owning the strategy, sourcing and implementation for data supporting AI research and development. Our goal is to maximize the speed, agility, and capability of biological AI research by connecting public data resources and Biohub's experimental platforms to AI systems. The data that trains biological frontier models comes in dozens of modalities (sequences, images, spatial coordinates, time series, molecular structures, metadata, publication artifacts) each with its own noise characteristics, biases, and information content. The question of how to represent this data for learning is one of the most important open problems in biological AI. As a Senior Staff Data Engineer at Biohub, you'll be designing systems that ingest data from public repositories, transform heterogeneous biological formats into AI-rea

What you’ll do Design and implement secure cloud pipelines that ingest very large scan datasets (multi-terabyte), reliably and resumably. Build orchestration for GPU-accelerated reconstruction and analysis with strong retry semantics, idempotency, and cost controls. Define end-to-end data lifecycle for medical imaging: raw vs intermediate vs derived artifacts, retention policies, and reproducibility. Implement security + compliance primitives appropriate for HIPAA/PHI: encryption in transit/at rest, key management, least privilege, audit logs, and access reviews. Build operational tooling: monitoring, alerting, runbooks, and incident-driven improvements for a growing device fleet. What we’re looking for Strong experience with cloud batch/queueing/orchestration, storage systems, and data pipeline reliability. Experience shipping production systems that handle large data volumes and failure-prone networks. Practical security mindset (least privilege, secrets, audit logging) and comfort operating in compliance-constrained environments. Useful experience Building reliable data pipelines at scale (queues/orchestration, resumable uploads, GPU batch execution) with strong observability. Security + privacy by default: encryption, least-privilege access, auditing, and practical HIPAA/PHI guardrails. Owning the “boring” backend details that keep a lean team moving: schemas/migrations, cost controls, retries, and runbooks. Understanding compute tradeoffs across hardware options, and specifying appropriate cloud resources.

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: The Data Stewardship team is a group of passionate data practitioners with a diverse background in analytics, data modeling, governance, compliance, and scaled data quality. We are responsible for ensuring Airbnb is meeting its compliance obligations across our data ecosystem and ensuring data consumers are able to easily identify the best data for their needs. We support the pipelines, programs, and policy-bodies that make this possible. You’ll be part of the overall Data Infrastructure organization that is responsible for online and offline data infrastructure across the company, and the components that transition data between these environments. The Difference You Will Make Set the North Star: you will define the multi-year vision for Data Governance and Data Quality that scales with our global business and evolving AI landscape. Organizational Influence: Act as a primary consultant for executive leadership on data governance, ensuring that compliance and stewardship are integrated into the data product lifecycle from day one. A Typical Day Architect Ecosystems: Lead the design of overarching data architectures that don't just solve today’s batch needs but anticipate future real-time and AI-driven requirements. Scale Best Practices: Rather than just "ensuring quality," you will define the best practices, tooling, and culture that enable data organizations to maintain high-quality data autonomously. Policy Stewardship: Lead cross-functional task forces (Infosec, Legal, Privacy) to navigate complex regulatory landscapes (like GDPR or AI Act) and translate them int

Discord has a highly engaged community of millions of daily active users who use the platform for many different reasons, but there’s one thing that nearly everyone does: play video games. Discord plays a uniquely important role in the future of gaming, and we are focused on making it easier and more fun for people to hang out before, during, and after playing games. Discord is looking for an experienced, proactive, and self-driven Senior Data Scientist to join our Experimentation Platform team. The Experimentation Platform puts data at the heart of the company’s decision-making and growth by enabling rapid and accurate product experimentation. Discord runs hundreds of experiments on any given day, turning experiment results into business decisions. As a Data Scientist on this team, you will ensure that the statistical underpinnings of our experiments are sound, that experimenters are able to design experiments with high rigor, and that Discord makes the best business decisions on the basis of these experiments. We are a small and quickly growing team; you will be presented with significant leadership opportunities as we evolve. Our team directly impacts the strategy and roadmaps that improve Discord for its more than 90M daily active users! If helping make Discord an even better place to hang out with your friends sounds like an exciting challenge - we'd love to chat with you! Check out our blog about how our team informs strategy & innovation at Discord here! What You'll Be Doing Collaborate closely with engineering and product teams to improve the reliability and scalability of experimentation at Discord. Provide statistical expertise, ensuring that all statistical choices, methodologies, and frameworks are sound and aligned with best practices in causal inference and experimental design. Lead initiatives to educate and train cross-functional teams on best practices in experimentation design, statistical methodologies, and causal inference, ensuring a de

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. WHY DATA SCIENCE & ANALYTICS? The Data Science & Analytics organization's mission is to increase our speed, frequency and acumen of making decisions at scale by instilling a data-influenced approach to building products. We cover a wide area of the data spectrum including analytical data engineering, product analytics, experimentation, causal inference, statistical modeling and machine learning. Aligned and partnering with product verticals, we use this extensive tool belt to discover new opportunities and unmet use cases, influence and shape the product roadmap and prioritization, build data products and measure impact on our community of players and developers. WHY DISCOVERY EXPERIENCES? Roblox is not just a platform; it is a complex, two-sided marketplace where millions of creators meet over 70 million daily active users. The Discovery Experiences team is the technical heartbeat of this ecosystem. We own the surfaces—Home, Search, Matchmaking, and Notifications—that determine how users navigate the metaverse. As a Data Scientist , you will own the ranking initiatives that power these canvases. You aren't just building models; you are building the economic and social f

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. Senior Data Analyst - Field Analytics As part of Okta’s Go-To-Market Strategy & Operations group, the Field Analytics team drives insight and scale within the global field organization. We serve as the core engine for the analytics and strategic analysis that enable our leaders to make data-driven decisions that determine the trajectory of Okta’s growth. You will be responsible for building high-impact technical assets, ranging from executive Tableau dashboards to standardized Snowflake datasets, while staying at the forefront of AI-driven analytics workflows. You will partner with US-based leadership and local team members to ensure our reporting framework is scalable, accurate, and deeply aligned with Okta’s global GTM strategy. Job Duties & Responsibilities: Analytics Delivery: Design, build, and maintain high-visibility Tableau dashboards and reporting assets that provide actionable insights to business partners across the global organization Data Set Engineering: Build and optimize production-grade data sets in Snowflake, ensuring that all field data (Pipeline, Bookings, Productivity) is clean, structured, and easily accessible for self-service analysis. Documentation & Governance: Take ownership of the technical documentation for all GTM reporting assets, ensuring data lineage, metric definitions, and logic are clearly defined and accessible. AI Implementation: Champion the use of Generative AI tools to accelerate the analytics lifec

Everpure (NYSE: P) has evolved from storage pioneer to data platform, closing fiscal 2026 with $3.7 billion in revenue, its first billion-dollar quarter, and accelerating growth into FY27. Our strategic agenda spans the companies defining the next era of technology - hyperscalers, AI labs, the AI hardware supply chain, data platform providers, and the broader AI ecosystem. This type of work—work that changes the world—is what the tech industry was founded on. So, if you're ready to seize the endless opportunities and leave your mark, come join us. The Role We are seeking a strategic Senior Analyst - Data & Analytics to champion our people analytics function and shape data-driven talent decisions. This is a high-impact, hands-on role where you will design, build, and deploy AI-augmented analytics products that drive workforce effectiveness and organizational performance. You bring deep HR domain fluency, a flair for compelling data storytelling, and proven experience turning complex workforce data into strategies that senior leaders actually act on. You will be a key partner to HRIS, IT, and global business leaders — sitting at the intersection of data engineering, business insight, and AI-powered tooling. What You'll Do Lead the full lifecycle of people analytics projects — from gathering stakeholder requirements to delivering scalable, validated, and actionable insights that guide business and talent strategy. Design, build, and deploy executive-ready analytics dashboards by leveraging AI and GenAI tools (including Claude) to rapidly prototype and implement custom web-based solutions — delivering intelligent, self-serve analytics experiences without the constraints of traditional BI platforms. Explore, prototype, and deploy AI powered analytics solutions to support workforce planning, attrition prediction, and people insights at scale. Partner with HRIS, IT, and business leaders to drive data integrity, governance, and seamless integration across core HR system

WE'RE HIRING Senior SQL Data Engineer Azure & Snowflake Remote (WFH) Immediate Joiners only Location: Remote Employment Type: Full-Time Immediate Joiners Preferred Salary: Up to 8 LPA (PF Included) Experience: 4+ Years We are looking for a highly skilled Senior SQL Developer with strong experience in Microsoft Azure and Snowflake to design, develop and optimize enterprise data solutions. Must have 4+ years as a SQL Developer, Data Engineer, or Data Warehouse Developer, The successful candidate will have excellent SQL development skills, a strong understanding of data warehousing and data modelling, and hands-on experience building cloud-based data pipelines and analytical platforms. The role involves working closely with data architects, engineers, analysts and business stakeholders to deliver reliable, scalable and high-performing data solutions. Key Responsibilities -Design, develop and maintain complex SQL queries, stored procedures, views, functions and database objects. -Must have 4+ years as a SQL Developer, Data Engineer, or Data Warehouse Developer -Build and optimise data transformation processes for large-scale datasets. Develop data pipelines using Azure Data Factory, Azure Synapse Analytics and related Azure services. -Design and implement data solutions within Snowflake. -Develop Snowflake tables, views, stored procedures, streams, tasks, stages and file formats. -Create and maintain ETL and ELT workflows across Azure, Snowflake and source systems. -Analyse and improve the performance of SQL queries, data pipelines and warehouse workloads. Design dimensional models, including star and snowflake schemas, fact -tables and dimension tables. Implement data quality checks, validation rules, reconciliation processes and error-handling frameworks. -Support data migration and modernisation programmes from on-premises platforms to Azure and Snowflake. -Develop incremental loading, Change Data Capture and Slowly Changing Dimension solutions. -Troubleshoot da

Must be based in Vancouver The role We're hiring a dedicated data engineer to own the production data platform that our delivery, product, and engineering teams run on; designing integrated, governed data pipelines and delivering automated reporting, AI-assisted workflows, and predictive signals on top of them. You'll write production code, design systems, own CI/CD, and be accountable for the correctness of data that leaders make decisions on. What you'll do Design and operate our cloud data platform: ingestion, transformation, orchestration and serving. Integrate data from across the business (delivery tooling, CRM, product telemetry, finance, support and customer feedback systems) with shared identifiers, data contracts and lineage. Build automated and continuously refreshed reporting so teams manage by exception rather than chasing status. Connect approved AI agents to governed data with structured outputs, provenance, guardrails and human approval in the loop. Build feature pipelines and the MLOps controls behind predictive use cases: tests, versioning, promotion gates and drift monitoring. Own the engineering standards for data: testing, observability, environment promotion, PII classification and access control. What you'll bring Strong software engineering fundamentals: production-quality code, API and interface design, testing discipline, systems design. Real experience building and operating production data platforms on a cloud warehouse or lakehouse (Snowflake and AWS preferred) with dbt and a modern orchestrator. Practical AI tooling experience: something shipped, not prototyped. LLM-backed classification, extraction or structured-output pipelines; agent and tool-calling workflows; retrieval; evals. You can reaso

Role Overview Build reliable software services that power products, platforms, and business decisions. As a Senior Software Developer, you’ll design and deliver scalable applications, backend services, and integrations that perform well in production and evolve with changing business needs. You’ll apply strong software engineering practices across APIs, data-intensive applications, cloud services, AI-enabled solutions, and deployment pipelines. You’ll help shape technical solutions, improve system reliability, and contribute to a high-quality engineering culture. Here’s a breakdown of what you’ll do (not all of it, just the important stuff) Design and develop scalable backend services and applications using Python or TypeScript. Lead the development of APIs, integrations, reusable software components, and AI-enabled features. Build reliable solutions for data ingestion, manipulation, service-to-service communication, and intelligent automation. Apply AI technologies and modern software engineering practices to improve product capabilities, developer productivity, and operational efficiency. Make sound technical decisions around architecture, performance, security, scalability, and maintainability. Deploy and operate applications using AWS services and CI/CD practices while improving testing, monitoring, documentation, and delivery standards. These are the essentials you’ll need to get an interview 5+ years of professional experience developing and delivering production software. Strong hands-on experience with Python; TypeScript or similar languages is also valuable. Proven experience building backend services, APIs, integrations, and service-oriented applications. Experience applying AI technologies, such as generative AI, machine learning services, intelligent automation, or AI-enabled application features. Strong understanding of software design principles, testing, debugging, performance optimization, and secure development. Experience working with cloud platfor

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. NVIDIA has a rapidly expanding ecosystem of data center platform designs. From single node HGX/DGX systems all the way up to large multi-node NVLink domain rack architectures. These designs have become core to NVIDIA's rapidly growing enterprise and cloud provider businesses. Each brings together the full power of NVIDIA GPUs, NVIDIA NVLink, NVIDIA InfiniBand networking, NVIDIA Grace CPUs, and a fully optimized NVIDIA AI and HPC software stack. We are searching for a highly motivated engineer to lead performance benchmarking and optimization efforts for our data center products. You will be instrumental in ensuring our data center solutions deliver industry-leading performance for accelerated computing workloads. What you will be doing: Design and execute comprehensive performance benchmarking strategies for our data center platforms and products Characterize real-world AI training, inference, and HPC workloads at scale Define, track, and report key performance indicators (throughput, latency, efficiency, scaling) Build automation tools and frameworks for performance monitoring and analysis Identify and analyze performance bottlenecks across compute, memory, network and storage subsystems Work closely with architecture, hardware,

Senior Cloud Data Security Engineer – Data Loss Prevention (DLP) — 18 Locations. Apply via Workday.

Data Engineering – Technical Lead About Us: Paytm is India’s leading digital payments and financial services company, which is focused on driving consumers and merchants to its platform by offering them a variety of payment use cases. To merchants, Paytm offers acquiring devices like Soundbox, EDC, QR and Payment Gateway where payment aggregation is done through PPI and also other banks’ financial instruments. To further enhance merchants’ business, Paytm offers merchants commerce services through advertising and Paytm Mini app store. Operating on this platform leverage, the company then offers credit services such as merchant loans, personal loans and BNPL, sourced by its financial partners. About the Role: This position requires someone to work on complex technical projects and closely work with peers in an innovative and fast-paced environment. For this role, we require someone with a strong product design sense & specialized in Hadoop and Spark technologies. Requirements: 4 to 8 years of experience in Big Data technologies. The position Grow our analytics capabilities with faster, more reliable tools, handling petabytes of data every day. Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability. Make changes to our diagnosing any problems across the entire technical stack. Design and develop a real-time events pipeline for Data ingestion for real-time dash- boarding.Develop complex and efficient functions to transform raw data sources into powerful, reliable components of our data lake. Design & implement new components and various emerging technologies in Hadoop Eco- System, and successful execution of various projects. Be a brand ambassador for Paytm – Stay Hungry, Stay Humble, Stay Relevant! Skills that will help you succeed in this role: Fluent with Strong hands-on experience with Hadoop, MapReduce, Hive, Spark, PySpark etc.Excellent progr