AI Research Scientist - Datadog AI Research (DAIR)

As a Research Engineer on our team, you will partner with Research Scientists to turn research ideas into working systems, building the data, tooling, and infrastructure that enable rapid iteration, trustworthy evaluation, and a smooth path from prototype to production. Building on our track record of AI-powered solutions (e.g., Bits AI , Bits Evolve , and our time series foundation model ), Datadog AI Research tackles high-risk, high-reward problems grounded in real-world challenges in cloud observability and security. We are focused on two research areas: World Models for Observability -- Training multimodal foundation models that learn the joint dynamics of distributed systems across metrics, traces, logs, topology, and events. These models power advanced forecasting, anomaly detection, root cause analysis, counterfactual simulation ("what if?"), and provide a learned planning backbone for our autonomous agents. Trained Agents for Observability -- Post-training models to operate autonomously across Datadog's domain. SRE incident response is our first target, with a clear path to code repair, security response, and infrastructure optimization. We build the simulation environments, RL training loops, and evaluation infrastructure needed to train agents that match or surpass frontier models at a fraction of the cost. What You'll Do: Build and operate multimodal data pipelines, training and evaluation infrastructure, benchmarks, and internal tooling Implement models, run experiments at scale, and profile for reliability, performance, and cost Build simulation environments and replay infrastructure for agent training and evaluation Orchestrate distributed training and distributed RL with Ray, including scheduling, scaling, and failure recovery Establish rigorous automated benchmarks and regression tests for world model predictions, agent performance, and simulation fidelity Collaborate with Research Scientists, Product, and Engineeri

The Applied AI team designs and builds algorithmically driven features in the Datadog app. We work across a range of applications, primarily focusing on analysis on streaming data such as anomaly detection , error outliers and faulty deployment analysis . As an Applied Scientist you will work on building models and algorithms for machine learning powered features within the Datadog platform. You will work closely with our engineering and product partners to explore, build, scale and deliver these features that we incubate within the Applied AI team. At Datadog, we place value in our office culture - the relationships that it builds, the creativity it brings to the table, and the collaboration of being together. We operate as a hybrid workplace to ensure our employees can create a work-life harmony that best fits them. What You’ll Do: Design solutions for our different use cases. You will research and benchmark relevant algorithms to find the best fit for our use-cases Leverage machine learning algorithms and statistical techniques to build new scalable product features Develop, deploy and monitor new and existing features to production Participate in our journal club by reading and presenting the latest academic research papers to the team Explore, analyze and tell the story behind high volumes of data flowing through Datadog systems Maintain and monitor the models, services and infrastructure owned by your team Participate in your team’s on-call rotation Who You Are: You have a BS/MS/PhD in a Computer Science, Engineering, Machine Learning or related scientific field or equivalent experience You have experience working with high-scale systems and datasets including building models, applying machine learning to real business problems, and writing production data pipelines You can explain complex ideas and algorithms to non-technical audiences You care about code simplicity and performance You are excited to work on

Applied AI is where Datadog's ambitious AI bets get built and shipped ( Bits Chat , updog ). We sit at the intersection of research and product: turning promising capabilities from Datadog AI Research lab and the research community into production systems that reach real customers. The team builds specialized models that replace frontier models where they are not necessary, making AI capabilities faster, cheaper, and more secure. The mandate is to move fast from idea to customer impact, and when a product finds its footing, to set it up for growth. As a Manager I in Applied AI, you will lead a team of engineers and applied scientists working on one of these challenges. You will define technical direction, run short feedback loops, make deliberate decisions about what to pursue or stop, and work closely with product managers, research teams, and cross-functional partners to ship AI capabilities that matter. At Datadog, we place value in our office culture, the relationships and collaboration it builds and the creativity it brings. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You'll Do Lead and develop a team of engineers and applied scientists focused on cost-efficient specialized models and AI security capabilities Work closely with product managers, research teams, and cross-functional partners to shape the team's bets from initial framing through to broader adoption, with a clear definition of success criteria at each stage Own end-to-end delivery of high-quality AI systems, from early research exploration to production-grade reliability, with high standards for operational excellence, system reliability, and technical quality Navigate the unique challenges of shipping AI-powered products: balancing quality, latency, cost, and safety considerations. Drive evaluation and iteration practices for AI systems: define the quality bar and guide the team in building the offline

The AI platform is responsible for all AI infrastructure across Datadog. Our mission is to provide tools and platforms that enable data scientists and engineers to conduct large-scale training and inference with ease. We support products such as Bits AI , LLMObs and all our AI research . As an engineering manager for the Training & Serving team, you’ll join a new and fast growing team and organization. You will support building and scaling the team, define our technical vision and help shape the roadmap. Your team will lead the charge on multiple critical technical challenges: distributed training of foundation models, serving at scale, designing the user experience. You’ll work closely with sister teams in the AI platform organization ensuring a seamless AI development cycle. You’ll also partner with the Applied AI org and with Datadog infrastructure & tooling teams to build out systems from the ground up. At Datadog, we place value in our office culture - the relationships that it builds, the creativity it brings to the table, and the collaboration of being together. We operate as a hybrid workplace to ensure our employees can create a work-life harmony that best fits them. What You’ll Do: Manage and grow the Training & Serving team, directly managing 10+ engineers Define our technical roadmap in alignment with AI platform goals and the Applied AI team roadmap. Work with our core platform teams to tailor Datadog's storage, infrastructure and data pipelines to our needs Create a strong team culture aligned with our engineering standards and our customer focus Participate in hands-on work: Code reviews, design reviews and some coding Who You Are: Previous experience (1+ years) leading software engineering teams, as a tech lead or people manager Strong technician with a mix of backend, data engineer and infrastructure experience who is interested in remaining a hands-on leader Excellent leader with strong

The AI platform is responsible for all AI infrastructure across Datadog. Our mission is to provide tools and platforms that enable data scientists and engineers to conduct large-scale training and inference with ease. We support products such as Bits AI , LLMObs and all our AI research . As an engineering manager for the Evaluation & Annotation team, you’ll join a new and fast growing team and organization. You will support building and scaling the team, define our technical vision and help shape the roadmap. Your team will lead the charge on multiple critical technical challenges: AI model evaluation both offline and online, designing tooling and processes around human annotation, and establishing the standard around synthetics and AI generated datasets. You’ll work closely with sister teams in the AI platform organization ensuring a seamless AI development cycle. You’ll also partner with the Applied AI org and with Datadog infrastructure & tooling teams to build out systems from the ground up. At Datadog, we place value in our office culture - the relationships that it builds, the creativity it brings to the table, and the collaboration of being together. We operate as a hybrid workplace to ensure our employees can create a work-life harmony that best fits them. What You’ll Do: Manage and grow the Evaluation & Annotation team, directly managing 4-6 engineers Define our technical roadmap in alignment with AI platform goals and the Applied AI team roadmap. Work with our core platform teams to tailor Datadog's storage and data pipelines to our needs Create a strong team culture aligned with our engineering standards and our customer focus Participate in hands-on work: Code reviews, design reviews and some coding Who You Are: A Software Engineer at heart with a previous experience leading software engineering teams, as a tech lead or people manager Excellent leader with strong interpersonal skills, and the

Datadog’s Product Analytics suite spans Product Analytics, Session Replay, Feature Flags, and Experimentation. Together they give product teams a complete, quantitative and qualitative picture of how users experience their applications, plus the tools to release, measure, and improve those experiences with confidence. Our team of Applied Scientists makes this space smarter and more autonomous, researching, prototyping, and industrializing AI/ML capabilities that make the suite proactive and agentic by default: AI-driven event understanding and instrumentation, conversational analytics, automated insight reporting, and UI/UX issue detection from session replays. The ambition is to let product teams act with autonomy, moving from question to insight to safely released change without depending on engineering or analysts. As the Engineering Manager for this team, you’ll lead a team of Applied Scientists through an early-stage, high-impact opportunity: defining the team’s technical vision, growing its footprint, and shaping how AI capabilities get built and delivered across the suite. This is greenfield work, both in the R&D itself and in the team you’ll grow around it. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Lead and grow a team of Applied Scientists researching, prototyping, and industrializing AI/ML capabilities across the suite Define the technical vision and roadmap for the suite’s agentic and proactive AI/ML capabilities: event understanding, conversational analytics, insight reporting, and session-replay issue detection Stay hands-on as a technical contributor and reviewer, helping the team move from research prototypes to production-grade capabilities Shape how AI capabilities get built and delivered across the suite, partnering closely wi