Senior Software Engineer - Autonomous Ops & Maintenance (ATOM)

Datadog’s Chaos Engineering team builds systems that surface reliability weaknesses before they become outages. As a Senior Software Engineer, you will initially focus on zonal resilience, building automation that helps services safely evacuate and recover from zonal failures, while also contributing to fault injection, incident replay, gameday orchestration, and reliability tooling. You will work across engineering teams to design systems that safely exercise production failure modes and turn findings into verified remediation. You will also help advance the use of AI and automation to identify, test, and close resilience gaps as Datadog’s software and infrastructure evolve. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Build zonal-resilience automation that coordinates safe workload evacuations, switchovers, and recovery in partnership with the teams that own affected services. Design and build fault-injection systems for production environments, including infrastructure- and application-level testing, incident replay, and controlled resilience experiments. Develop safeguards such as blast-radius controls, kill switches, validation mechanisms, and rollback paths that keep production experiments contained and reversible. Build agents and automation that help propose failure scenarios, triage experiment results, and connect reliability findings to tracked remediation and verification. Lead gamedays from hypothesis and scenario design through execution, documented findings, remediation tracking, and validation of completed fixes. Design and implement reliable distributed systems, including gRPC services, Kubernetes controllers, and shared platform components, while contributing to technical design and mentoring other engineers. Who You Are: You have str

The Incident Insights & Readiness SRE team at Datadog fosters a resilient culture by using incidents as learning opportunities and catalysts for growth. Our users are Datadog engineers, and we build the software, tooling, and operational frameworks that help them prepare for, respond to, and learn from incidents. We work closely with engineering teams across Datadog to analyze incidents and turn those insights into better tools, stronger incident response, and organizational learning. Our efforts empower Datadog to navigate unexpected failures confidently, efficiently, and with a commitment to continuous learning and systems improvement. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Own and improve the on-call experience for the company by establishing best practices and building platforms to support on-call rotations and compensation. Define how we respond to incidents, lead the design and implementation of software to streamline the process, and collaborate with product teams to improve incident response across Datadog. Our aim is to fully support our incident responders in dealing with complexity. Contribute to the post-mortem process for the company, collaborating with teams on writing them, and identifying opportunities to reduce friction and enhance learning value for the organization. Our team also runs a weekly postmortem reading group. Support various teams in facilitating incident reviews that emphasize learning and blamelessness. Help them share their learnings across the organization to improve the resilience of our people. Provide technical leadership and day-to-day coaching to team members, accelerating their growth through design reviews, collaborative problem-solving and operational excellence best practices. Tra

We are looking for a strong technical leader to join the Private Action Runner team, part of the larger Action Platform group and help shape one of the core execution layers behind Datadog’s action-taking and remediation capabilities. Private Action Runner (PAR) enables Datadog products and AI agents to securely run actions inside customer infrastructure with controls for authentication, permissions, auditing and safe execution. The role will be hands-on, covering architecture, implementation, reliability and collaboration with teams integrating PAR across Datadog. It also offers leadership exposure through leading a team of 3 engineers, with the expectation that the role will quickly transition into a formal Engineering Manager 1 position as the team grows. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: (Describe role responsibilities here) Solve a scaling bottleneck in a critical service Deploy a new feature to production, progressively rolling it out with feature flags Investigate and fix a production issue from a service your team owns Design a way to scale up a service for more traffic With your team, plan the most important projects to work on next Mentor and lead a small team of 3 engineers Who You Are: (Describe role qualifications here) You have significant experience in one or more languages You value code simplicity and performance You can design architecture to solve problems at high scale You have a BS/MS/PhD in a scientific field or equivalent experience You want to work in a fast, high-growth startup environment that respects its engineers and customers You’re excited about leveraging AI tools to enhance how you code, solve problems, and build – or eager to learn how You have demonstrated ability to use AI

As a Senior Security Engineer within Application Security at Datadog, you will help secure emerging application architectures where established security patterns may not yet exist, including agentic applications, LLM-enabled products, protocol-driven tool use, and untrusted execution environments. You will establish and validate security models, requirements, and controls while partnering with product and platform teams to develop secure-by-default approaches. Your work will address evolving trust boundaries, agent identity, data access, tool permissions, and runtime behavior through threat modeling, security solutions, and practical implementation. You will have the opportunity to shape how Datadog safely supports new application paradigms and address emerging security risks before they become systemic. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Define security models, requirements, and guardrails for emerging application architectures where established patterns are incomplete or do not yet exist. Design, implement, and validate practical security controls that reduce risk in emerging architectures and high-risk system behaviors. Shape how emerging systems authenticate, authorize actions, isolate workloads, and access data, applying least-privilege principles throughout. Partner with product, platform, Infrastructure Security, and other engineering teams to identify security gaps and embed secure-by-default requirements from early design through production. Lead threat modeling and security assessments for new and existing products, focusing on evolving threats, unfamiliar trust boundaries, and new interaction patterns. Prioritize and drive remediation of emerging security risks, including continuous security testing where it provides durable cover

Husky is what we call the distributed, petabyte-scale columnar event store at the heart of our Event Platform, which powers dozens of Datadog’s most popular products – Logs, RUM, APM , Cloud Network Monitoring, Netflow, and many more. Husky was built from the ground up at Datadog to store and query massive volumes of event data at low cost, with data fully queryable within seconds of arrival. As an Engineering Manager on Husky, you will lead one of a few closely collaborating teams that work directly on Husky’s internals. We own the full lifecycle of events stored by our Event Platform – whether it’s managing the ingestion of over 100 million events per second exactly-once , optimizing the persistence layer with lightning-fast compaction , tweaking our columnar format, timely deletion across hundreds of thousands of customer tables, or building the metadata service that enables hundreds of thousands of queries per second – you will be responsible for the growth and success of a team constantly meeting new scalability requirements driven by Datadog’s growing customer base and suite of products. At Datadog, we place value in our office culture - the relationships that it builds, the creativity it brings to the table, and the collaboration of being together. We operate as a hybrid workplace to ensure our employees can create a work-life harmony that best fits them. What You’ll Do: Manage a team of software engineers ranging from new grads to senior engineers, focusing on career development, inclusivity, and high impact Be a technical leader of the team; while the role may not involve frequent hands on coding, you’ll be reviewing architecture decisions, RFCs, and pull requests and ensuring what the team ships is high quality Partner with product teams to evaluate use cases, ensure smooth implementation, and prioritize new platform capabilities. Share on-call responsibilities with the rest of the team and ensure a culture of operational exc

The Behavior AI team builds the AI-based anomaly detection behind Datadog's security products. Our models learn what normal looks like across the billions of logs, events, and telemetry records flowing through the platform every second, and they flag the behavior that does not fit, on every record, in real time, at a cost that makes sense at our scale. What we build does not ship to a single feature. The same models power detection across many of Datadog's security products at once, so the work has impact well beyond any one team. Large general-purpose models are too slow and too expensive to run in that path, so we take the opposite approach: small, custom models, designed for high-throughput stream processing and optimized to run cheaply on every record. We are hiring a Senior Applied Scientist to build these models from start to finish. You will contribute to designing the architecture, training at scale, and the optimization work that takes a model from training to running efficiently on production traffic. This optimization requires a deep understanding of the constraints imposed by both the software and the hardware, together with the applied mathematics to work within them: often it comes down to finding a mathematical reformulation that fits those constraints better, and that judgment can decide whether a model reaches production at all. The work involves a number of open questions. How do you obtain most of the quality of a large model from one that is far smaller and cheap enough to run on the full stream? Where is it worth trading exactness for speed, and how do you reason about the error you accept? How do you make a small model's outputs clear enough that the detection engineers and analysts who rely on it can trust what it reports? If these are the problems you want to work on, we would like to hear from you. At Datadog, we place value in our office culture: the relationships and collaboration it builds, and the creativity it bring