Model Behavior Engineer Jobs

The ML Observability team builds cutting-edge tools to monitor, explain, and improve AI systems in production, particularly those leveraging Large Language Models (LLMs) and generative AI. We provide robust, scalable observability for AI workloads, including drift detection and model evaluation, and behavior tracing, enabling customers to ship AI with confidence. As a Staff Engineer, you’ll lead the development of new features and foundational capabilities within Datadog’s LLM Observability product. You will shape product direction, drive experimentation, and apply your deep understanding of both AI systems and software engineering to solve open-ended problems in the fast-moving AI landscape. Your work will directly impact how our customers monitor, troubleshoot, and optimize LLM-based applications in production. Join us in building the foundational tools that make AI systems observable, understandable, and reliable in the real world. At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Drive design and implementation of LLM observability features. Ideate, prototype, and scale new product features to provide insights and drive improvements for generative AI systems Work cross-functionally with other eng teams, product, UX, and applied science to iterate fast and find product-market fit Develop and extend tools for tracing, evaluating, and debugging LLMs Influence architecture decisions and mentor engineers to build resilient, high-performance systems Stay close to customer pain points and use those insights to guide product and engineering priorities Stay current with industry trends and advancements in machine learning and observability, driving innovation within the team Who You Are: You have a BS/MS/PhD in a Computer Science, Engineering or r

About the Team GTM Growth Engineering builds AI-native products that help OpenAI's go-to-market and B2B marketing organizations scale with greater speed, intelligence, and operational effectiveness. We apply OpenAI models to real business workflows and build the systems that make those applications useful and dependable: customer context, agent behavior, feedback, evaluation, experimentation, and appropriate human oversight. Our work brings together software engineering, applied AI, product, data, and GTM operations. We measure success through the quality of customer engagement, pipeline, conversion, and the effectiveness of our sales and marketing teams. About the Role We're looking for an Applied AI Engineer to build production systems that help AI-powered go-to-market workflows improve over time. You will connect agent behavior, customer and operator feedback, evaluation, experimentation, and business outcomes to make these systems more effective, reliable, and responsive to evolving customer needs. This is a deeply technical, cross-functional role with end-to-end ownership of the agent improvement loop: understand production behavior, identify failure modes, improve how the system decides or acts, and validate the resulting impact. You will partner with Engineering, Product, Data Science, Sales, and B2B Marketing to turn real-world signals into safer, more effective agent behavior and measurable improvements in customer engagement, conversion, qualified pipeline, and team productivity. In this role, you will: Own the production improvement loop across agent behavior, customer and operator feedback, evaluation, experimentation, and verified business outcomes. Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and downstream outcomes can be understood in context. Define meaningful quality standards, representative evaluation datasets, regression coverage, and production monitoring for real GTM workflows. Investigate why a

About the Team The Platform Analytics team builds the systems OpenAI researchers use to understand the quality and behavior of the models we train including what models are doing, why they behave in a particular way, and how that behavior changes across experiments. Neptune is a core part of this work. It ingests, stores, queries, and visualizes large volumes of metrics from pretraining, post-training, and reinforcement learning. Hundreds of researchers depend on these systems in their daily work to compare experiments, debug unexpected behavior, and decide what to try next. Our scope is broader than metrics. We also build platforms that help researchers analyze samples, traces, evaluation results, and other structured or unstructured data through dashboards, APIs, and increasingly agent-driven workflows. These systems need to remain fast, reliable, and understandable as the scale and complexity of research change quickly. We are not trying to become a consulting team that builds a separate solution for every research project. We work directly with researchers to understand recurring problems, then turn them into reusable infrastructure and platform capabilities that many teams can build on. About the Role We’re looking for a hands-on experienced software engineer who can take ownership of a critical system and drive it from problem definition through production adoption. This person should be able to own a platform such as CacheHouse end to end: define its technical direction, design its data model and storage architecture, integrate it with several research dashboards and workflows, guide one or two engineers, and ensure the system works reliably for its users. The right candidate should already bring the technical judgment, ownership, and execution expected at this level. The primary learning curve should be OpenAI’s stack and research problem space, not learning how to lead a complex engineering effort or deliver a production system. You will work directly with

About the Team Our team analyzes inference stack performance across the application, model, and fleet layers to identify bottlenecks and drive faster, cheaper inference. We combine systems profiling, benchmarking, and analysis to understand where time and cost are spent, then turn that understanding into performance optimizations and models that project performance and capacity needs for future launches. About the Role In this role, you will model inference performance across application, model, and fleet layers with higher fidelity. You will build cost-to-serve estimates from microbenchmarks and create tools that help cross-functional teams reason about latency, capacity, utilization, and cost tradeoffs. In this role, you will Build and refine performance models that translate microbenchmark results into cost-to-serve estimates. Analyze inference workloads end to end across applications, models, and fleet infrastructure. Enhance tooling to identify bottlenecks across layers for latency and throughput. Partner with other teams to turn performance insights into concrete improvements and project how future changes affect inference. You might thrive in this role if you: Enjoy reasoning from first principles about distributed systems, model inference, and hardware efficiency. Are comfortable working across abstraction layers, from application behavior to kernels, accelerators, networking, and fleet scheduling. Have deep expertise with performance profiling, benchmarking, analysis, and optimization. Enjoy collaborating with engineering and research teams to improve real production systems. About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve o

CONTRACTOR: Forward Deployed Engineer, Pendo Predict The team + the role Pendo Predict helps companies act on customer behavior before problems surface. We build custom predictive models — churn, upsell, expansion, and more — on top of product usage, CRM, and data warehouse data, and put those insights directly into the workflows of CS, Sales, and Product teams. By analyzing historical product usage alongside CRM and data warehouse outcomes, Predict forecasts which accounts are at risk, ready to expand, or likely to convert — surfacing intelligence where teams can actually act on it. As a Forward Deployed Engineer on the Predict team, you'll embed directly with customers and prospects to design, build, and ship predictive models inside their real-world data environments — and feed what you learn back into the product. This is a hands-on, hybrid role at the intersection of engineering, technical consulting, and customer-facing work. You'll own the technical side of the customer journey end-to-end: from pre-sales discovery and proof of value, through data modeling and integration, to enablement and adoption. SQL is a daily tool in this role. This is a contractor engagement based in Herzliya, Israel, working on United States EST hours. This role is based in Herzliya, Israel and works hybrid from Pendo's Herzliya office. What this looks like day-to-day Run technical discovery with customers to assess product analytics maturity, data availability, and business processes — and use that to set realistic implementation expectations and position Predict for their stage. Work hands-on with customers to design the data model and define the business metric the model will predict; map customer data sources (Pendo, Salesforce, Snowflake, and other warehouses) to Predict requirements and best practices. Build and present compelling predictions in the prospect's own domain — explain methodology, model accuracy, and activation to earn credibility with technical stakeholders in

We believe that the way people interact with their finances will drastically improve in the next few years. We’re dedicated to empowering this transformation by building the tools and experiences that thousands of developers use to create their own products. Plaid powers the tools millions of people rely on to live a healthier financial life. We work with thousands of companies like Venmo, SoFi, several of the Fortune 500, and many of the largest banks to make it easy for people to connect their financial accounts to the apps and services they want to use. Plaid’s network covers 12,000 financial institutions across the US, Canada, UK and Europe. Founded in 2013, the company is headquartered in San Francisco with offices in New York, Seattle, Washington D.C., Raleigh, London, and Amsterdam. The Fraud Data team at Plaid builds the machine learning systems that power Plaid’s fraud detection products, leveraging insights from across Plaid’s network to help identify and stop fraud before it happens. Our team works across the full data science and machine learning lifecycle—from discovering new signals and experimenting with models to deploying and optimizing them in production. We continuously learn from real-world model performance and customer feedback to improve our systems and develop new ways to protect customers and consumers from evolving fraud threats. As a Senior Machine Learning Engineer on Plaid's Fraud Data team, you will develop models that improve fraud detection for our customers. You will identify predictive patterns in Plaid's network data and lead projects from initial experiments through model deployment and ongoing improvement. Investigate fraud patterns and model errors to identify new signals, improve detection, and expand coverage across customers and use cases. Develop training datasets and predictive features, addressing challenges such as incomplete labels, class imbalance, data leakage, and changing fraud behavior. Design, train, and tune model

About the Team API Frontiers turns OpenAI’s frontier models into production APIs that developers can use to build reliable products and agents. We own the core path connecting models to developers through the Responses API, with a focus on safety, reliability, and speed. Working closely with Research, Safety, Codex, and other API teams, we bring new model capabilities into production and improve them through developer feedback. About the Role We are looking for a backend software engineer to build and operate the services behind the Responses API. You will shape API behavior, bring new capabilities from research into production, and make long-running agent workflows dependable and fast. The work combines distributed systems engineering with product judgment: designing useful developer interfaces, managing staged rollouts, and following production issues through to durable fixes. In this role, you will: Design, build, and operate APIs and backend services that bring frontier model capabilities to developers. Partner with Research, Safety, Codex, and API teams to define API behavior and deliver safe, staged launches. Build API capabilities for agent workflows, including task delegation, context sharing, and parallel execution. Strengthen long-running request reliability across timeouts, cancellation, streaming, and background execution. Improve request-processing performance and tail latency through profiling, efficient systems code, and persistent connections. Turn developer feedback and production failures into better observability, diagnostics, and lasting product improvements. Your background might look something like: 5+ years of experience building and operating backend services or developer-facing APIs in production. Strong software engineering fundamentals, with practical knowledge of distributed systems, concurrency, and asynchronous execution. Ability to diagnose production failures and performance bottlenecks using observability data and profiling. Product

What you’ll do Act as the technical lead for large parts of the scanner platform: system architecture, codebase structure, and long-term maintainability. Own core runtime foundations: distributed control, state management, fault handling, and reliability. Drive engineering rigor: testability, code quality, review standards, performance regression prevention, and release processes. Build robust observability: logs, metrics, traces, and replayable diagnostics (with privacy constraints). Collaborate with hardware and recon/ML teams to define interfaces, data contracts, timing/synchronization, and failure modes. Lead complex refactors (e.g., message passing / RPC boundaries, modularization, concurrency model) without halting forward progress. What we’re looking for Deep software architecture experience for real-world systems: robotics, instrumentation, medical devices, or other complex distributed products. Strong Python and concurrency background (asyncio, multiprocessing, profiling, performance engineering). Track record of shipping systems that are observable, debuggable, and resilient. Strong technical leadership: clarity, pragmatic trade-offs, and mentoring. Useful experience Building but rock-solid systems: clear interfaces (gRPC/protobuf or equivalent), strong state modeling, and failure handling. High-leverage engineering habits on a lean team: good tests, CI, reproducible dev environments, and fast code review. Practical performance + concurrency work in Python (asyncio, profiling, multiprocessing) and comfort debugging distributed behavior. Security-minded device software: safe defaults, encrypted data paths, and disciplined handling of PII/PHI. Operational thinking: remote updates/management, excellent logging, and diagnostics that make real hardware debuggable.

Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the world around us, that’s why we’re building a universal autonomy platform: self-driving for all roads and all rides. Founded in 2016, Nuro is a physical AI company developing Level 4 autonomous driving technology for a wide range of vehicles, use cases, and markets. Powered by the Nuro Driver™, our universal autonomy platform enables the global mobility ecosystem to deploy autonomy at scale, from robotaxis and logistics fleets to personal vehicles. With years of real-world deployment experience and a flexible, partner-led business model, Nuro is working toward a future where millions of autonomous vehicles powered by our technology help make everyday life safer, easier, and more connected. Nuro has raised over $2B in capital from Uber, NVIDIA, Google, Softbank, Fidelity, T. Rowe Price, and other leading investors. About the Role Build and evolve our automated hardware-in-the-loop (HIL) validation framework used for ADAS/autonomous driving ECU and system testing. Translate formal validation requirements into repeatable automated test cases covering nominal operation, fault handling, timing, communication integrity, and safety-related behavior. Execute and debug validation on HIL benches using vehicle network interfaces, embedded target access, fault injection, and system emulation. Partner with firmware, systems, and validation teams to define test coverage, investigate failures, and improve release readiness. Support automated regression and release testing, including pre-test firmware deployment, orchestrated test execution, and structured result reporting. Build/Maintain shared framework components as new vehicle platforms, ECU releases, and validation domains are added. About the Work Practical experience with HIL/bench hardware integration (po

Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the world around us, that’s why we’re building a universal autonomy platform: self-driving for all roads and all rides. Founded in 2016, Nuro is a physical AI company developing Level 4 autonomous driving technology for a wide range of vehicles, use cases, and markets. Powered by the Nuro Driver™, our universal autonomy platform enables the global mobility ecosystem to deploy autonomy at scale, from robotaxis and logistics fleets to personal vehicles. With years of real-world deployment experience and a flexible, partner-led business model, Nuro is working toward a future where millions of autonomous vehicles powered by our technology help make everyday life safer, easier, and more connected. Nuro has raised over $2B in capital from Uber, NVIDIA, Google, Softbank, Fidelity, T. Rowe Price, and other leading investors. About the Team The Robotics Reliability Engineering (RRE) team at Nuro focuses on fleet reliability as our AV capabilities and operating footprint grow. We work across software, hardware, infrastructure, and operations to understand fleet behavior and improve operational readiness. When high-severity events impact missions or fleet-wide performance, RRE coordinates investigations and ensures that lessons learned result in durable platform improvements. About the Role As a Software Reliability Engineer at Nuro, you will help design, build, and operate systems that support our autonomous vehicle platform and fleet operations. You will work across the full development lifecycle, from early design and deployment through operations and continuous improvement. You’ll join an on-call rotation to stay close to real-world operations. At its core, the role is building resilient systems through automation, observability, and operational feedback. T

Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the world around us, that’s why we’re building a universal autonomy platform: self-driving for all roads and all rides. Founded in 2016, Nuro is a physical AI company developing Level 4 autonomous driving technology for a wide range of vehicles, use cases, and markets. Powered by the Nuro Driver™, our universal autonomy platform enables the global mobility ecosystem to deploy autonomy at scale, from robotaxis and logistics fleets to personal vehicles. With years of real-world deployment experience and a flexible, partner-led business model, Nuro is working toward a future where millions of autonomous vehicles powered by our technology help make everyday life safer, easier, and more connected. Nuro has raised over $2B in capital from Uber, NVIDIA, Google, Softbank, Fidelity, T. Rowe Price, and other leading investors. About the Role We’re looking for an Autonomy Engineer focused on onboard autonomy—the software that runs on the robot/vehicle/embedded computer and makes real-time decisions using onboard sensors and compute. You’ll build and ship reliable autonomy features that operate under tight latency, compute, and safety constraints in the real world. What You’ll Do Develop, integrate, and deploy onboard autonomy behaviors (e.g., navigation, obstacle avoidance, lane/route following, docking, interaction behaviors). Implement and maintain real-time decision-making components: behavior planning, state machines/behavior trees, local planning, and control interfaces. Build robust sensor-driven autonomy pipelines on-device (camera, lidar, radar, IMU, wheel odometry, GNSS), including synchronization, calibration hooks, and fault handling. Optimize autonomy performance for latency, CPU/GPU usage, memory, and power on embedded compute (e.g., NVIDIA Jetson,

MongoDB’s Replication Team builds the infrastructure that enables high availability, fault tolerance, automatic failover, and tunable consistency. As an engineer on the team, you will design and implement distributed-systems features that protect data and keep applications available under demanding operating conditions. You will work primarily in C++ on core database code, partner with engineers across MongoDB, and help shape features that are central to major MongoDB releases. This is an opportunity to apply distributed-systems fundamentals to a widely used database while solving challenging problems in correctness, performance, and operability. We're looking to speak with candidates based in New York City for our in-office working model. What you will do Design and implement replication features based on the Raft consensus protocol Improve failover behavior, availability, correctness, and performance across the replication system Write production-quality C++ and the unit, integration, and system tests needed to demonstrate correctness Use JavaScript and Python where appropriate to extend test coverage and validate end-to-end behavior Diagnose test failures, investigate bugs, and drive issues through root-cause analysis and resolution Measure the performance impact of code changes and prevent or resolve regressions Collaborate with partner engineering teams and stakeholders on large, cross-functional initiatives Investigate distributed-systems issues raised by customers and Technical Support, communicate findings clearly, and help deliver durable fixes Participate in code reviews, design reviews, and technical discussions that improve the quality of the team’s work Interview candidates and mentor junior engineers and interns What you bring Required At least five years of experience programming, debugging, and performance-tuning distributed or highly concurrent software systems Strong systems fundamentals, including multithreaded programming, concurrency, debugging,

About the Team OpenAI’s Forward Deployed Engineering team partners with customers to turn research breakthroughs into production systems. We operate at the intersection of customer delivery and core platform development. About the Role As a foundational FDE manager, you’ll lead FDE through high-stakes, ambiguous customer deployments and own technical and business value outcomes end to end. You’ll grow a team that can operate under pressure and help OpenAI learn from the field. You’ll partner closely with Product, Research, Sales, and GTM to ensure fieldwork informs roadmap priorities, drives new exploration, and supports safe deployment at scale. Your decisions will influence how OpenAI is trusted by the customers closest to our deployment work. Your success will be measured by how consistently your team ships, how clearly you deliver signal to Research and Product, and how durable your team and delivery model prove to be. This role is based in Tokyo. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. This role also will require travel up to 40%, although mainly local. To succeed in this position you must be bilingual—fluent in both Japanese and English (spoken and written). Please note that your resume must be submitted in English, and the interview process will include conversations in both languages. In this role, you will: Lead and grow a team of FDE delivering production systems with frontier models Own end-to-end delivery outcomes through clarity, speed, tight coordination, and technical quality Codify what works into tools, playbooks, and roadmap inputs that create leverage for both OpenAI and our wider developer community Notice early indicators and raise them with urgency, whether in product behavior, customer environments, or delivery practices Use judgement to distinguish what requires action and what does not Set a high bar for FDE performance and support each person’s growth through direct, acti

About the Team The Post-Training Frontiers team is responsible for training the frontier agents OpenAI ships to the world (GPT-Next). We train the flagship agentic models behind Codex, ChatGPT, and the API through large-scale reinforcement learning. The team’s work spans four areas. First, execution and science: working with teams across OpenAI to decide what can go into the final model and how, using scientific experiments and evals that are representative of the final pipeline so issues can be recognized early. Second, RL scaling: executing the final large-scale reinforcement learning run, making sure GPUs are used efficiently and training stays healthy. Third, research: improving horizontal capabilities like instruction following, factuality, memory, and multi-agent behavior, where the team’s broad visibility helps identify cross-cutting improvements across teams and domains. Fourth, engineering: maintaining the infrastructure stack and internal tools to ensure that both the final run and all integrations go as smoothly as possible and that the systems are easy to work with. About the Role This role focuses on keeping our frontier RL training runs fast, reliable, and unblocked. You will work across engineering and infrastructure problems as they emerge, from scaling and orchestration issues to inference bottlenecks, numerical problems, and hardware failures, as well as supporting large horizontal integrations in the big run, like multi-agent capabilities or memory. This is a role for a strong generalist who quickly learns anything needed for the task, has high attention to detail, debugs deeply, and is motivated by fixing the highest-impact problem in front of the team. In this role, you will: Keep large-scale async RL training runs moving by jumping into the most urgent engineering and infrastructure problems. Debug issues across training systems, inference, orchestration, scaling, and distributed infrastructure. Improve the reliability and efficiency of RL trai

About the Team OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. The Consumer Devices team is building a new generation of AI-powered products that seamlessly integrate hardware and software to create intuitive, transformative experiences. We bring together experts across embedded systems, machine learning, hardware, design, and product engineering to develop products at the intersection of AI and consumer technology. About the Role OpenAI is seeking a System Power Engineer to characterize, measure, and optimize power consumption across our embedded hardware products. In this role, you will work closely with Electrical Engineering and system software teams to build power test automation, measure subsystem-level power usage, and drive improvements that directly impact battery life, thermal behavior, charging performance, and system reliability. You will help establish the methodologies and metrics used to understand and improve power efficiency across real-world product experiences, from controlled lab environments to representative day-in-the-life usage scenarios. This role requires hands-on experience with embedded hardware platforms, power instrumentation, and the analysis of power profiles and system behavior. This role is based in San Francisco, CA. We use a hybrid work model of four days per week in the office and one day working remotely. Relocation assistance is available for new hires. In this role, you will: Define and develop power testing automation to evaluate system behavior across a range of workloads and operating conditions. Measure subsystem-level power consumption using power breakout probes and other lab instrumentation. Develop and execute power characterization tests spanning basic workloads, complex mixed-use scenarios, and representative day-of-use experiences. Partner closely with Electrical Engineers to identify opportunities to improve system power efficiency. Collaborate with software engineering