Human Evaluator in United States

As the Senior Product Manager for the Actions & Automations team, you will own the ecosystem that enables customers, partners, and Datadog teams to build, deploy, and operate AI agents on Datadog. You will drive the strategy and execution for the platform capabilities, developer experience, integrations, and extensibility model that make Datadog the best place to build agents that understand and act on production systems. Modern engineering organizations are entering a new era where software is not only monitored and operated by humans, but increasingly by AI-powered agents. As agentic workflows reshape how teams build, operate, secure, and troubleshoot systems, customers need a platform for creating specialized agents, connecting them to business and engineering systems, governing their behavior, and extending them to solve unique organizational problems. You will define and build the ecosystem that makes this possible. Agent Builder sits at the intersection of Datadog's products, AI capabilities, and ecosystem strategy. You will have the opportunity to work across the breadth of the Datadog platform, partner with teams throughout the company, and help establish Datadog as the foundation for operational AI. At Datadog, we place value in our office culture, the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Define the vision, strategy, and roadmap for Datadog's Agent Builder platform and ecosystem. Own the core platform capabilities that enable customers and partners to create, customize, deploy, and manage AI agents. Drive the extensibility model for agents, including integrations, tools, actions, context sources, APIs, SDKs, and developer workflows. Shape how agents perform actions across Datadog products and third-party systems. Partner closely with AI, platform, infrastructure, and product tea

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

About Sentry Software runs the world and the pace is faster than ever. Sentry helps developers fix errors and performance issues before users notice, so teams can spend less time firefighting and more time building. Trusted by 200,000+ organizations, Sentry is today’s application monitoring standard and our team is building its AI-native future. About the role Sentry's issue platform processes billions of events every day to help millions of developers find and fix bugs. The hard part is deciding which events point to a problem, which belong together, and what a developer needs to know to investigate. As a Senior Software Engineer on the Issue Detection team, you'll design, build, and operate the systems that make those decisions. You'll work on real-time processing pipelines, monitors, and analysis systems that detect problems and turn them into issues. The work combines distributed systems with product engineering. Choices about detection accuracy and processing latency affect which problems developers see and how soon they can act. You'll help shape how developers monitor their applications and how Sentry groups related events into issues. You'll also build the context developers and AI agents need to investigate what went wrong. Keeping these systems reliable and fast as Sentry grows is part of the job, alongside making the issues they produce more useful. In this role you will Build and scale features on a product surface handling billions of events daily, where both query latency and correctness are immediately visible to users. Own the design and delivery of substantial projects end to end, scoping alongside product and design, making the technical calls within your scope, shipping, and instrumenting what you ship so the team can measure it. You will contribute to meaningful technical product decisions : grouping quality, search performance, migrations and backfills against enormous datasets, and making the surface work well for both humans and agents. Champi

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. About Replit Replit is building the world’s most ubiquitous AI coding agent. Replit Agent can be used by anybody to bring their ideas to life. Whether it’s an app for yourself, the next great startup idea, or a tool to make you more productive at work, Replit Agent can help build it. Replit is also the leader in secure vibe coding. We protect apps, give users features to manage security risks, and help them vibe code more safely. About the Role In this role you will build powerful tools that help product engineers iterate rapidly on the Agent experience and directly enhance the core Agent itself. You’ll bridge the gap between the AI team (working on the core Agent logic) and the UX team (crafting delightful Agent experiences), enabling both groups to excel within their specialties. This role blends systems engineering, developer experience and product engineering. We tackle complex challenges across the full stack, from browser-based interfaces to high-performance backends to Linux systems engineering. We’re looking for engineers who have a keen sense of the product experience and how to power it with performant systems. On this team, you’ll have the opportunity to grow your skills across our infrastructure and product, and to lead end-to-end efforts with meaningful impact. We value diverse perspectives and encourage candidates from all backgrounds and experiences to apply. You Will Build high-throughput backend applications and services, like streaming chat between user and agent. Design a collaborative "Multiplayer Computer" that lets humans and AI agents work together on shared shells, filesystems, and state—conflict-free and in real time. Develop infrastructure (frontend & backend) that empowers product enginee

GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital transformation. More than 50 million registered users and more than 50% of the Fortune 100* trust GitLab to ship better, more secure software faster. The same principles built into our products are reflected in how our team works: we embrace AI as a core productivity multiplier, with all team members expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact. GitLab is where careers accelerate, innovation flourishes, and every voice is valued. Our high-performance culture is driven by our values and continuous knowledge exchange, enabling our team members to reach their full potential while collaborating with industry leaders to solve complex problems. Co-create the future with us as we build technology that transforms how the world develops software. * Fortune 500® is a registered trademark of Fortune Media IP Limited, used under license. Claim based on GitLab data. Fortune 100 refers to the top 20% ranked companies in the 2025 Fortune 500 list, published in June 2025. Fortune and Fortune Media IP Limited are not affiliated with, and do not endorse products or services of GitLab. An overview of this role As a Senior Product Manager , you will own GitLab's Secret Detection offering and the Vulnerability Research function that produces the detection content across security products. Secret Detection is one of the highest-signal, highest-volume security capabilities on the platform. Leaked credentials are the most common initial access vector in real breaches, and as AI agents write, commit, and configure more of the software, the surface area for exposed secrets grows faster than the number of humans watching it. You will own the full loop: detect a secret with high precision, tell the customer whether it i

About the Team The Cooperative AI team is scaling to devices and embedded operations and user experiences. Our model-powered scaled workforce and knowledge system are moving on to the edge and powering our devices and edge experiences. By leveraging OpenAI’s state-of-the-art models and technologies, in production and in the lab, we develop systems that reason and work autonomously with customers and with our workforce responsible for operational work. We carry real workloads for critical systems across finance, sales, customer support, integrity, product insights, internal operations, and now devices to drive insights into product and industry. We partner closely with internal teams and external customers globally, operating in a hyper-fast feedback loop where many of our users are just a few steps away. This proximity allows us to iterate quickly, validate impact in real time, and accelerate industry impacting learnings and systems builds. We are a highly multidisciplinary, self-contained team focused on transforming the workplace via smart systems, knowledge, scalable and reliable primitives that apply world-class AI capabilities across domains. Our mission is to learn fast and transform how humans collaborate with AI at scale. About the Role We are looking for a Technical Lead Manager to lead a team of engineers building AI-native embedded experiences and operations-forward systems. In this role, you will perform both hands-on technical leadership and small team management. You will drive business outcomes, architecture and technical strategy for complex systems, contribute directly to implementation, and help grow a high-performing team. You will work closely with internal stakeholders to understand operational challenges, identify high-leverage opportunities for automation, and deliver solutions that create measurable impact. This role is ideal for someone who enjoys moving between technical design, coding, mentoring engineers, and working directly with users t