Human Evaluator Jobs

Amplitude's Cloud Platform team builds the systems that every Amplitude engineer relies on every day to ship code — and we're rebuilding them for the AI era. As a Staff Platform Engineer, you'll set technical direction for the platform across teams, lead our highest-complexity and highest-leverage initiatives, and shape a platform where AI agents are first-class users alongside humans: kicking off deploys, opening pull requests against infrastructure, and triaging incidents, so a single engineer can get the throughput of a team. You'll operate across team boundaries — partnering with product engineering, fellow Staff+ engineers, and engineering leadership to make Kubernetes and cloud infrastructure effortless across the entire engineering org. You'll build the self-service automation, shared standards, and scalable AWS and GCP infrastructure that let dozens of product teams ship faster, safer, and with less cognitive load — and you'll multiply the engineers around you while you do it. Key Responsibilities Set technical direction — shape platform and domain-level technical strategy that improves developer experience, reliability, security, and cost, and lead the high-complexity, cross-cutting initiatives that deliver it with measurable impact for the organization. Drive clarity through ambiguity. Take on the most loosely-defined problems, validate the critical assumptions early, and create alignment with stakeholders across teams so others can move quickly and confidently — driving cross-team decisions to a timely close and escalating when needed. Build the AI-augmented platform. Design org-wide tooling, guardrails, and policy-as-code that help every engineer get more out of AI-assisted development — infra primitives an LLM can safely reason about and PR against, automated review, and standards that hold as AI changes how code gets written. Own Infrastructure-as-Code standards for Kubernetes, AWS, and GCP using Terraform, Helm, Kustomize, and emerging tooling — setti

The Product Management team drives Asana’s product strategy and execution, translating customer needs and opportunities into a compelling roadmap and working cross-functionally to deliver impactful solutions for team productivity and collaboration. We’re helping to define the Work Management category and inventing new ways for teams to work more efficiently together and see the larger purpose behind their work. Our team works directly with Engineering, Design, Experience Research, and Data Science, as well as with Sales, Support, and Marketing to ship great products. You’ll take part in every type of product work here — from strategy to product to process improvements — conceptualizing, launching, and iterating on Asana itself for millions of teams around the world. We are looking for a product manager to join Asana’s Work Intake team, which enables teams of humans and agents to get more work context into Asana and seamlessly execute on it. You’ll be responsible for defining the path to product-market fit for new work intake capabilities, for both internal and external customers. You’ll synthesize and prioritize customer and market feedback into product features that can move the needle, and set a compelling roadmap that delivers maximum return on investment. You’ll develop capabilities that other internal teams can incorporate into their product solutions and collaborate with partner teams to develop joint solutions where your surface areas and responsibilities overlap. We are looking for someone who can find the signal in the noise, with deep customer empathy for high-complexity team collaboration workflows. The ideal product manager has an end-to-end ownership mindset and drives for results by breaking work into clear increments, sequencing dependencies, leveraging data and AI, and collaborating with technical partners on tradeoffs affecting reliability and performance. You thrive on learning from customers, and are excited to partner with engin

We are entering a new era of work — one where AI agents collaborate alongside people to plan, act, and deliver outcomes at a scale and speed that wasn't possible before. At Asana, we call this Agentic Work Management: the orchestration of humans and AI agents toward shared goals, with full clarity on who is responsible for what, every step of the way. As a Corporate Account Executive, you will be at the forefront of this shift. You won't just be selling software — you'll be helping DACH organisations understand and solve their most pressing work management challenges, and guiding them toward a future where their teams and AI agents move together with purpose and trust. This role is based in our Munich office with an office-centric hybrid schedule. The standard in-office days are Monday, Tuesday, and Thursday. Most Asanas have the option to work from home on Wednesdays. Working from home on Fridays depends on the type of work you do and the teams with which you partner. If you're interviewing for this role, your recruiter will share more about the in-office requirements. What you’ll achieve Lead with Discovery: Open every engagement with genuine curiosity to uncover hidden friction, misaligned priorities, and coordination failures that cost customers time, money, and trust. Surface Real Pain: Ask the questions that reveal deep operational challenges—not just the stated need—so that solutions you propose are anchored in what actually matters to the business. Map Stakeholder Motivations: Map stakeholder motivations from frontline teams to the C-suite, identifying where the greatest leverage lies for transformative change. Drive Customer Outcomes: Manage a portfolio of corporate customers with a relentless focus on measurable business impact and outcomes that your champions can proudly report upward. Expand Account Revenue: Hunt and develop new business within your book while deepening relationships across existing accounts, expanding usage into new teams and department

As the Senior Product Manager for the Actions & Automations team, you will own the ecosystem that enables customers, partners, and Datadog teams to build, deploy, and operate AI agents on Datadog. You will drive the strategy and execution for the platform capabilities, developer experience, integrations, and extensibility model that make Datadog the best place to build agents that understand and act on production systems. Modern engineering organizations are entering a new era where software is not only monitored and operated by humans, but increasingly by AI-powered agents. As agentic workflows reshape how teams build, operate, secure, and troubleshoot systems, customers need a platform for creating specialized agents, connecting them to business and engineering systems, governing their behavior, and extending them to solve unique organizational problems. You will define and build the ecosystem that makes this possible. Agent Builder sits at the intersection of Datadog's products, AI capabilities, and ecosystem strategy. You will have the opportunity to work across the breadth of the Datadog platform, partner with teams throughout the company, and help establish Datadog as the foundation for operational AI. At Datadog, we place value in our office culture, the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them. What You’ll Do: Define the vision, strategy, and roadmap for Datadog's Agent Builder platform and ecosystem. Own the core platform capabilities that enable customers and partners to create, customize, deploy, and manage AI agents. Drive the extensibility model for agents, including integrations, tools, actions, context sources, APIs, SDKs, and developer workflows. Shape how agents perform actions across Datadog products and third-party systems. Partner closely with AI, platform, infrastructure, and product tea

MongoDB serves as the data layer powering the most important AI applications in the world today. This role exists to ensure the market knows what MongoDB does and to secure the next generation of builders before they default to another platform. The VP of AI Marketing Strategy & Ecosystem will shape the training data, documentation, integrations, and ecosystems that determine what gets built and how, thereby positioning MongoDB as the generational data platform for agentic applications. Builders and agents alike need to reach for MongoDB by default. That requires presence in the systems and tools that influence how applications get built, not just the channels that reach the humans building them. This position will report to the CMO and be part of the Marketing Leadership Team. The VP of AI Marketing Strategy & Ecosystem will work closely with the Chief Product Officers, Chief Technology Officer, Chief Customer Officer, and regional marketing leaders. This role elevates AI visibility and representation, ecosystem partnerships, and developer content into a single VP seat, working closely with product and partner teams to align efforts that currently span the company. This role can be based out of our San Francisco or Palo Alto offices, or remotely in the region. What you will own AI market narrative: own the strategic case for MongoDB as the default data platform for agentic applications and AI-mediated technology selection broadly; ensure the story holds up under scrutiny from analysts and competitors; and ensure it resonates with builders. Partner with Product Management and Product Marketing to keep the AI story integrated into MongoDB's core narrative, not a separate one AI visibility and representation: own how MongoDB appears in AI-generated responses, agent framework recommendations, and developer tool suggestions; This goes beyond content; It includes documentation quality, technical accuracy, and the underlying infrastructure that determines how Mong

AI agents are transforming the way developers interact with software - and databases are no exception. We're seeking a Senior Software Engineer to join our AI Interfaces team within AI Builder Experience (ABX), where you'll provide technical direction, shape architecture, and build the core products that make it seamless for developers and AI agents to work with MongoDB. Our team owns the surfaces through which humans and agents connect to MongoDB - including the MongoDB MCP Server, Agent Skills, our Intelligent Assistant Platform, and purpose-built agents. In short: if it's how an agent talks to MongoDB, we're building it. This is a new team charting new territory, and as a Senior Software Engineer here, you'll be right at the frontier - building the technologies that let AI applications and agentic workflows work seamlessly with MongoDB at scale. You'll integrate with fast-moving, often unproven technologies, make pragmatic calls in the face of ambiguity, and own high-visibility projects end to end with minimal guidance. We're looking for product-minded engineers who thrive on autonomy and take pride in shipping. MongoDB engineering teams pride themselves on building high-quality software and living our cultural values every day - we value intellectual curiosity and honesty, and building together in an environment that prioritizes collaboration over competition. This position requires participation in a 24/7 on-call rotation to ensure business continuity and incident response capabilities. This role can be based out of our Gurugram office. Position Expectations Work closely with research, product management, product engineering, product design, peers, as well as other teams within the company to define the first version and future evolution of our AI interfaces Design, build, and deliver well-tested core pieces of the platform - including the MongoDB MCP Server, Agent Skills, the Intelligent Assistant Platform, and purpose-built agents - in collaboration with othe

About Bolna Bolna is Voice AI infrastructure built for India. We help businesses deploy intelligent voice agents that can call, converse, and convert in any language, at scale. From collections to customer support to sales, our agents handle millions of conversations so humans don't have to. We're a YC F25 company, backed by General Catalyst, with 1,050+ paying customers and growing fast. Our team of ~25 is based in Bengaluru. The Role You're the person who makes our voice agents actually sound good and actually work for real customers. As an AI Solutions Engineer, you'll sit at the intersection of our product and our customers. Your primary job is to design, write, and iterate on the prompts and tools that power Bolna's voice agents, making them smarter, more natural, and more effective for each use case. You'll work closely with customers to understand their goals, build agent flows, test conversations, and fix what breaks. No heavy coding required-if you can vibe-code a basic script or write a solid system prompt, you're qualified. What You'll Do Write, test, and iterate on system prompts for voice agents across industries like D2C, fintech, healthcare, and logistics Listen to real call recordings, identify where agents fail, and fix them Build conversation flows and call pathways for new customer deployments Help onboard new customers-understand their use case, set up their agent, and get it live Maintain a growing library of prompts, templates, and best practices across Bolna's verticals Red-team agents-try to break them, find edge cases, and make them bulletproof Work with vernacular inputs: test agents in Hindi, Hinglish, and regional languages Feed insights back to product and engineering-you'll see what customers need before anyone else does What We're Looking For Must-have: You've spent serious time prompting ChatGPT, Claude, or similar LLMs-not just casually, but to actually build or solve something You're obsessive about language-you notice when a senten

Product Manager – Innovation & Design Location: Paris (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Imaginez et construisez des produits qui répondent aux vrais besoins des utilisateurs et transforment durablement les services financiers. Capco recherche des Product Managers pour accompagner les acteurs majeurs des services financiers et de l'énergie dans la conception de produits utiles à leurs clients et créateurs de valeur pour leur organisation. Au sein de notre capability Tech, Product & Engineering , vous intégrerez des équipes pluridisciplinaires réunissant Product Managers, ingénieurs, développeurs et experts métiers. Votre rôle sera de comprendre les besoins des utilisateurs, identifier les bonnes problématiques à résoudre et transformer ces insights en produits simples, viables et performants. Vous intégrerez également de manière pragmatique les outils d’IA et d’automatisation approuvés dans vos méthodes de travail pour accélérer la recherche, l’analyse et l’expérimentation, tout en conservant le jugement humain au cœur des décisions produit. Vos responsabilités Travailler avec des équipes produit pluridisciplinaires pour identifier les problématiques utilisateurs prioritaires et concevoir des solutions à la fois désirables, viables et techniquement réalisables. Intervenir sur l’ensemble du cycle de vie produit, de la discovery et de la recherche utilisateur jusqu’au lancement, à la mesure de la performance et à l’optimisation continue. Combiner études de marché, recherche qualitative, données quantitatives et comportements observés pour construire une compréhension structurée des utilisateurs et exploiter, lorsque pertinent, des outils d’IA approuvés pour accélérer l’analyse et faire émerger des insights. Intégrer les contraintes propres aux services financiers – réglementation, risques et contrôles, protection des données et viabilité économique – dans les décisions produit. Piloter le produit par l

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corporate cards and banking with intuitive spend management, bill pay, and travel software, Brex enables founders and finance teams to accelerate operations, gain real-time visibility, and control spend effortlessly. Brex’s AI-native automation and world-class service eliminate manual expense and accounting tasks for customers so they can focus on what matters most. Tens of thousands of the world's best companies run on Brex, including DoorDash, Coinbase, Robinhood, Zoom, Plaid, Reddit, and SeatGeek. Working at Brex allows you to push your limits, challenge the status quo, and collaborate with some of the brightest minds in the industry. We’re committed to building a diverse team and inclusive culture and believe your potential should only be limited by how big you can dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career. Customer Experience at Brex The Customer Experience team is the connective tissue between Brex’s product and the companies that depend on it every day. We don’t just resolve issues — we surface insights, close feedback loops, and help shape the product roadmap. As Brex becomes an AI-native company, our CX organization leads that shift: building the systems, metrics, and workflows that let AI handle routine complexity while humans focus on judgment and relationships. We measure success not by tickets closed, but by customers who don’t need to contact us in the first place. What you’ll do Brex is looking for an experienced AI operator & leader to own the strategy and execution of AI-powered customer experience. You’ll lead the team responsible for Brex’s AI customer service platform — optimizing instruction sets, driving quality, and expanding AI across chat, voice, and email. You’ll also own the internal side of the equation: build

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide Job Description PayPay In

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide. Job Description : PayPay India is look