Human Evaluator Jobs

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide Job Description This role

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide. Job Description PayPay India is lookin

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide. Job Description PayPay India&nb

About PayPay India PayPay, a fintech company providing a service enjoyed by over 75 million users since its launch in 2018 in Japan. The company is now home to a very diverse team of members from more than 50 countries. We grew to a team of several thousand employees in Japan but are far from over. We are still in the Day 1. Every day, new members join us from all over the world to create new value and deliver it to society. Why India ? To build our Payment services, we got technical cooperation from Paytm (A large payment service company in India). And based on their customer-first technologies , we created and expanded the smartphone payment service in Japan. Therefore, we have decided to establish a development base in India, because it is a major IT country with many talented engineers, as evidenced by the fact that cutting-edge mobile payments can continue to be generated. OUR VISION IS UNLIMITED We dare to believe that we do not need a clear vision to create a future beyond our imagination. PayPay will always stay true to our roots and realise a vision (future) that no one else can imagine by constantly taking risks and challenging ourselves. With this mindset, you will be presented with new and exciting opportunities on a daily basis and have the opportunity to grow and reach new dimensions that you could never have imagined. AI-First Culture at PayPay At PayPay, we believe the future of software development is AI-augmented engineering. Every engineer is expected to actively leverage AI tools for code reviews, testing, documentation, debugging, and productivity improvements. AI is not optional — it is a mandatory part of our daily engineering workflow. Candidates must demonstrate both hands-on experience and the right mindset to integrate AI effectively into their workflow, while maintaining the engineering rigor, ownership, and problem-solving skills that only humans can provide. Job Description PayPay India is lookin

About Sentry Software runs the world and the pace is faster than ever. Sentry helps developers fix errors and performance issues before users notice, so teams can spend less time firefighting and more time building. Trusted by 200,000+ organizations, Sentry is today’s application monitoring standard and our team is building its AI-native future. About the role Sentry's issue platform processes billions of events every day to help millions of developers find and fix bugs. The hard part is deciding which events point to a problem, which belong together, and what a developer needs to know to investigate. As a Senior Software Engineer on the Issue Detection team, you'll design, build, and operate the systems that make those decisions. You'll work on real-time processing pipelines, monitors, and analysis systems that detect problems and turn them into issues. The work combines distributed systems with product engineering. Choices about detection accuracy and processing latency affect which problems developers see and how soon they can act. You'll help shape how developers monitor their applications and how Sentry groups related events into issues. You'll also build the context developers and AI agents need to investigate what went wrong. Keeping these systems reliable and fast as Sentry grows is part of the job, alongside making the issues they produce more useful. In this role you will Build and scale features on a product surface handling billions of events daily, where both query latency and correctness are immediately visible to users. Own the design and delivery of substantial projects end to end, scoping alongside product and design, making the technical calls within your scope, shipping, and instrumenting what you ship so the team can measure it. You will contribute to meaningful technical product decisions : grouping quality, search performance, migrations and backfills against enormous datasets, and making the surface work well for both humans and agents. Champi

At Trustpilot, we're on an incredible journey. We're a profitable, high-growth FTSE-250 company with a big vision: to become the universal symbol of trust. We run the world's largest open customer review platform, and while we've come a long way, there's still so much exciting work to do. Come join us at the heart of trust! Test 679 About us Trustpilot began in 2007 with a simple yet powerful idea that is more relevant today than ever — to be the universal symbol of trust, bringing consumers and businesses together through reviews. Trustpilot is open, independent, and impartial — we help consumers make the right choices and businesses to build trust, grow and improve. Today, we have more than 350 million reviews and 60 million monthly active users across the globe, with 149 billion annual Trustpilot brand impressions, and the numbers keep growing. We have more than 1000 employees and we’re headquartered in Copenhagen, with operations in Amsterdam, Denver, Edinburgh, Hamburg, London, Melbourne, Milan and New York. We’re driven by connection. It’s at the heart of what we do. Our culture keeps things fresh –– it’s built on the relationships we create. We talk, we laugh, we collaborate and we respect each other. We work across borders and cultures to be the universal symbol of trust in an ever-changing world. With vibrant office locations worldwide and over 50 nationalities, we’re proud to be an equal opportunity workplace with diverse perspectives and ideas. Our purpose to help people and businesses help each other is a tall order, but we keep it real. We’re a great bunch of humans, doing awesome stuff, without fuss or pretense. A successful Trustpilot future is driven by

Synthesia is the world’s leading AI video platform for business, used by over 90% of the Fortune 100. Founded in 2017, the company is headquartered in London, with offices and teams across Europe and the US. As AI continues to shape the way we live and work, Synthesia develops products to enhance visual communication and enterprise skill development, helping people work better and stay at the center of successful organizations. Following our recent Series E funding round, where we raised $200 million, our valuation stands at $4 billion. Our total funding exceeds $530 million from premier investors including Accel, NVentures (Nvidia's VC arm), Kleiner Perkins, GV, and Evantic Capital, alongside the founders and operators of Stripe, Datadog, Miro, and Webflow. About the role As a Staff Research Engineer, you will join a team of 40+ Researchers and Engineers within the R&D Department working on cutting edge challenges in the Generative AI space, with a focus on avatar-centric interactive video diffusion models. Within the team you’ll have the opportunity to work on the applied side of our research efforts and directly impact our solutions that are used worldwide by over 60,000 businesses. This is a unique opportunity for experts in machine learning and diffusion models to shape the future of AI video agents that can think, act, and react like humans. As part of our Interactive Avatars Team, you’ll work on cutting-edge research with a clear focus on turning breakthrough ideas into real product capabilities. You’ll join a team that moves fast, iterates often, and builds models that ship and make a meaningful impact. Example tasks and responsibilities include: Adapt diffusion models to incorporate diverse conditioning signals (e.g., audio, motion, interaction cues). Develop methods for streaming infinitely long video sequences at real-time rates. Work on the perceptual layer of interactive agents, including understanding user audio and generating appropriate contextua

Amplitude is moving fast, from product analytics to an AI-native platform with agents embedded across the product, used by over 4,700 companies. We are looking for a strong design leader with a builder mindset to lead a 15-person design team through the next phase of growth. This is a player-coach role . You manage and grow the team. You also stay in the actual work: elevating quality through feedback, pairing with designers on the hardest UX puzzles, and asking the right questions to unlock the crux of the problem. You'll know every product surface well enough to raise its quality — even if you don't own any single one end-to-end. Designers are embedded in autonomous pods and need to keep shipping fast. Maintaining speed without compromising on coherence and quality is the job. If that tension sounds like an exciting problem to solve rather than a contradiction to complain about, keep reading. What you'll do Build the team. Hire, grow, and retain a high-performing design team, identifying people who will thrive and raise the bar at Amplitude. Develop them through direct, specific feedback. Help someone who's stuck get moving again. Set direction. Drive the design strategy for Amplitude's product suite: Core Amplitude, Statsig, and Wave (a proactive agent that helps teams build self-improving products). Help define how agents and humans work together in the same workflows. Champion craft. Strong visual design builds trust, clarity, and delight in the era of AI. Lead by example, elevating the quality of the product through exceptional design execution. Hold a high bar. AI makes it cheap to ship UI changes, which makes it easy for quality to erode quietly. Set a clear standard for what good looks like and enforce it without becoming an approval bottleneck. Stay in production. You believe that “the design” is what ships. Code is the material that brings the design to life. Push on the details, the papercuts, the edge cases. Speak for the customer. Show up in leadership

Who are we? Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role? We are on a mission to build machines that understand the world and make them safely accessible to all. Data quality is foundational to this process. Machines (or Large Language Models to be exact) learn in similar ways to humans - by way of feedback. By labelling, ranking, auditing, and correcting text output, you will improve Large Language Model’s performance for iterations to come, thus having a lasting impact on Cohere’s tech. Cohere seeks dynamic and dedicated Data Annotators with exceptional Modern Standard German Writing, Translation, and Data Manipulation skill. Please Note: This is a part-time independent contractor position available within Canada . We seek candidates who are able to commit to 16 hours per week minimum at a 30 CAD/hour contract rate. This role is BYOD 💻 - Bring Your Own Device (laptop). Remote work within Canada. 12 months contract. Performance incentives included! As a Data Annotation Specialist, you will: Label and Rank: Accurately label and rank machine learning data with advanced proficiency in German, ensuring data integrity and quality. Audit and Correct: Sc

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. About Replit Replit is building the world’s most ubiquitous AI coding agent. Replit Agent can be used by anybody to bring their ideas to life. Whether it’s an app for yourself, the next great startup idea, or a tool to make you more productive at work, Replit Agent can help build it. Replit is also the leader in secure vibe coding. We protect apps, give users features to manage security risks, and help them vibe code more safely. About the Role In this role you will build powerful tools that help product engineers iterate rapidly on the Agent experience and directly enhance the core Agent itself. You’ll bridge the gap between the AI team (working on the core Agent logic) and the UX team (crafting delightful Agent experiences), enabling both groups to excel within their specialties. This role blends systems engineering, developer experience and product engineering. We tackle complex challenges across the full stack, from browser-based interfaces to high-performance backends to Linux systems engineering. We’re looking for engineers who have a keen sense of the product experience and how to power it with performant systems. On this team, you’ll have the opportunity to grow your skills across our infrastructure and product, and to lead end-to-end efforts with meaningful impact. We value diverse perspectives and encourage candidates from all backgrounds and experiences to apply. You Will Build high-throughput backend applications and services, like streaming chat between user and agent. Design a collaborative "Multiplayer Computer" that lets humans and AI agents work together on shared shells, filesystems, and state—conflict-free and in real time. Develop infrastructure (frontend & backend) that empowers product enginee

A World-Changing Company Palantir builds the world’s leading software for data-driven decisions and operations. By bringing the right data to the people who need it, our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more. The Role As a Deployment Strategist Intern, you'll work as part of a diverse, generative, and creative team of Engineers, Product Designers, and other Deployment Strategists to deploy software against the most important problems our world faces. Your mission is to synthesize disconnected streams of thought into a cohesive understanding of what the problem is, what the data means, what the product needs, what users are motivated by, and where the impact could be. The problems you'll solve will require a curious and analytical mindset, a sharp intuition for product, an ability to speak the language of data, and an understanding that humans should make decisions. At its core, this role is about uncovering dots and—without knowing the shape they form—figuring out how to connect them. Our customers come to us with a hunch that the only way to protect their troops, manufacture high-quality products, structure effective healthcare policies, or deliver aid to refugees safely is to make better use of their data. Deployment Strategists are responsible for turning that hunch into reality. Deployment Strategists are do-ers who thrive in ambiguity, react quickly to new stimuli, and don't equate pivoting with failing as they operate in pursuit of truth, value, and meaningful impact. In this internship, you'll immerse yourself in intricate customer workflows, explore data using quantitative analytics across large datasets, and dive into the product landscape. If you believe in the transformational power of data and technology and in your own ability to bring that power to bear against complex problems, we want to meet you. No two days are the same, but as a Deployment Strategist

A World-Changing Company Palantir builds the world’s leading software for data-driven decisions and operations. By bringing the right data to the people who need it, our platforms empower our partners to develop lifesaving drugs, forecast supply chain disruptions, locate missing children, and more. The Role As a Deployment Strategist Intern, you'll work as part of a diverse, generative, and creative team of Engineers, Product Designers, and other Deployment Strategists to deploy software against the most important problems our world faces. Your mission is to synthesize disconnected streams of thought into a cohesive understanding of what the problem is, what the data means, what the product needs, what users are motivated by, and where the impact could be. The problems you'll solve will require a curious and analytical mindset, a sharp intuition for product, an ability to speak the language of data, and an understanding that humans should make decisions. At its core, this role is about uncovering dots and—without knowing the shape they form—figuring out how to connect them. Our customers come to us with a hunch that the only way to protect their troops, manufacture high-quality products, structure effective healthcare policies, or deliver aid to refugees safely is to make better use of their data. Deployment Strategists are responsible for turning that hunch into reality. Deployment Strategists are do-ers who thrive in ambiguity, react quickly to new stimuli, and don't equate pivoting with failing as they operate in pursuit of truth, value, and meaningful impact. In this internship, you'll immerse yourself in intricate customer workflows, explore data using quantitative analytics across large datasets, and dive into the product landscape. If you believe in the transformational power of data and technology and in your own ability to bring that power to bear against complex problems, we want to meet you.

GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital transformation. More than 50 million registered users and more than 50% of the Fortune 100* trust GitLab to ship better, more secure software faster. The same principles built into our products are reflected in how our team works: we embrace AI as a core productivity multiplier, with all team members expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact. GitLab is where careers accelerate, innovation flourishes, and every voice is valued. Our high-performance culture is driven by our values and continuous knowledge exchange, enabling our team members to reach their full potential while collaborating with industry leaders to solve complex problems. Co-create the future with us as we build technology that transforms how the world develops software. * Fortune 500® is a registered trademark of Fortune Media IP Limited, used under license. Claim based on GitLab data. Fortune 100 refers to the top 20% ranked companies in the 2025 Fortune 500 list, published in June 2025. Fortune and Fortune Media IP Limited are not affiliated with, and do not endorse products or services of GitLab. An overview of this role As a Senior Product Manager , you will own GitLab's Secret Detection offering and the Vulnerability Research function that produces the detection content across security products. Secret Detection is one of the highest-signal, highest-volume security capabilities on the platform. Leaked credentials are the most common initial access vector in real breaches, and as AI agents write, commit, and configure more of the software, the surface area for exposed secrets grows faster than the number of humans watching it. You will own the full loop: detect a secret with high precision, tell the customer whether it i

Fin is the AI Customer Agent company on a mission to help businesses provide perfect customer experiences. Our AI Agent Fin is the highest-performing AI Customer Agent on the market today, enabling businesses to deliver impeccable, always-on customer support across the customer journey – from service, to sales, to ecommerce. Powered by our own AI models, Fin resolves complex customer issues end-to-end across every channel, with minimal set-up and integration. Fin can also be combined with our natively integrated Intercom help desk for one single system that is designed to meet the needs of modern day support teams. Founded in 2011, Fin became one of the fastest growing companies and remains one of the largest private software companies in the world with nearly 30,000 global businesses using our products to transform their customer support. Driven by our core values, we push boundaries, build with speed and intensity, and relentlessly deliver incredible value to our customers. What's the opportunity? With the advent of AI, the nature of customer support has changed dramatically - as have the humans who provide it. Our next Technical Support Specialist will be someone who thrives in autonomy, is deeply curious, fluent in AI tools and holds an extremely high bar not just for themselves, but for their teammates. As a Technical Support Specialist, you'll own complex technical queries around some of our most nuanced features, serve as a point of escalation for your colleagues for the most tricky investigations, consistently provide world class customer support, collaborate with our customers proactively to help them make the most of our platform and help drive improvements in product, processes, and peer development. You'll use AI tools alongside your own expertise to handle complex cases efficiently and play a role in shaping a culture of continuous improvement. We ship product updates consistently, so if constant learning excites and energises you, this environmen

Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma’s platform helps teams bring ideas to life—whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world. If you're excited to shape the future of design and collaboration, join us! We are looking for a Content Support Engineer to own the knowledge systems, strategy, and infrastructure that enable our Support team to deliver exceptional customer experiences at scale — ensuring every product launch, policy change, and process update is understood and executed by our Support Specialists and AI systems. You'll design the architecture that organizes how knowledge lives, build and optimize the automated workflows that keep it accurate at scale, and ensure both humans and AI can retrieve the right information at the right time. This is a role for someone who thinks in systems, measures everything, and wants to own a function end to end. This is a full time role that can be held from one of our US hubs or remotely in the United States. What you'll do at Figma: Design, build, and iterate on AI-powered con