Human Evaluator Jobs

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is looking for a Solutions Engineer who has a passion for solving complex challenges. In this role, you will partner directly with Sales to provide technical leadership throughout the sales process, navigating intricate deal cycles, providing value-based demonstrations, and serving as a trusted advisor to prospects and clients throughout their evaluations. You will be a member of the Solutions Engineering organization that is on a mission to grow revenue and to empower anyone to drive meaningful change — for themselves, their businesses, and the world. This role is part of the Worldwide Field Operations Commercial Solutions Engineering team and can be based remotely from anywhere in the US where Smartsheet is a registered employer. You Will: Identify, build, and demonstrate Smartsheet solutions that solve specific customer problems Lead technical sales engagements, collaborate closely with sales teams to identify customer requirements, summarize challenges, design solutions, and provide technical guidance that demonstrates the value of Smartsheet's offerings Conduct discovery sessions to uncover business processes and challenges to inform the sales and solution delivery strategy Partner with Sales, Product, Customer Success, and

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Senior Enterprise Security Engineer, you will play a critical role in executing Roblox’s Enterprise Security Strategy. You will design, deploy, and manage security solutions to protect Roblox’s corporate infrastructure and ensure secure, compliant operations across the organization. Working closely with Corporate Engineering and Trust & Safety teams, you will translate business requirements into robust security implementations that enable secure productivity while mitigating risk. You will be reporting directly to the Senior Manager of Enterprise Security Engineering. You'll partner with security professionals across the Information Security organization, and work cross-functionally with teams throughout Roblox to drive security initiatives that scale with our business. You will: Evaluate and implement security technologies and vendor solutions to ensure alignment with enterprise security requirements, compliance standards, and overall risk management strategy Lead and drive initiatives across core security domains, including Endpoint Security, SaaS Security, Identity & Access Management (IAM), Agentic AI Governance, and Supply Chain Security. Collaborate closely with IT, engin

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Principal Security Software Engineer in the Enterprise Security team, you will advance Roblox's Enterprise Security strategy by building the systems and integrations that protect Roblox's corporate infrastructure. Where traditional security engineers evaluate and deploy vendor solutions, you will design and build production-grade security software - Identity and Access governance, policy enforcement engines, and security integrations that scale with Roblox's business. You'll partner with security professionals across InfoSec and work cross-functionally with Corporate Engineering, DevOps, and Product teams to drive security initiatives. You will: Build identity and access systems : Design, implement, and own integrations across Roblox's IAM ecosystem, including SSO federation, SCIM provisioning/deprovisioning pipelines, OAuth 2.0 authorization servers, and token lifecycle management. Lead security automation : Develop production-quality tools and services that enforce security policies at scale, replace manual workflows, and surface actionable signals Drive secure-by-design implementations : Partner with Corporate Engineering, DevOps, and product teams to embed security controls in

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a member of the Infrastructure Foundation Hardware Engineering team, you will play a key role in enabling our mission to deliver a reliable, high-performing, and cost-efficient infrastructure that powers the world’s play. In this specialized role, you will be the technical lead for our GPU and AI accelerator ecosystem. You will be responsible for the full lifecycle of GPU hardware, from initial architectural evaluation and firmware qualification to large-scale fleet integration and performance tuning. You will ensure that Roblox’s massive-scale rendering and ML workloads run on the most optimized and stable hardware possible. You Will: Architect & Prototype: Prototype next-generation GPU-accelerated hardware platforms, ensuring seamless integration between high-density compute nodes, high-speed interconnects (NVLink/PCIe Gen5/6), and system firmware. GPU Optimization: Drive the integration, performance testing, and debugging of GPUs in our fleet, focusing specifically on hardware-level optimizations, driver tuning, and thermal/power management. Validation & Certification: Develop and execute rigorous evaluation and stress-testing strategies for GPU-heavy server platforms to ensur

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a member of the Infrastructure Foundation Hardware Engineering team, you will help develop and validate next-generation server platforms that power a reliable, high-performing, and cost-efficient infrastructure at scale. You will work across platform bring-up, firmware qualification, hardware validation, fleet integration, and performance optimization to support large-scale production deployments. You Will: Bring-up & Sustaining: Drive key aspects of the hardware development lifecycle, including feasibility studies, hardware bring-up, validation, deployment, and ongoing production support. Platform Optimization: Perform platform integration, performance characterization, and system-level debugging across compute infrastructure, focusing on hardware optimization, driver tuning, and thermal/power efficiency. Hardware Validation: Develop and execute rigorous evaluation and stress-testing strategies for server platforms to ensure reliability and performance under production-scale workloads. Firmware & Fleet Enablement: Support BIOS/BMC firmware qualification, hardware health monitoring, and automation tooling for firmware deployment and lifecycle management. Vendor & Cross-Functi

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Consumer App Foundation team is looking for a Senior Software Engineer to improve Roblox performance across our supported platforms. Our platform is used by over 150M people daily. You will work primarily in cross-platform C++ Roblox game engine code, partnering with teams across the company to identify bottlenecks, ship measurable improvements, and prevent regressions over time. Our team is responsible for the end-to-end performance of the Roblox client on Android, iOS, macOS, Windows, Xbox, Playstation and TV. We tackle the platform’s most difficult technical constraints, ranging from speed and responsiveness to efficiency and stability, to ensure a high-performance experience across all of our devices and platforms. You will: Own the end-to-end performance of the Roblox client across all platforms directly moving the needle on our key metrics: TTI, app size, battery usage and more. Build the benchmarking, profiling, and telemetry systems required to catch regressions early and continue scaling our application even further. Collaborate closely with stakeholders across Engineering, Product, and Design to evaluate technical tradeoffs, align on company-wide priorities, and

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. About the role: Roblox Studio is the creation engine behind millions of immersive 3D games built by creators around the world. We are entering a major platform transition, evolving Studio into an AI-Native IDE where intelligent systems plan, act, validate, and iterate alongside creators to amplify their output. We are seeking a Technical Director, Applied AI to own the technical direction of the agentic AI systems embedded deeply into Roblox Studio. This is a hands-on individual-contributor role operating at the intersection of platform engineering, AI systems, and developer experience, where you will set the technical direction, design the architecture, and write the code. You will: Design and build the agentic AI systems at the core of the AI-Native IDE (planning, execution, validation, evaluation) so they are production-grade and reliable. Decide how Studio uses coding models, agents, retrieval, tool calling, and context management across the Assistant and the broader creator workflow. Own the quality bar and evaluation strategy , standing up quantitative and qualitative eval pipelines, including human-in-the-loop, so we ship with confidence. Optimize these systems for latency, throughpu

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Infrastructure Compute Site Reliability Engineering mission is to own and manage the successful operation of our underlying cell infrastructure system, along with elements of service discovery, secrets management and related software layers. We’re looking for a skilled Senior Site Reliability Engineer with strong programming skills to help us build Roblox's private cloud, productionize our growing Kubernetes-based infrastructure, and institute reliability best practices across the Roblox Compute team. You will: Design and Develop systems & libraries that promote fault-tolerance and resilience, automate much of the management and lifecycle of our clusters, and ensure systems are observable. Promote and Institute reliability best practices across the Infra Compute group, drive common reliability initiatives. Provides collaborative technical reviews and operational guidance to strengthen system reliability. Build, Automate and Standardize process automation to create a "golden path" of tooling and platform support that powers the fundamental Roblox ecosystem. Create Tooling that provides production guardrails, by evaluating release candidate capacity with load testing tooling before de

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Senior Privacy Engineer on the Security and Privacy GRC team, you will shepherd and grow Roblox's Privacy Review Program, the technical and process backbone that ensures privacy is engineered into every product before it ships. You will join the Security and Privacy GRC team, reporting to the Sr Engineering Manager for Privacy Governance and Operations. This is a hands-on privacy engineering role: you'll set standards for privacy-enhancing technologies, act as the go-to SME for policy-as-code, and partner closely with Product teams, Trust & Safety, Legal, and Regulatory Compliance as part of Roblox's broader cross-functional privacy program. You will: Shepherd and evolve the Privacy Review Program, designing consistent intake workflows, evaluation criteria, and documentation to systematically assess privacy risk across new products, features, and infrastructure changes. Develop standards and guidelines that enable product teams to adopt privacy-enhancing technologies (PETs) such as differential privacy, k-anonymity, data minimization, and secure computation, with clear guidance on trade-offs and implementation patterns. Act as the Privacy SME for policy-as-code, translating privacy

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Consumer App Foundation team is looking for a Senior Software Engineer to improve Roblox performance across our supported platforms. Our platform is used by over 150M people daily. You will work primarily in cross-platform C++ Roblox game engine code, partnering with teams across the company to identify bottlenecks, ship measurable improvements, and prevent regressions over time. Our team is responsible for the end-to-end performance of the Roblox client on Android, iOS, macOS, Windows, Xbox, Playstation and TV. We tackle the platform’s most difficult technical constraints, ranging from speed and responsiveness to efficiency and stability, to ensure a high-performance experience across all of our devices and platforms. You will: Own the end-to-end performance of the Roblox client across all platforms directly moving the needle on our key metrics: TTI, app size, battery usage and more. Build the benchmarking, profiling, and telemetry systems required to catch regressions early and continue scaling our application even further. Collaborate closely with stakeholders across Engineering, Product, and Design to evaluate technical tradeoffs, align on company-wide priorities, and

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a User Researcher on the Economy team, reporting directly to the Director of Design Strategy & Operations, you will generate actionable qualitative and quantitative insights that shape core economic systems, developer monetization workflows, and immersive brand experiences. Embedded directly on product teams, you will conduct generative and evaluative research centered on how creators build businesses, how brands interact on the platform, and how users participate in virtual economies. You will tackle high-priority features, solve complex behavioral puzzles, and help translate multi-faceted economic infrastructure into intuitive, trustworthy, and scalable experiences. Partnering closely with Product, Product Design, Engineering, Data Science, and Policy, you will ensure user needs directly inform platform decisions while driving mutual value across players, creators, and brands. You will: Execute end-to-end mixed-methods research projects to understand creator monetization, payouts, advertising, and commerce workflows, translating abstract user behaviors into clear product feature recommendations. Conduct foundational research to uncover key user pain points, mental models, and behav

Asana is looking for a strategic, supportive, and business-savvy leader to manage our global Administrative Business Partner (ABP) community. This is more than a traditional operations role; it is a people-first leadership position designed to bring stability, clarity, and growth to a team that is the backbone of our executive leadership. You will lead our full-time, global ABPs and oversee our offshore ABP support program. Your mission is to balance operational rigor with a human-centric approach, ensuring our ABPs feel supported in their career journeys while providing world-class partnership to Asana’s top executives. This role is based in our San Francisco office with an office-centric hybrid schedule. The standard in-office days are Monday, Tuesday, and Thursday. Most Asanas have the option to work from home on Wednesdays. Working from home on Fridays depends on the type of work you do and the teams with which you partner. If you're interviewing for this role, your recruiter will share more about the in-office requirements. What you’ll achieve Bridge Strategy & Execution: Act as a high-level partner to the Head of RE & Workplace, ensuring the ABP function aligns with Asana’s broader business goals. Evaluate & Optimize: Conduct comprehensive strategic audits of internal programs. You will determine their long-term viability, impact on executive productivity, and whether to scale, pivot, or maintain the current model. Development of Team: Coach the high-performing team of ABP’s, ensuring they are accountable for the overall quality and consistency of their work, ensuring every interaction is grounded in business context and aligned to company priorities. Establish clear expectations for business acumen, judgment, and impact across the ABP team. Standardize Excellence: Develop a standard of excellence based on the Asana Operating Principles for executive support—creating playbooks for calendar management, global trav

Join us in building the future of finance. Our mission is to democratize finance for all. An estimated $124 trillion of assets will be inherited by younger generations in the next two decades. The largest transfer of wealth in human history. If you’re ready to be at the epicenter of this historic cultural and financial shift, keep reading. About the team + role We are building an elite team, applying frontier technologies to the world’s biggest financial problems. We’re looking for bold thinkers. Sharp problem-solvers. Builders who are wired to make an impact. Robinhood isn’t a place for complacency, it’s where ambitious people do the best work of their careers. We’re a high-performing, fast-moving team with ethics at the center of everything we do. Expectations are high, and so are the rewards. Our Crypto team is dedicated to building and expanding our cryptocurrency offerings, making crypto accessible and understandable for everyone. The team partners closely with engineering, product, and compliance to bring new crypto capabilities to customers while maintaining strong regulatory standards. We focus on making complex blockchain technologies more understandable and accessible to a broad audience. As a Senior Partnerships Leader, you will identify and manage relationships with key participants across the cryptocurrency ecosystem. You will evaluate new partnership opportunities, lead negotiations, and support the launch of integrations that expand Robinhood’s crypto offerings. You will also oversee existing partnerships, ensuring strong performance and alignment with product goals. This role plays a central par

About the Team The Strategic Initiatives & Operations team is in need of a Technical Program Manager (TPM) to streamline our processes, including full safety governance and integration of various safety research and mitigations into our ChatGPT, API, and any frontier models. This role is critical for driving safe deployment of our new models, synthesizing inputs from multiple stakeholders, ranging across research, product, engineering, legal and policy, and ensuring all the risks are effectively and properly monitored, mitigated or resolved. About the Role As a TPM, you will be responsible for critical tasks ranging from tracking safety research progress and risk tables to overseeing the quality of human data campaigns – acting as the connective tissue to enhance the deployment of OpenAI’s safety system. Additionally, you will create and execute a compute roadmap for your team to ensure that our top priorities are resourced while taking advantage of new opportunities to make key safety research discoveries. Your primary focus will be to ensure our models are qualified for safe deployment. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Manage key risk areas and corresponding stakeholders. Keep track of a stack of existing and future mitigations for every major product and model deployment. Standardize the lifecycle of risk assessment, setting safety bars, consolidating inputs from multiple stakeholders across research, product, engineering, legal and policy, pre-launch safety reviews and post-launch followup. Manage pre-launch safety reviews. Share launch calendars and key safety practices and evaluations with our key parter (i.e. Microsoft). Develop comprehensive documentation for all the safety work, including metrics, evaluations, and progress tracking across multiple teams within OpenAI. Help with publishing and open sourcing safety

About the Team The Personal AGI team seeks to empower all of humanity to benefit from frontier intelligence in whatever way they choose. We are responsible for training models to deploy to millions of users globally via ChatGPT, the API, and future products. We aim to evolve ChatGPT from a chatbot to an infinitely capable and personalized superassistant supporting human flourishing. We work on defining, measuring, and improving capabilities across the training stack. Our focus areas include but are not limited to model behavior, personalization, safety, factuality, instruction following, personality, interactivity, multilingual fluency, world interaction, and bringing agents to everyone. We chart the course for what to strive towards. We partner closely with research and product teams across the company ensuring that our models are safe, efficient, and reliable. About the Role You’ll work as a Research Engineer / Scientist on the North Stars team within the broader Personal AGI research org. You will work on bringing the next generation of AI-enabled experiences to all of humanity by closing the capability overhang between power users and the average consumer, including areas like tool-use, feature discovery, connectors, and instruction following. You will think deeply about the current bottlenecks in model behavior, translate these insights into robust evals, training data, reward signals, and model and harness improvements. We're looking for individuals with strong ML engineering skills and research experience passionate about creative, product-driven research. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Own and pursue a research agenda to improve model capability and performance. Collaborate closely with the other research and product teams, allowing customers to optimize their own models. Build robust evaluations for tracking modelin