Human Evaluator in United States

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. Role Overview We are searching for a self-motivated individual who is passionate about Roblox and thrives in detail. You will be responsible for classifying content based on a specific set of guidelines to help us improve Roblox systems. As a Human Evaluator you will have the opportunity to provide us direct feedback on the performance of various products across the company, but primarily for features within the studio and creator store. You will: Classify content based on a set of instructions; Content classification includes the review and classification of text, image, video, scripts, and audio Track and document insights and trends related to annotation projects Test out new features within studio and provide detailed feedback Become an expert in a variety of topics to enable more accurate evaluations Dedicate between 25-29 hours per week with a work schedule from Monday to Friday You have: At least 1+ years of experience developing on Roblox or other creation platforms. Solid knowledge of the technical aspects of Roblox Studio. Proficiency with Lua Strong verbal and written communication skills Demonstrated patience for repetitive tasks and attention to detail Effective time mana

Human Factors Engineer Description - This role is responsible for shaping the usability and overall user experience of physical products or systems. The role requires a strong blend of human factors expertise, research rigor, and user empathy. The role collaborates closely with cross-functional teams, including designers, product managers, engineers, and stakeholders, to create user-centered design solutions that meet both user needs and business objectives. The role conducts thorough user research to uncover insights, behaviors, and pain points, translating findings into actionable design solutions. Responsibilities Plans, designs, and conducts usability studies and human factors evaluations across multiple stages of product development. Executes formative and summative usability testing, including protocol development, participant recruitment, data collection, and analysis. Leads human factors validation activities in alignment with relevant standards. Applies quantitative and qualitative research methods to generate actionable insights. Analyzes research data and translates findings into clear, practical design recommendations. Prepares and delivers research reports and presentations for cross-functional stakeholders. Collaborates with industrial designers, engineers, and product managers to integrate user insights into product development. Benchmarks product usability and experience against key competitors. Identifies and surfaces usability risks early in development to reduce downstream cost and rework. Supports continuous improvement of research processes, tools, and best practices. Education & Experience Recommended Four-year or Graduate Degree in Design, Human Fac

Who We Are Notion is the collaborative AI workspace where teams and agents think together . We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion. Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work. About the Role: We’re seeking an experienced UX Researcher to define and scale how we evaluate Notion’s AI-powered experiences—focusing on what “good” looks like not only for model output quality, but for the end-to-end product experience where people discover, set goals, delegate work, review results, and build trust over time with AI. This role sits at the intersection of research craft and evaluation operations: you’ll run studies that uncover user mental models, expectations, and failure/recovery behaviors, then translate those insights into reusable rubrics, workflows, and measurement approaches that product, design, engineering, and data science can apply consistently. This role can be based in either San Francisco or New York City. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days. What You'll Achieve: Define what “good” looks like (frameworks & rubrics): Establish clear, reusable evaluation criteria that reflect real user expectations—helpfulness,

Test & Evaluation Lab Tech (Electrical/Electronic Equipment) Company: The Boeing Company The Boeing Company has an exciting opportunity for Test & Evaluation Lab Tech to join Boeing Defense, Space & Security (BDS) in Berkeley, MO. As a Boeing employee you’ll be part of a winning team that does great things every day BDS is a global leader in the development, production, maintenance and enhancement of fixed-wing and rotary wing aircraft, commercial and government satellites, human spaceflight programs and weapons. Key markets include aeronautics, space and weapons. Core capabilities are in development, production and mission enabling upgrades of integrated solutions. BDS delivers the most digitally advanced, simply and efficiently produced and intelligently supported solutions to its customers. Position Responsibilities: • Installing and maintaining Flight Simulation equipment consisting of computers, monitors, audio systems, visual systems, crew stations and rack equipment • Troubleshoot and resolve electronic hardware issues with minimum down time • Plans, conducts and documents tests on products, systems, components, materials, and manufacturing processes and technologies • Designs, fabricates, builds and analyzes test systems, materials, components, software and methodologies • Coordinates test requirements, determines test methods and capabilities • Conducts system test preparations by coordinating system integration, fabrication, modification, setup, checkout and tear-down • Troubleshoots and resolves laboratory and test-related problems • Coordinates test laboratory activities and maintenance and repairs • Coaches and trains others • Works under general direction </

Test & Evaluation Lab Tech (Electrical/Electronic Equipment) Company: The Boeing Company Job Description The Boeing Company has an exciting opportunity for Test & Evaluation Lab Tech to join Boeing Defense, Space & Security (BDS) in Berkeley, MO. As a Boeing employee you’ll be part of a winning team that does great things every day BDS is a global leader in the development, production, maintenance and enhancement of fixed-wing and rotary wing aircraft, commercial and government satellites, human spaceflight programs and weapons. Key markets include aeronautics, space and weapons. Core capabilities are in development, production and mission enabling upgrades of integrated solutions. BDS delivers the most digitally advanced, simply and efficiently produced and intelligently supported solutions to its customers. Position Responsibilities: • Installing and maintaining Flight Simulation equipment consisting of computers, monitors, audio systems, visual systems, crew stations and rack equipment • Troubleshoot and resolve electronic hardware issues with minimum down time • Plans, conducts and documents tests on products, systems, components, materials, and manufacturing processes and technologies • Designs, fabricates, builds and analyzes test systems, materials, components, software and methodologies <span style

About the Team The Human Data team turns human feedback into reliable signals for training and evaluation. We design and run end-to-end programs that capture the depth of human intent behind everyday and high-stakes uses of our models. Our remit spans bespoke data campaigns, scalable synthetic data generation, and product-embedded signals. We partner closely across all research teams to translate these signals into training datasets, novel evaluations, and feedback loops that push the frontier of our models and advance their applications. About the Role As a Program Manager (PGM) in the Human Data team you will partner with our research teams, operations and engineering to execute complex programs for collecting high-quality data. You will be a key interface between our external vendors and AI trainers, ensuring human data campaigns are successfully completed. Your work will play a key role in enabling OpenAI to train safe models that will land in the real world This role is based in our San Francisco HQ. In this role, you will: Work in a high velocity environment, where the outcome of your work will have a direct impact on the models that OpenAI deploy in the real world Work closely with external vendors, trainers and internal researchers to collect, review, and deliver high-quality data Gather requirements, write instructions, define success criteria, and calibrate the AI trainers Use internal tooling to assess labeled data and provide feedback to AI trainers Think critically and share recommendations on tooling and process improvements, optimizing for quality, throughput, and AI trainer experience You’ll thrive in this role if: You thrive in dynamic environments. You are comfortable navigating ambiguity, managing shifting priorities, and adapting to fast-paced changes without missing a beat. You’re curious about AI, LLMs, Agents. While not required, an interest or background in these areas will help you connect the dots in our broader mission. You have a can-do a

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As Senior Technical Program Manager for the Human-in-the-Loop (HITL) Operations , you will own the end-to-end lifecycle of Roblox's AI data labeling and model evaluation ecosystem. This is a high-leverage, high-ownership role at the intersection of Research, Engineering, and Operations. You will scale our data operations that is handling million of items evaluated annually today with projection of 4x the current volume by 2030, managing a multi-million annual budget and a distributed workforce of 100+ remote contractors — all while driving the platform evolution from manual workflows to AI-assisted, LLM-accelerated operations. This is a rare opportunity to build the data infrastructure that directly determines the quality of Roblox's AI models across creator tools, content discovery, in-experience AI, and 3D generative content — at the exact moment when human judgment is the critical ingredient for getting these models right. You Will Lead data programs end-to-end. Own the full lifecycle of labeling and model evaluation workflows across Roblox's AI teams — from translating ambiguous ML requirements into structured annotation tasks, to overseeing contractor execution, quality review, a

Human Engineer (Associate or Experienced) Company: The Boeing Company Are you ready to join a team of innovators, strategic disruptors, and dreamers who dare to redefine the future of the defense industry? Are you driven to create the unimaginable? Are you passionate about addressing human performance within the design and development of cutting-edge aerospace systems? Boeing Defense, Space & Security (BDS ) has an exciting opportunity for an associate or experienced Human Engineer to join our Human Engineering team in Colorado Spings, CO working on the front end of the next generation of Space Mission Systems technology. As a Human Factors Engineer, you will play a pivotal role in the design, development, and integration of complex aerospace systems, ensuring that all components function cohesively to meet user needs and operational requirements. You will collaborate with cross-functional teams, including systems engineers, software developers, and hardware specialists, to create systems that are not only technically robust but also intuitive and user-friendly, enhancing overall efficiency and user satisfaction. Position Responsibilities : Apply Human Engineering knowledge and principles to the analysis, design, and evaluation of complex systems Define system performance requirements to ensure safe and successful human operation of physical, functional, and program interfaces for all system operators Apply human performance principles, methodologies, and technologies to the design of complex systems Develop and implement research methodologies and analysis plans to test and evaluate developmental prototypes Apply a knowledge of military standards (such as MIL-STD-1472, MIL-STD-1474,

About the Team OpenAI's Human Data Team creates custom data solutions driving groundbreaking research. Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-5, and Sora, and contributes to safety initiatives through collaboration with our Preparedness and Safety Systems teams. About the Role As a Research Program Manager (RPM) in the Human Data team you will partner with research and engineering to design and implement pragmatic solutions for collecting high-quality data. You will be a key interface between our research roadmap, external vendors, AI trainers, and the Human Data engineering team. This role is based in our San Francisco HQ. In this role, you will: Collaborate with Research: Partner with researchers to scope data collection needs, define success metrics, and establish quality measurement frameworks. Design & Execute Data Collection Campaigns: Translate research needs into actionable plans and accelerate execution by leveraging existing tooling and iterating to reach the desired outcome. In many cases, you will need to implement scrappy new solutions while partnering with engineering to design robust/scalable solutions. Unblock Yourself: You must be deeply uncomfortable with the idea of sitting around waiting for external dependencies, and have the technical acumen and drive to figure out how to achieve at least partial success in the interim. Optimize Systems & Processes: Build and optimize dashboards to track campaign performance, leveraging SQL and Python for data analysis and actionable insights. Drive Technical Roadmaps: Collaborate with engineers to enhance data platforms, resolve blockers, and ensure security best practices such as access management. Scale Your Impact : Advise and empower program managers and vendors to drive day-to-day execution so that you can focus on addressing high priority opportunities. You might thrive in this role if you: Are proficient in SQL and Python for data analysis, including q

About the Team OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all of humanity. A key part of achieving that mission is training models that deeply understand and reflect human preferences — the Human Data team is at the heart of that effort. The Human Data engineering team creates the systems that enable scalable, high-quality human feedback. These systems are essential to how OpenAI trains and improves its most advanced models. Engineers on this team collaborate closely with world-class researchers to bring alignment techniques to life — from experimental ideas to production-ready feedback loops. About the Role We’re looking for software engineers to join the Human Data team and build the platforms, prototypes, tools, and infrastructure that power how our AI models are trained, aligned, and evaluated. You’ll partner with researchers and cross-functional teams to bring alignment ideas to life, influence future model training, and shape how models interact with the real world. We’re looking for people who are excited by technical ownership, enjoy working across the stack, and are eager to solve ambiguous problems in a high-impact, fast-paced environment. This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees. In this role, you will: Build and maintain robust full-stack systems for feedback collection, data labeling, and evaluation pipelines, while maintaining high levels of security. Translate experimental alignment research into scalable production infrastructure, including inference and model training stacks. Design and iterate on user-facing tools and backend services to support high-quality data workflows Partner with researchers, engineers, and program leads to shape feedback loops and model interaction paradigms Drive infrastructure improvements that enable faster iteration and scaling across OpenAI’s frontier models, from internal r

Job Responsibilities Provide leadership and guidance to team members to achieve performance targets. Monitor and evaluate team performance, providing regular feedback and coaching. Collaborate with other departments to ensure seamless workflow and quality service delivery. Identify and resolve operational issues and guest concerns promptly. Promote a positive and inclusive team culture, fostering associate engagement and development. Ensure compliance with company policies, procedures, and safety standards. Prepare and present regular reports on team performance and key metrics. Perform other operational duties as assigned by your manager Critical Skills & Experience Requirements High school diploma or equivalent work experience. High volume restaurant experience preferred Proven experience in a leadership or supervisory role. Excellent communication, interpersonal, and problem-solving skills. Strong organizational and time-management abilities. Ability to motivate and inspire a diverse team to achieve goals. Proficiency in relevant software and tools. Knowledge of industry-specific regulations and best practices. Availabilit to work varied shifts, including evenings, weekends and holidays Ability to stand and walk for long periods of time including maneuvering up and down stairs BENEFITS Free Play & 1/2 price food! Health, dental, vision, 401(k) team member match, free mental well-being platform – and that’s just for starters for those who qualify. View team member benefits here. ADA The above stat

Job Responsibilities Provide leadership and guidance to team members to achieve performance targets. Monitor and evaluate team performance, providing regular feedback and coaching. Collaborate with other departments to ensure seamless workflow and quality service delivery. Identify and resolve operational issues and guest concerns promptly. Promote a positive and inclusive team culture, fostering associate engagement and development. Ensure compliance with company policies, procedures, and safety standards. Prepare and present regular reports on team performance and key metrics. Perform other operational duties as assigned by your manager Critical Skills & Experience Requirements High school diploma or equivalent work experience. Proven experience in a leadership or supervisory role. Excellent communication, interpersonal, and problem-solving skills. Strong organizational and time-management abilities. Ability to motivate and inspire a diverse team to achieve goals. Proficiency in relevant software and tools. Knowledge of industry-specific regulations and best practices. Availabilit to work varied shifts, including evenings, weekends and holidays Ability to stand and walk for long periods of time including maneuvering up and down stairs MIN 20.00 - MAX 22.00 Position Pay Range BENEFITS Free Play & 1/2 price food! Health, dental, vision, 401(k) team member match, free mental well-being platform – and that’s just for starters for those who qualify. View team member benefits here. ADA The above statements cover wha

Job Responsibilities Provide leadership and guidance to team members to achieve performance targets. Monitor and evaluate team performance, providing regular feedback and coaching. Collaborate with other departments to ensure seamless workflow and quality service delivery. Identify and resolve operational issues and guest concerns promptly. Promote a positive and inclusive team culture, fostering associate engagement and development. Ensure compliance with company policies, procedures, and safety standards. Prepare and present regular reports on team performance and key metrics. Perform other operational duties as assigned by your manager Critical Skills & Experience Requirements High school diploma or equivalent work experience. Proven experience in a leadership or supervisory role. Excellent communication, interpersonal, and problem-solving skills. Strong organizational and time-management abilities. Ability to motivate and inspire a diverse team to achieve goals. Proficiency in relevant software and tools. Knowledge of industry-specific regulations and best practices. Availabilit to work varied shifts, including evenings, weekends and holidays Ability to stand and walk for long periods of time including maneuvering up and down stairs ADA The above statements cover what are generally believed to be principal and essential functions of the job. Specific circumstances may allow or require some incumbents assigned to the job to perform a different combination of duties. EEO Statement Topgolf is a global sports and entertainment community and is committed to equal opportunity and is firmly committed to preventing discrimination and harassment, includi

Job Responsibilities Provide leadership and guidance to team members to achieve performance targets. Monitor and evaluate team performance, providing regular feedback and coaching. Collaborate with other departments to ensure seamless workflow and quality service delivery. Identify and resolve operational issues and guest concerns promptly. Promote a positive and inclusive team culture, fostering associate engagement and development. Ensure compliance with company policies, procedures, and safety standards. Prepare and present regular reports on team performance and key metrics. Perform other operational duties as assigned by your manager Critical Skills & Experience Requirements High school diploma or equivalent work experience. Proven experience in a leadership or supervisory role. Excellent communication, interpersonal, and problem-solving skills. Strong organizational and time-management abilities. Ability to motivate and inspire a diverse team to achieve goals. Proficiency in relevant software and tools. Knowledge of industry-specific regulations and best practices. Availabilit to work varied shifts, including evenings, weekends and holidays Ability to stand and walk for long periods of time including maneuvering up and down stairs BENEFITS Free Play & 1/2 price food! Health, dental, vision, 401(k) playmaker match, free mental well-being platform – and that’s just for starters for those who qualify. View playmaker benefits here. ADA The above statements cover what are generally believed to be principal

Job Responsibilities Provide leadership and guidance to team members to achieve performance targets. Monitor and evaluate team performance, providing regular feedback and coaching. Collaborate with other departments to ensure seamless workflow and quality service delivery. Identify and resolve operational issues and guest concerns promptly. Promote a positive and inclusive team culture, fostering associate engagement and development. Ensure compliance with company policies, procedures, and safety standards. Prepare and present regular reports on team performance and key metrics. Perform other operational duties as assigned by your manager Critical Skills & Experience Requirements High school diploma or equivalent work experience. Proven experience in a leadership or supervisory role. Excellent communication, interpersonal, and problem-solving skills. Strong organizational and time-management abilities. Ability to motivate and inspire a diverse team to achieve goals. Proficiency in relevant software and tools. Knowledge of industry-specific regulations and best practices. Availabilit to work varied shifts, including evenings, weekends and holidays Ability to stand and walk for long periods of time including maneuvering up and down stairs Min 25.00 - Max 28.00 Position Pay Range BENEFITS Free Play & 1/2 price food! Health, dental, vision, 401(k) team member match, free mental well-being platform – and that’s just for starters for those who qualify. <span style="color:#000