Human Evaluator Jobs

🚀 About WRITER WRITER is where the world's leading enterprises orchestrate AI-powered work. Our vision is to expand human capacity through superintelligence. And we're proving it's possible – through powerful, trustworthy AI that unites IT and business teams together to unlock enterprise-wide transformation. With WRITER's end-to-end platform, hundreds of companies like Mars, Marriott, Uber, and Vanguard are building and deploying AI agents that are grounded in their company's data and fueled by WRITER's enterprise-grade LLMs. Valued at $1.9B and backed by industry-leading investors including Premji Invest, Radical Ventures, and ICONIQ Growth, WRITER is rapidly cementing its position as the leader in enterprise generative AI. Founded in 2020 with office hubs in San Francisco, New York City, Seattle, Austin, Chicago, and London, our team thinks big and moves fast, and we're looking for smart, hardworking builders and scalers to join us on our journey to create a better future of work with AI. 📐 About this role We’re looking for a collaborative and builder-oriented enterprise sales rep experienced at helping prospective customers at large companies navigate the evaluation, business case development, and procurement of transformative technology. Your objective will be to help convert enterprise prospects (3K-8K FTE, although tilted towards Fortune 1000) who are active in our trials or who request a sales demo from our website or within our product. While most of your pipeline will come inbound, you'll also be responsible for generating pipeline from ideal customer profile (ICP) accounts within your account set. Your positivity, sense of curiosity, and ability to create champions from early adopters in the AI space will help shape our entire culture. This is a hybrid role based out of our London hub. You'll be reporting to our RVP. 🦸🏻‍♀️ Your responsibilities Develop a deep understanding of our users and why they are exploring WRITER Become a trusted product speciali

ABOUT THE TEAM The AI Foundations Team at Mural is pioneering how generative AI transforms visual collaboration and decision-making. We’re a remote-first group of engineers, designers, and product thinkers focused on helping teams work together more effectively. Our goal isn’t to replace human creativity. It’s to amplify it, building AI that enhances how people align, communicate, and make decisions visually. YOUR MISSION You will design and build the core AI systems and platforms that enable Mural’s next wave of agentic, AI-driven collaboration experiences. Rather than building isolated AI features, you’ll work on the core backend systems that power Mural’s agent platform, including agent orchestration, durable execution, contextual memory, tool integration, observability, and evaluation. Your work will enable intelligent agents to reason over product context, act on behalf of users, and operate reliably and safely at scale. Our stack at Mural includes Azure OpenAI, React, Node, MongoDB. WHAT YOU'LL DO Build the core backend systems that power Mural’s agent platform, including orchestration, durable execution, tool execution, memory, observability, and evaluation infrastructure Design scalable services and APIs that allow AI agents to retrieve context, coordinate multi-step workflows, interact with Mural data, and act reliably on behalf of users Develop the agent memory layer, including systems for conversation context, product context, retrieval, summarization, compaction, and long-term context management Create infrastructure to monitor, debug, and improve agent behavior through traces, metrics, feedback loops, and offline evaluation Translate complex, open-ended product needs into clear backend architectures, service boundaries, data models, and implementation plans that align technical capabilities with user value Help define the technical direction for agentic AI at Mural, contributing to long-term architecture and strategy Champion engineering excellence, men

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. Senior Director, Generative AI About the Role Roblox Build is our generative creation product, the platform where creators design, build, and publish 3D experiences. We are looking for a Senior Director of Generative AI to lead the Applied AI organization inside Build, responsible for turning state-of-the-art foundation models into high-quality, reliable creation systems at Roblox scale. This leader will own the full applied AI stack: model strategy and routing, model adaptation and fine-tuning, code generation (CodeGen), 3D layout generation (LayoutGen), and the evaluation science and infrastructure that tells us what actually works. You Will Own model strategy and routing for Build. Design and build an intelligent model layer that selects the right model for each creation task based on quality, capability, latency, cost, and safety, leveraging both frontier models and Roblox-adapted open-source models. Lead model adaptation across the Applied AI org, including fine-tuning, distillation, synthetic data generation, human feedback pipelines, and preference optimization for Roblox-specific creation tasks such as Luau code generation and 3D scene understanding. Drive CodeGen capabilities

About Flexport: At Flexport, we believe global trade can move the human race forward. That’s why it’s our mission to make global commerce so easy there will be more of it. We’re shaping the future of a $10T industry with solutions powered by innovative technology and exceptional people. Today, companies of all sizes—from emerging brands to Fortune 500s—use Flexport technology to move more than $19B of merchandise across 112 countries a year. The recent global supply chain crisis has put Flexport center stage as we continue to play a pivotal role in how goods move around the world. We are proud to have the support of the best investors in the game who believe in our mission, solutions and people. Ready to tackle global challenges that impact business, society, and the environment? Come join us. The opportunity: Reporting to the Head of Finance APAC, the Senior Manager of FP&A will play a pivotal business partnering role and will provide financial direction to senior leadership for the region, while providing leadership on forecasting, business planning, management reporting, business analytics activities along with ad hoc projects when needed. A hands-on and thoughtful team player who can build and oversee the tools and processes to help us scale for the future. He/she will also bring a strong analytical toolset, and experience in managing the planning activities of a global matrix organization. This role also helps to build financial literacy among the APAC Leadership Team with transparency and accountability for driving profitability and opportunity, and ensure that finance is seen as a value-added business partner. You will: Be the business partner for a number of countries within APAC, to support key business decisions and evaluate progress as well as own a number of initiatives across the region such as OPEX and productivity measurement and tracking Manage and participate in the analysis and interpretation of operating results. Preparation of exe

About Flexport: At Flexport, we believe global trade can move the human race forward. That’s why it’s our mission to make global commerce so easy there will be more of it. We’re shaping the future of a $10T industry with solutions powered by innovative technology and exceptional people. Today, companies of all sizes—from emerging brands to Fortune 500s—use Flexport technology to move more than $19B of merchandise across 112 countries a year. The recent global supply chain crisis has put Flexport center stage as we continue to play a pivotal role in how goods move around the world. We are proud to have the support of the best investors in the game who believe in our mission, solutions and people. Ready to tackle global challenges that impact business, society, and the environment? Come join us. The Opportunity The Ocean Planning Manager plays a critical role in Flexport's mission to make global trade easy for everyone. Flexport today connects almost 10,000 clients and suppliers across 109 countries. In this role, your direct customers are the partners within your region — account management, customs teams, subcontractors, carriers, terminals, and trucking providers — and your work directly supports Flexport's end customers in achieving their logistics goals. This role is designed to hire, coach, and develop a team of planning and booking experts while navigating the complex world of freight forwarding. You'll work cross-functionally to establish a world-class planning and booking function, balancing tactical execution with adaptive, strategic leadership as Flexport builds its next operational model. What You'll Do Team Leadership Hire, coach, and develop a 5–10 person team responsible for end-to-end planning and booking of ocean freight shipments (imports and exports, including intermodal movements) Manage team performance and evaluation, guiding the team to meet OKRs and SLAs Provide direction on training and onboarding for new team members Empowe

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. About the Role The AI Platform Engineering team is looking for a highly motivated and talented engineer who are passionate about continuous learning and excited to grow in a fast-paced, innovative environment. We are an agile team that operates iteratively, focused on building high-quality software and adhering to rigorous operational best practices across complex, cross-functional distributed systems. This full-time position reports to a Software Engineering Manager and can be located in our Bellevue, WA office, or you may work remotely from anywhere in the US where Smartsheet is a registered employer. What You'll Do Build the AI Platform Foundation : Lead the design and ownership of the core infrastructure that serves as the backbone for all Smartsheet AI experiences. Focus on building a robust, multi-tenant environment that reduces friction for internal teams, allowing them to deploy reliable and scalable AI features with ease. Standardize the AI Developer Path : Architect high-level abstractions and "Golden Path" APIs that democratize AI development across Smartsheet. By insulating product teams from infrastructure complexity, you will enable them to ship intelligent features with high velocity while guaranteeing safety and consistency at scale. Engineer AI Trust & Safety Systems : Establish the mission-critical monitoring and quality assurance layers that protect Smartsheet customers. By creating rigorous evaluation pipelines, you will ensure every AI-driven feature meets the high bar for safety, data privacy, a

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is hiring a Senior Machine Learning Operations Engineer to architect our machine learning production lifecycle. Your mission is to maintain and deploy ML models to a scalable, reliable, and secure production environment. You will design and maintain the infrastructure, automation, and monitoring systems that ensure our AI products are high-performing and cost-effective. You will report to our Director, Analytics Engineering & Data Governance and work from our Bangalore, India office. You Will: Model and Pipeline Automation Automate the deployment and retraining of ML models, from training through to production inference, by building and managing complete CI/CD/CT (Continuous Training) pipelines, adhering to MLOps best practices. Build, fine-tune, or use pre-trained LLMs, deep learning models or traditional machine learning models. Evaluate and recommend AI or ML solutions for the product using any combination of vendor solutions and/or custom-built models. Governance & Compliance Implement model versioning, lineage tracking, and auditing to ensure compliance with security and ethical standards. Performance Monitoring Continuously monitor the health and performance of production machine learning models, proactively identifying and correcting model drift, staleness, and performance degradation. Incorporate user feedback for iterative improvements and manage necessary model retraining cycles. Cross-Functional Collaboration Act as the "glue" between Data Scientists (who build models

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Role Overview: As the Sr. CE Program Manager, you will serve as a strategic partner to the Sr. Director, Customer Adoption Programs & Strategy in leading customer excellence programs. This role combines day-to-day program management with strategic planning, cross-functional collaboration, and performance analytics. You will drive strategic customer experience initiatives, evaluate new tools and processes, own performance reporting, and build frameworks that inform how we optimize for adoption, retention, and customer value. This role requires both tactical execution excellence and the ability to think strategically about how business changes impact our customer experience programs, while partnering closely with BI, Customer Success, Scale, Product Marketing, and other teams to deliver unified customer experiences. You Will: Strategic Leadership & Representation Represent the customer experience in field meetings and cross-functional forums, ensuring customer experience strategies align with broader Customer Success objectives Serve as a strategic thought partner to the Customer Experience and Customer Success teams on opportunities and decision-making that further enhance the customer journey Support the evaluation of tools, processes, and workflows Partner with business partner leadership on strategic initiatives that bridge digital and scale motions Present on behalf of leadership in internal and external forums Program Strategy & Execution Design, build, and iterate multi-channel customer engagement

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. We are seeking a seasoned and strategic Senior Product Manager II to join our Product Organization. You will be instrumental in developing new capabilities that address the advanced project management needs of our most valuable and strategic customers, driving meaningful change in their organizations. You will be focused on delivering a completely new Gantt View experience. You will work closely with a dedicated engineering team based in Bangalore and report to the US-based Group Product Manager for Core Views experiences. You Will: Product Strategy & Vision: Define, prioritize, and drive the future roadmap for project management capabilities, ensuring alignment with customer needs, company objectives, and industry best practices. Feature Delivery: Own the end-to-end delivery of a new, scalable Gantt View experience, ensuring a compelling, high-quality customer experience that sets a new bar for Smartsheet. Cross-Functional Partnership: Work in close partnership with design, engineering, user research, data science, and analytics teams to create, test, and launch proposed new capabilities and validate hypotheses. Data-Driven Decision Making: Leverage internal and external data to guide experimentation, make informed product decisions, and evaluate the success of launches. Executive Alignment: Regularly present product strategy, concepts, health metrics, and roadmap updates to senior leadership to educate, influence, and align stakeholders across the company. Product Leadership: Operate effectively at both strategic a

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. As a Customer Success Renewals Manager , you will support the Customer Success Organisation in driving a portfolio of customer renewals. The ideal Renewal Manager is an experienced professional with a full understanding of the SaaS contracting space. You will resolve a range of issues in creative ways while mitigating risk and providing a world-class renewal experience for our customers. This key role is part of the Customer Success team , reporting to the Manager, Customer Success , and is based in London (Hybrid eligible). You Will: Serve as an extended member of the Smartsheet account team to bring all assigned contract renewals to a successful conclusion Ensure a positive customer experience throughout all stages of the renewal process Partner with sales, customer success, order management, our partner organisations and senior leadership to accelerate the contract renewal process and create positive outcomes Build and communicate an accurate forecast for upcoming renewals Identify and highlight any renewal risk to your partners and work with all team members in order to develop resolution strategies Evaluate your portfolio to prioritise time to render the most favourable outcomes Highlight opportunities for process improvement Perform other duties as assigned You Have: 5+ years of Sales, Customer Success or Finance experience. 2+ years of experience working in a software renewal role. Strong negotiation skills with experience driving

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is seeking an experienced sales leader to oversee a team of sales managers as a Regional Vice President (RVP) of Commercial Sales . The ideal candidate will scale, operate and continuously improve our commercial sales business through high performing teams, strategies and execution. This leadership role is based in the US and reports directly to a VP of Commercial Sales. You Will: Recruit, hire, coach, and develop sales talent across 35+ individual contributors and 4+ managers Ensure the team exceeds sales and operational targets on a monthly and quarterly basis Engage with our SMB customer base to directly work on strategic sales opportunities Provide day-to-day leadership of sales team and ensure proper application of sales methodology and departmental standards Develop, execute, evaluate and refine aspects of sales strategy, process, and workflow to deliver maximum customer engagement, bookings and revenue growth Monitor performance of sales representatives and teams for quota attainment, adherence to policies, and development of action plans, as needed. Develop and deliver business operational reviews summarizing key performance drivers and critical insight to increase future sales opportunities and required strategies and tactics. Create a high-performing, energized, and supportive culture, emphasizing reward and recognition You Have: Minimum of 7+ years of sales management experience, preferably in a multi-contact, high volume inbound and outbound environment 2+ years of experienc

For over 20 years, Smartsheet has empowered teams to manage work seamlessly and scale solutions smarter. Now, in our most ambitious chapter yet, we are uniting human teams with AI agents. By orchestrating the work agents do best, automating manual tasks and uncovering insights at scale, we create the space for people to focus on what truly matters: judgment, creativity, and big thinking. That is magic at work, and it’s what we show up for every day. Smartsheet is looking for a highly technical Sr. Manager to lead our India-based Service Desk and IT Automation operations. This is a hands-on, working manager role — you will be personally executing technical work (scripting, automation builds, escalated ticket resolution) and leading a team, not just overseeing one. If you're looking for a purely strategic or delegation-only leadership role, this isn't it. You will manage a team spanning Service Desk, Tier 1 SOC support, and IT Automation Engineering — driving the technical maturity of our global support infrastructure through automation, scripting, deep systems integration, and increasingly, AI-augmented workflows (e.g. intelligent ticket triage, SOC alert summarisation, AI-assisted knowledge base drafting). This role reports directly to the Director of End User Systems , with close, cooperative partnership with US-based Desktop Support managers. You Will IT Automation & Architecture Support the IT automation function, driving initiatives that reduce repetitive work and scale operations efficiently Identify repetitive Tier 1 tasks and drive end-to-end automation from identification through build, testing, and measurement Evaluate and build AI-augmented automation where it reduces toil — e.g. intelligent ticket triage, SOC alert summarisation, AI-assisted knowledge base drafting Mentor team on automation best practices and review technical designs before deployment Team Leadership & Execution Manage India-based Service Desk team with daily coordination with US

About Stitch Fix, Inc. Stitch Fix (NASDAQ: SFIX) Stitch Fix is redefining retail by combining human creativity with advanced data science and Generative AI. As we build the future of personalized shopping, we’re equally committed to building yours. We believe in investing in our team as much as our technology. Join us to be a trendsetter in the industry and help us redefine what’s possible for our clients, while we help you reach your full potential. About the Team The Stitch Fix Finance team enables our clients to be their best self. We broadly define our clients as customers, business partners, and shareholders. We drive value through business partner support, innovative and insightful analysis, and financial integrity. We are looking for bright, kind, and motivated-by-challenge individuals who succeed in a fast-paced environment and thrive in building and improving processes to scale the business to the next level. About the Role Stitch Fix is seeking an Accounts Payable Manager to join our dynamic Finance organization. In this role, you will serve as a high-impact leader capable of balancing rigorous accounting integrity with business practicality. This is a highly cross-functional position where you will direct the accounts payable and T&E team. You will manage daily operations within the procure-to-pay lifecycle, overseeing vendor relations, travel and business expense activities, and master vendor data. Additionally, you will serve as a key stakeholder in partnership with Accounting, Merchandising, Operations, Engineering, and Financial Systems teams to evaluate the impact of organizational changes on our P2P processes and provide strategic input on system design and UAT approaches. The ideal candidate is an inquisitive, system-savvy self-starter who possesses superior attention to detail and a commitment to continuous process improvement and team development. You're excited about this opportunity because you will… Manage and develop a high-pe

About Stitch Fix, Inc. Stitch Fix (NASDAQ: SFIX) Stitch Fix is redefining retail by combining human creativity with advanced data science and Generative AI. As we build the future of personalized shopping, we’re equally committed to building yours. We believe in investing in our team as much as our technology. Join us to be a trendsetter in the industry and help us redefine what’s possible for our clients, while we help you reach your full potential. About the Role We are looking for a bright, kind and goal oriented Industrial Engineering Lead to join the Stitch Fix Industrial Engineering Team. In this role you'll be responsible for onsite process improvement and cost savings initiatives within our warehouses. Reporting into the Director of Industrial Engineering, the Industrial Engineering Lead will work closely with other members of the team and cross-functional operational partners in our warehouses located across the country. Responsibilities: Design and implement new technologies, processes, and equipment that will enable our warehouses to support business growth. Manage large cross functional projects from ideation to implementation. Create and update engineering labor standards within our labor management system. Conduct the economic evaluation of ideas and technologies, and write the business cases to justify projects and track results to demonstrate success Design layouts for facilities, equipment, inventory storage, workspaces, and material flows Create the technical specifications that define our equipment and facility standards, and warehouse automation concepts. Write the documents necessary to procure equipment and technical services such as scope of work, performance requirements, and acceptance criteria. Support maintenance with automated equipment downtime resolution. Conduct inventory capacity forecasting and related capital investment planning. Lead the implementation of lean warehousing programs, includin

About Stitch Fix, Inc. Stitch Fix (NASDAQ: SFIX) Stitch Fix is redefining retail by combining human creativity with advanced data science and Generative AI. As we build the future of personalized shopping, we’re equally committed to building yours. We believe in investing in our team as much as our technology. Join us to be a trendsetter in the industry and help us redefine what’s possible for our clients, while we help you reach your full potential. About the Role The Client Experience Product Algorithms team is responsible for the machine learning, AI, experimentation, and product analytics capabilities that power personalized experiences for Stitch Fix clients and stylists. Partnering across Product, Engineering, Design, Styling, Marketing, Merchandising, Finance, Enterprise Analytics, Data Platform, and DSN, the team translates data and algorithms into measurable business impact. As Director, Product Algorithms, you will lead the strategy, execution, and people behind our Growth, Styling, and Fix & Freestyle Algorithms portfolios. You'll define how AI, machine learning, experimentation, and analytics shape the future of personalized shopping while building the operating discipline, technical excellence, and cross-functional alignment needed to deliver scalable business results. Responsibilities Lead the Product Algorithms portfolio across Growth, Styling, and Fix & Freestyle, setting strategy and driving measurable outcomes across acquisition, engagement, retention, styling quality, Fix, Freestyle, outfitting, and related commerce experiences. Define the vision and roadmap for applying data science, machine learning, AI, experimentation, and product analytics to improve client experiences, stylist effectiveness, and business performance. Drive innovation by identifying, evaluating, and scaling modern AI, machine learning, personalization, and experimentation techniques that create meaningful impact while balancing technical feasibility, exec