Trust And Safety Associate Jobs

About the Team The Safety Systems team is dedicated to ensuring the safety, robustness, and reliability of AI models and their deployment in the real world. Learn more about OpenAI’s approach to safety. Building on the many years of our practical alignment work and applied safety efforts, Safety Systems addresses emerging safety issues and develops new fundamental solutions to enable the safe deployment of our most advanced models and future AGI, to make AI that is beneficial and trustworthy. About the Role At OpenAI, we're dedicated to advancing artificial intelligence, and we know that creating a secure and reliable platform is vital to our mission. That's why we're seeking a software engineer to help us build out our trust and safety capabilities. In this role, you'll work with our entire engineering team to design and implement systems that detect and prevent abuse, promote user safety, and reduce risk across our platform. You'll be at the forefront of our efforts to ensure that the immense potential of AI is harnessed in a responsible and sustainable manner. Your Responsibilities: Architect, build, and maintain anti-abuse and content moderation infrastructure designed to protect us and end users from unwanted behavior. Work closely with our other engineers and researchers to utilize both industry standard and novel AI techniques to measure, monitor and improve AI models’ alignment to human values. . Diagnose and remediate active incidents on the platform and build new tooling and infrastructure that address the root causes of system failure. You might thrive in this role if: You have built and run production services in a high growth, rapidly scaling environment. You can debug live issues and restore systems quickly. You have worked on content safety, fraud, or abuse, or are motivated and excited to work on present-day (“now-term”) AI safety. You have experience with Python or with modern languages such as C++, Rust, or Go, and are able to quickly ramp up on Py

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Distinguished Machine Learning Engineer/Technical Director in the Safety organization at Roblox, you will drive the overall technical vision and execution for all machine learning initiatives focused on maintaining the safety and civility of our users. Our industry-leading safety features ensure Roblox remains a safe and inclusive environment for our community to express themselves creatively and share experiences without fear. The Safety org is the reason why Roblox is the safest place on the internet, protecting users You will provide technical leadership on AI/ML efforts for Trust and Safety as the Roblox platform scales to serve different age groups and geographic locations. You Will: Own the technical direction and implementation of machine learning solutions for safety-related systems Lead and mentor other engineers, fostering a culture of technical excellence and inclusivity Break down long-term product requirements into iterative deliverable stages, ensuring continuous improvement Craft and build large-scale machine learning models with billions of parameters, ensuring production-readiness Facilitate challenging technical decisions across multiple teams, demonstrating empathy a

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Senior Product Designer, you’ll be a key member of the Safety design team, playing a critical role in building the systems that keep Roblox safe, civil, and inclusive for a global community. You’ll focus on creating platform-level designs and frameworks that support users in moments of need and uphold community standards across the entire Roblox ecosystem, across Creator, Player, and Brands. Roblox’s mission is to connect a billion people with optimism and civility. The safety organization is at the heart of that mission, working to detect, remove, and prevent problematic content and behavior at scale. We also ensure account security, platform compliance with global regulations, and a healthy, inclusive environment for users of all ages. You’ll collaborate closely with cross-functional teams across product, engineering, and research, shaping the platform's approach to trust and safety through thoughtful design. This is an opportunity to work at the systems level—defining how safety is embedded across Roblox and designing reusable frameworks that power our vision of civil immersive online communities. You will: Create clear, compelling, and empathetic experiences that help users underst

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role The Fraud Prevention team protects Anthropic's payment and monetization surfaces from financial abuse — keeping fraud losses, dispute rates, and network monitoring exposure in check while preserving a smooth experience for legitimate customers. As a software engineer on this team, you will build the systems that make risk decisions in real time, manage the dispute and chargeback lifecycle, and detect monetization abuse across subscriptions, in-app purchases, and promotions. The ideal candidate can see things from attackers' perspectives, anticipate their responses to countermeasures, and never loses sight of the fact that a false positive here is a paying customer. Payments fraud is more externally coupled than most trust and safety work — you'll collaborate closely with finance, support, and legal teams internally, and with payment processors and platform partners externally. Responsibilities: Design and build real-time risk decisioning that scores transactions at authorization time, balancing fraud loss, approval rates, and latency constraints Build tooling and automation for the dispute and chargeback lifecycle, from review queues to evidence collection and loss reporting Engineer fraud signals at scale — device fingerprinting, BIN and issuer signals, velocity features, and cross-account linkage — and detect monetization abuse across subscriptions, trials, promotions, and in-app purchases Own a portfolio of metrics — loss rate, dispute rate, authorization approval impact, and false-positive rate — rather than optimizing any single number Lead investigations into emerging fraud patterns, building multi-layered defense

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . We’re looking for an enthusiastic and thoughtful team player to join our Policy team as a Product Policy Lead for spam. The spam world is full of dynamic adversaries who change tactics regularly to avoid detection and we need to stay a step ahead of bad actors to keep Pinterest the inspiring place our users love. You will draft policies that define spam on Pinterest, create detailed policy enforcement guidance to help our Operations teams fight spam, and collaborate closely with many cross-functional partners across the company, including Product and Engineering teams, to tackle spam and abuse at scale. This role will sometimes involve reviewing graphic and disturbing content, discussing difficult subjects, and responding to escalations. What you’ll do Lead spam policy development for Pinterest. Partner with our Operations, Product and Eng

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role We are looking for software engineers to help build safety and oversight mechanisms for our AI systems. As a software engineer on the Safeguards team, you will work to monitor models, prevent misuse, and ensure user well-being. This role will focus on building systems to detect unwanted model behaviors and prevent disallowed use of models. You will apply your technical skills to uphold our principles of safety, transparency, and oversight while enforcing our terms of service and acceptable use policies. Responsibilities: Develop monitoring systems to detect unwanted behaviors from our API partners and potentially take automated enforcement actions; surface these in internal dashboards to analysts for manual review Build abuse detection mechanisms and infrastructure Surface abuse patterns to our research teams to harden models at the training stage Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale You may be a good fit if you have: A Bachelor’s degree in Computer Science, Software Engineering or comparable experience Proficiency in Python and Typescript An ability to work across the stack Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders Strong candidates may also: 8+ years of experience in a software engineering position Have experience with integrity, spam, fraud, or abuse detection and mitigation Have experience building trust and safety detection mechanisms and intervention for AI/ML systems Have experience with prompt engineering, jailbreak attacks, and other adversarial inputs Have worked

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role We are looking for software engineers to help build the foundational pieces for safety, oversight and intervention mechanisms of our AI systems. As a software engineer on the Safeguards team, you will work to monitor models, prevent misuse, and ensure user well-being. This role will focus on building systems to detect unwanted model behaviors and prevent disallowed use of models. You will apply your technical skills to uphold our principles of safety, transparency, and oversight. Responsibilities: Develop the foundational systems which power Safeguards, including infrastructure for data storage and management, metric and evaluation systems, and tooling for human and agentic review. Ensure the day-to-day running of Safeguards systems and hold a high operational bar which serves both safety and customers while reducing the amount of human intervention and oversight required. Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale You may be a good fit if you have: Bachelor’s degree in Computer Science, Software Engineering or comparable professional experience Proficiency in Python Ability to work across the stack Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders Strong candidates may also: 7+ years of experience in a software engineering position Have experience building trust and safety, anti-spam, fraud or abuse detection and mitigation mechanisms and interventions for AI/ML systems Have experience building metrics and measurement systems or data and privacy management systems Have worked closely with op

About the Role Chime is looking for a hands-on engineering leader to guide the Human Agent Tooling team within Trust & Safety. As a Tech Lead Manager, you’ll lead a squad of 3–5 engineers, set the hiring bar, coach and develop talent, and own execution and delivery. You’ll also remain deeply involved in the technical work, writing and shipping production code alongside your team. The Human Agent Tooling team builds the software Chime’s support agents use to understand member needs, take the right actions, and resolve issues quickly and safely. This includes the agent workspace, case management tools, real-time member context, and access controls. You’ll help modernize this platform and partner closely with Product, Operations, and support agents to make every human support interaction faster, safer, and more effective. In this role, you can expect to Drive technical design and architectural decisions, while mentoring engineers and fostering cross-functional collaboration Lead and contribute to multiple projects simultaneously, balancing ownership of key initiatives with support across broader team efforts Build and maintain robust, low-latency APIs and backend services that power our Trust and Safety platforms Become the domain expert in our platform architecture, providing technical decisions and solutions to refine our maturing platforms Partner with data engineers, analysts, and product managers to ship impactful features that enhance member protection and experience Improve platform reliability and performance through thoughtful system architecture, observability practices, and automation Evolve our experimentation, policy lifecycle, and ML integration capabilities to support smarter and faster decision-making at scale Engineers are required to participate in on-call rotation. Being on call may include responding to incidents outside of regular working hours when necessary. To thrive in this role, you have 5+ years of experience developing software in a prod

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Trust and Safety Operations team is focused (ok, maybe obsessed) on scaling Roblox's Operations organization and transforming our customer experience through our multi-year vision and strategy execution. The team provides support to Roblox's global players, developers, and advertisers. As a Lead, Product Support Ops on our Safety Ops Team, you’ll be mentor and lead the team that is focused on Escalations, Product Reliability, Process Improvement and Design and plays a critical role in managing and resolving escalations and high-impact incidents affecting our Roblox users and platform. Your team will be responsible for identifying trends, enhancing support processes, improving agent efficiency, and reducing operational costs while maintaining high reliability and service quality. You will be reporting to the Trust & Safety India Operations Manager and leading the Product Support Specialist team, providing coverage across billing, accounts, creator, and adjacent domains. This is a player-coach role, weighted toward leadership. You'll still go deep on the hardest problems and stay close enough to the work to be a genuine execution expert, but your primary leverage is the team, and you'

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Trust and Safety Operations team is focused (ok, maybe obsessed) on scaling Roblox's Operations organization and transforming our customer experience through our multi-year vision and strategy execution. The team provides support to Roblox's global players, developers, and advertisers. As a Lead, Product Support Ops on our Safety Ops Team, you’ll be mentor and lead the team that is focused on Escalations, Product Reliability, Process Improvement and Design and plays a critical role in managing and resolving escalations and high-impact incidents affecting our Roblox users and platform. Your team will be responsible for identifying trends, enhancing support processes, improving agent efficiency, and reducing operational costs while maintaining high reliability and service quality. You will be reporting to the Trust & Safety India Operations Manager and leading the Support Operations Excellence team, providing coverage across billing, accounts, creator, and adjacent domains. This is a player-coach role, weighted toward leadership. You'll still go deep on the hardest problems and stay close enough to the work to be a genuine execution expert, but your primary leverage is the team, and y

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. The Trust and Safety Operations team is focused (ok, maybe obsessed) on scaling Roblox’s Operations organization and transforming our customer experience through our multi-year vision and strategy execution. You will be reporting to the Trust & Safety India Operations Manager. The team provides support to Roblox’s global players, developers, and advertisers. As a Product Support Specialist on our Safety Ops Team, you’ll be focused on Escalations, Product Reliability, Process Improvement and Design along with playing a critical role in managing and resolving escalations and high-impact incidents affecting our Roblox users and platform. You will coordinate across multiple teams to ensure rapid response to minimize the impact on customers and business operations, identifying trends, enhancing support processes, improving agent efficiency, and reducing operational costs while maintaining high reliability and service quality. Additionally, you will serve as a product expert working closely with the PSM team, providing detailed synthesis of new and existing issues our users experience to prioritize and enable engineering teams to improve the platform. You Will: Coordinate a

About Us Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day. We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities, so if this sounds good to you, see what we’re up to on LinkedIn and X , and discover the projects we’re solving on our Blog . Be sure to explore our Interviewing Guide to learn how to ace our interview process. About the Team Community Trust (CT) exists to foster an environment where streamers and their communities thrive, while proactively reducing harm and ensuring support is accessible when needed. Our organization encompasses safety policy & enforcement, fraud prevention, and customer experience. Within CT, the Business Operations & Integrations team is the operational backbone – encompassing four functions (Business Operations, Financial & Contract Management, Go-to-Market, and Safety by Design) split across two sub-teams: BizOps and Integrations. The Integrations team – which this role sits on – is the connective tissue between Community Trust and the rest of Twitch. We own the CT Readiness Program, ensuring every product, monetization, regulatory, or business development launch is operationally ready before it reaches our community (200+ launches per year). We also co-own Safety By Design, the process by which safety risks are identified and mitigated before products ship. We're building a team that can anticipate what's coming and react to what's arrived – getting CT into planning conversations earlier, simplifying how Twitch launches safely at scale, and building the systems that allow a small team to have outsized impact. About the Role This is a Program Manager II role on the Integration