Trust And Safety Policy Jobs

Reddit is a community of communities. It’s built on shared interests, passion, and trust, and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote, and comment on the topics they care most about. With 100,000+ active communities and approximately 130 million daily active unique visitors, Reddit is one of the internet’s largest sources of information. For more information, visit www.redditinc.com . We are looking for an experienced Ads Policy Lead who has significant subject matter expertise, strong policy development capabilities, and thrives in dynamic environments. This is a senior individual contributor role, with no people management responsibilities, reporting to our Head of Ads Policy. This hire will be primarily responsible for driving the research and development of large-scale policies that impact Reddit's global advertising platform. This critical new role will partner with colleagues and leaders across Legal, Sales, Operations,Product, Trust & Safety Policy and other departments to tackle cutting edge challenges, enable responsible business opportunities, and work on other high impact advertising policy projects that help Reddit continue to scale its business across the world. What You’ll Do: Drive the development, drafting, and maintenance of strategic advertising policies across Reddit's ads products acting as subject matter expert in the EMEA region Draft risk assessments and strategic policy recommendations for senior cross-functional leaders and executive stakeholders, representing the Ads Policy perspective with clarity and confidence. Act as a role model of Reddit’s Company Values, assisting teammates and providing guidance as a senior peer leader Serve as the decision-maker on highly complex, grey-area ads policy escalations, delivering definitive, risk-balanced rulings on high-impact issues Qualifications: 7+ years of experience in compliance, standards, or policy development and implement

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . Pinterest is the world's leading visual search and discovery platform, serving over 500 million monthly active users globally on their journey from inspiration to action. As we scale experiences in a complicated ecosystem, ensuring they are safe, fair, and trustworthy is paramount. We are looking for a Senior Data Scientist to help lead Pinterest's Trust and Safety mandate by designing the foundations for measuring the prevalence of unsafe content across the platform. In this role, you will design and build sampling frameworks, complex data aggregations, and measurement methodologies to track Trust & Safety policy violations across complex, multi-component user interactions. You will work in a highly collaborative and cross-functional environment, partnering with ML Engineers, Trust & Safety Ops, subject matter expert teams, and Product

About Us Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day. We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities, so if this sounds good to you, see what we’re up to on LinkedIn and X , and discover the projects we’re solving on our Blog . Be sure to explore our Interviewing Guide to learn how to ace our interview process. About the Team Community Trust (CT) exists to foster an environment where streamers and their communities thrive, while proactively reducing harm and ensuring support is accessible when needed. Our organization encompasses safety policy & enforcement, fraud prevention, and customer experience. Within CT, the Business Operations & Integrations team is the operational backbone – encompassing four functions (Business Operations, Financial & Contract Management, Go-to-Market, and Safety by Design) split across two sub-teams: BizOps and Integrations. The Integrations team – which this role sits on – is the connective tissue between Community Trust and the rest of Twitch. We own the CT Readiness Program, ensuring every product, monetization, regulatory, or business development launch is operationally ready before it reaches our community (200+ launches per year). We also co-own Safety By Design, the process by which safety risks are identified and mitigated before products ship. We're building a team that can anticipate what's coming and react to what's arrived – getting CT into planning conversations earlier, simplifying how Twitch launches safely at scale, and building the systems that allow a small team to have outsized impact. About the Role This is a Program Manager II role on the Integration

At Lyft, our purpose is to serve and connect. We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. Lyft’s Safety Team is looking for a caring and empathetic individual to be part of the highest level of Safety response. This person will assist and support current Compassionate Care Sr. Associates and Specialists as the first response to customers involved in the most egregious incidents that occur on the Lyft platform. You must be able to provide a sense of comfort to customers involved in traumatic situations, as well as identify potential ways we can support them as they move forward and recover. You must be comfortable working with Safety and Customer Cares team members, acting as the subject matter expert for all things Safety-related. Communicating confidently, efficiently and effectively is key, turning complex incidents and customer needs into concise reports for Safety leadership. We’re looking for someone who has the confidence and creativity to continue elevating Lyft’s Safety Support as the standard in world-class customer care. Responsibilities: Be a strong, caring, and empathetic victim advocate for the Lyft community. Act as the highest level of escalation for our Safety Support teams. Focus on building trust and long-term relationships with customers during extremely sensitive situations to ensure they feel supported. Respond with empathy and sound judgment, working with the customer end to end. Identify potential risks, mediate, and diffuse escalated situations. Support other tiers of Safety Support with case reviews. Partner with the Lyft Operations Center on credible threats to team members or locations Partner with the Crisis Communications team on safety incidents involving media. Partner with the Safety Policy and Community Compliance team on escalated cases that require review. Learn and maintain advanced levels of training focused on supporting and being a victim advoc

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: Our hosts are opening their homes or providing experiences to guests from around the world. We’re creating new relationships & placing people in vulnerable situations. Therefore trust is the fundamental currency of Airbnb. We are the Trust & Safety team, composed of engineers, data scientists, designers, product managers, and operations, responsible for earning this trust by building for safety at each moment of the guest and host journey. We build cutting edge machine learning models to detect risk, defenses to deter risky actors, and operations tools that enable investigators worldwide. We define community standards and policies, set high expectations, and keep bad actors out while providing delightful experiences for good users. The Difference You Will Make: As a Product Manager in the Trust & Safety team, you will identify customer pain points, business opportunities, and define/deliver the products that help us create a safe and trusted community around the world. Your work will be pivotal to shaping the future of hospitality and travel. Within the Trust & Safety team, you’ll be focused on Safety: To reduce safety incidents (e.g. interpersonal harm and illegal activities). A Typical Day: Creating vision and strategy for Trust & Safety products and platforms. Leading a cross-functional working group of engineers, data scientists, designers, operations experts, and policy experts to ensure alignment, execution, and launches Maintaining product backlog and the product roadmap. Own prioritization process of product solutions based on business v

About the Team Our Safety Systems team is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. Within Safety Systems, the Model Policy team works to ensure that increasingly capable models behave safely and reliably in real-world environments. We investigate emerging model failures, define the behavior models should exhibit instead, and develop the data, evaluations, monitoring, and safeguards needed to improve and validate that behavior. Our work connects alignment research with the practical challenges of training and deploying frontier models. About the Role In this role, you will shape how OpenAI understands and addresses real-world risks that emerge from model misalignment as models become more autonomous and operate over longer horizons. You will investigate how misaligned behavior emerges across extended trajectories - including when models persist toward the wrong objective, take unsafe shortcuts, lose track of instructions, exploit weaknesses in their environment, or circumvent constraints - and translate these insights into behavioral policies, evaluations, monitoring, and safeguards. This role is ideal for someone who wants to turn alignment and safety concerns into concrete, empirically grounded improvements to frontier AI systems. Your Responsibilities: Identify vulnerabilities that emerge as models interact with tools, data, and external systems, and translate them into model- and system-level safeguards. Develop threat models and empirical frameworks for understanding harmful outcomes from misaligned behavior. Build frameworks for understanding harmful outcomes arising from model misalignment. Identify the underlying behaviors and system conditions that drive those outcomes. Turn findings into policy frameworks, evaluation criteria, online measurement and safeguards. Develop human data campaigns and gold sets to ground measurement and evaluation of eme

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. About the Role We're redefining how software is built and who gets to build it. Our mission is to achieve Autonomy for All: making programming accessible, collaborative, and powered by AI. Realizing that vision requires a platform that legitimate users can trust and adversarial actors cannot exploit. We're hiring a Data Scientist to help build Replit's Trust & Safety and Anti-Abuse program from the ground up. You'll turn noisy behavioral, identity, payment, infrastructure, and content signals into the measurement systems, detections, and decisions that protect Replit's users, platform, and economics. You'll work closely with Engineering, Support, Legal, Security, Infrastructure, Money, and Growth to make abuse economically unviable while keeping friction low for legitimate users. Replit sits at the frontier of AI-native abuse. Our platform is a target for phishing and scam hosting, cryptomining, LLM token farming, card and coupon fraud, referral abuse, and increasingly, abuse driven by AI agents themselves. You'll help define how we identify, measure, and respond to these threats without compromising the experience of good users. Who You Are You're a data scientist who moves fast, goes deep, and thinks adversarially. You can spin up an analysis in hours that would take others days, not by cutting corners, but because you've built the intuition and technical toolkit to get to the right answer quickly. You dig past the top-line abuse rate to understand selection effects, missing labels, policy changes, attacker adaptation, and the false positives hidden inside an aggregate metric. You understand that Trust & Safety data is imperfect and outcomes are high stakes. Ground truth is delayed, biased, and often incomple

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . As a Staff Product Manager for the GenAI Safety team within Trust & Safety, you'll define and drive the product strategy for ensuring Pinterest's GenAI-powered systems are safe, fair, and trustworthy. You'll be responsible for building proactive safety frameworks that scale with our growing AI capabilities, partnering deeply with engineering, policy, data science, and design to protect our users while enabling Pinterest to innovate responsibly. This is a high-impact role for someone who is passionate about the intersection of AI and user safety, and who thrives in ambiguous, fast-evolving problem spaces. You'll work at the frontier of responsible AI - anticipating novel harms before they emerge, red-teaming new AI features, and translating complex policy goals into measurable product requirements. What you'll do: GenAI Safety Strategy: Own a

Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. Role Summary Replit is the agentic software creation platform that enables anyone to build applications using natural language. With millions of users worldwide, Replit is democratizing software development by removing traditional barriers to application creation. As a Disputes and Chargeback Program Manager at Replit, you’ll be part of our Trust and Safety Operations team, who owns customer-facing workflows that handle sensitive and policy-driven interactions across the Support organization. This includes areas such as privacy requests, DMCA and intellectual property claims, law enforcement requests, and abuse appeals. You'll own Replit's disputes and chargeback function end-to-end: from the individual case in front of you to the policies, playbooks, and automation that let it hold up as we scale to millions of users. This role combines strategic ownership with hands-on execution: you'll determine dispute and chargeback strategy, and build the systems that make those decisions repeatable at scale, working closely with Engineering, Legal and Finance. In this role you will… Own the full dispute lifecycle: investigate incoming chargebacks and disputes, decide whether and how to represent, gather evidence, and respond within network deadlines. Design and maintain the policies, decision trees, and SOPs that govern how disputes and chargebacks are handled. Define the key metrics, health targets, and goals for the disputes program and track performance against them, reporting to leadership. Monitor chargeback rates and dispute trends to identify the underlying drivers of volume, and partner with Engineering and Finance to address root causes. Partner with Engineering and Support Operations to automate manual parts of the wor

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we’re on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other’s unique experiences and embrace the flexibility to do your best work. Creating a career you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful partner that augments our creativity and amplifies our impact, and we’re looking for candidates who are excited to be a part of that. To get a complete picture of your experience and abilities, we’ll explore your foundational skills and how you collaborate with AI. Through our interview process, what matters most is that you can always explain your approach, showing us not just what you know, but how you think. You can read more about our AI interview philosophy and how we use AI in our recruiting process here . The Compliance product and engineering team is responsible for ensuring that Pinterest scalably complies with all new and existing regulations, and advocates for Pinner privacy and safety, data governance, and rights holders. What you’ll do: Understand global and domestic legislative landscape, specifically related to content and IP laws, to inform prioritization Own and drive the Compliance Engineering product roadmap, including tools for rights holders, legal reporting channels, and associated internal workflows Partner with Engineering to identify and build systems and features that scale across legislation Own and improve suite of copyright and trademark tools to allow rights holders to manage their content at scale Lead key compliance projects, partnering with Legal, Operations, Policy, Engineering and o

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. As a Quality Analyst, you will support the Quality Manager and the broader team in maintaining quality controls, executing calibration reviews, and ensuring smooth auditing operations with our vendor partners. You will play a crucial role in validating audit accuracy, analysing performance trends, and identifying grading discrepancies, helping to safeguard platform integrity and streamline vendor quality processes. You Will: Assist in executing regular quality compliance reviews and operational audits to ensure vendors adhere to established grading rubrics. Conduct daily audits of vendor customer service and moderation audits to verify accuracy and policy alignment. Gather, format, and maintain quality performance metrics and audit data to highlight recurring errors or vendor grading drift. Support the Lead in coordinating with Moderation, Customer Support, and Global Scaled Operations teams to resolve quality disputes. Participate in calibration sessions to ensure consistent interpretation of trust and safety policies across all vendor sites. Help test and implement new quality automation tools or auditing platforms introduced by the technical and leadership teams. You Have: 3-5 years of e

About the Team The Monetization team is a new cross-functional group spanning engineering, product, research, and design. We are building the foundational systems that will help OpenAI scale access to intelligence responsibly. Our mission is to create user-first, privacy-preserving monetization products—including next-generation ads experiences—that strengthen trust, expand economic opportunity, and support OpenAI’s long-term innovation. We believe monetization should deliver clear user value, uphold rigorous privacy and safety standards, and sustain a healthy ecosystem for developers and businesses. The team operates in a greenfield environment, moving quickly through prototyping, experimentation, and iterative deployment. We partner closely across Product, Design, and Research to bring new capabilities into real-world systems at global scale. About the Role We’re looking for an experienced Software Engineer to build the integrity systems that keep OpenAI’s ads products safe, trustworthy, and resilient to abuse. In this foundational role, you will design infrastructure that detects and prevents harmful, deceptive, fraudulent, or policy-violating ads and advertiser behavior at scale. This role is well suited to an engineer who has built large-scale systems in ads integrity, trust and safety, anti-abuse, fraud, security, risk, or a related domain—and who wants to apply that experience in an ambiguous 0→1 environment. You will work across risk signals, detection and enforcement platforms, review tooling, adversarial resilience, advertiser controls, and integrity measurement. We are hiring engineers who can independently own complex systems, make sound technical tradeoffs, and help define what should be built. You will collaborate with Ads Delivery, Ads ML, Product, Research, Safety, Policy, Privacy, Legal, Security, and operations teams to create a high-integrity ads ecosystem from first principles. This role is based in San Francisco. We offer relocation assistance t

About the Team The Integrity team builds the systems OpenAI uses to understand, prevent, and respond to misuse across our products. We partner with Product, Policy, Safety Systems, User Operations, Security, Legal, Privacy, OpenAI for Government, and research teams to turn policy and threat models into product controls, review workflows, measurement systems, and enforcement paths. About the Role We are hiring a Product Manager to own Integrity's product strategy for sensitive deployments: contexts where model capability, customer or deployment setting, privacy constraints, and misuse potential make the operating bar unusually high. This includes government and other high stakes deployments, regulated or high-trust enterprise contexts, zero data retention and privacy-constrained environments, and agentic workflows where harm can unfold across many steps rather than a single prompt. The Sensitive Deployments PM will focus on high-consequence use cases relevant to government deployments and broader deployment-readiness questions for high-risk domains (e.g., healthcare), while building reusable Integrity capabilities for agentic detection and enforcements in these environments. This position is based in San Francisco, CA, with relocation assistance available. In this role, you will: Own the roadmap for sensitive deployment Integrity controls and readiness criteria. Define how we measure residual risk, decision quality, review quality, and mitigation effectiveness. Build agentic investigation and context-assembly workflows for complex, multi-step misuse patterns. Partner with Policy, Legal, Privacy, Safety Systems, User Ops, Government, and product teams to build shared capabilities. Create launch and deployment playbooks for sensitive customer, capability, and product contexts. You might thrive in this role if you: Have shipped complex product systems in AI, integrity, trust and safety, security, privacy, risk, government, regulated enterprise, or AI safety. Can turn am