Trust & Safety team members are tasked with identifying and taking on the biggest problems that challenge the safety and integrity of our products. They use technical know-how, excellent problem-solving skills, user insights, and proactive communication to protect users and our partners from abuse across our products. On this team, you're a big-picture thinker and strategic team-player with a passion for doing what's right. You work globally and cross-functionally with engineers and product managers to identify and fight abuse and fraud cases with urgency. And you take pride in knowing that every day you are working hard to promote trust and ensure the highest levels of user safety.
Our team in Trust and Safety Intelligence is a new team that will use unstructured persona-based adversarial testing techniques to identify 'unknown unknowns' and new/unexpected loss patterns on premier generative AI products. The team will work alongside product, policy and enforcement teams to proactively detect harm patterns, and help build the safest possible experiences for our users.
In this role, you will be at the forefront of generative AI testing in Trust and Safety, and support our efforts to launch bold and responsible products in this space. You will be exposed to graphic, controversial, or upsetting content.
This role works with sensitive content or situations and may be exposed to graphic, controversial, or upsetting topics or content.
We work hard to earn our users' trust every day. Trust & Safety is our team of abuse fighting and user trust experts working daily to make the internet a safer place. We partner with teams to deliver bold solutions in abuse areas such as malware, spam and account hijacking. A diverse team of Analysts, Policy Specialists, Engineers, and Program Managers, we work to reduce risk and fight abuse across all of our products, protecting our users, advertisers, and publishers across the globe in over 40 languages.
Responsibilities
- Write scripts that will help multiply the impact of the team, including through systematised/automated prompt creation and scraping of content.
- Monitor and research emerging abuse vectors for generative AI from open web and specialised sources, and work individually and collaboratively to promptly uncover new risk vectors in main Generative AI products.
- Apply insights for creative prompting of generative AI tools such as Gemini, Search Generative Experience, and Vertex API. Support the creation of persona-based adversarial playbooks to guide the team's red teaming.
- Develop repeatable processes that can yield valuable insights regardless of topic and attack vector. Annotate and cluster harm types detected in structured prompting exercises.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 5 years of experience in data analysis and working with datasets.
- Experience working on counter abuse strategies for online platforms.
- Experience working with Large Language Models.
Preferred qualifications:
- Experience working on product policy analysis and identifying policy risks.
- Experience with modelling, experimentation, and causal inference.
- Experience with adversarial testing of online consumer products.
- Experience in Data Analysis and SQL.
- Exceptional communication and presentation skills to deliver findings of analysis.
- Excellent problem-solving skills with attention to detail in an ever-changing environment.
Morgan McKinley is acting as an Employment Agency and references to pay rates are indicative.
BY APPLYING FOR THIS ROLE YOU ARE AGREEING TO OUR TERMS OF SERVICE WHICH TOGETHER WITH OUR PRIVACY STATEMENT GOVERN YOUR USE OF MORGAN MCKINLEY SERVICES.
