AI/ML - Evaluation Scientist, Siri Data

Cupertino, CA 95014
  • Job Code
    200221752
Summary

Summary

Posted: Aug 13, 2021

Weekly Hours: 40

Role Number:200221752

Would you like to play a part in the next revolution in human-computer interaction? Contribute to a product tha...Summary

Summary

Posted: Aug 13, 2021

Weekly Hours: 40

Role Number:200221752

Would you like to play a part in the next revolution in human-computer interaction? Contribute to a product that is redefining mobile and desktop computing, and work with the people who built the intelligent assistant that helps millions of people get things done - just by asking?

The vision for the Siri Data Organization is to improve Siri by using data as the voice of our customers. Within this organization the mission of the Analytics team is to inform the evolution of Siri through measurement and analysis of the user experience. Part of this mission is achieved through human evaluation; as an Evaluation Scientist, you will drive how we design our evaluation tasks and guidelines.

Key Qualifications

  • Experience with human evaluation and/or an equivalent type of human-subjects task design (e.g., large scale studies)
  • Excellent writing skills, especially with regards to guidelines. The tasks you design drives our data collection, so people must understand what you want them to do!
  • Strong communication skills and the ability to drive projects. Collaborate with cross-functional partners to ensure our evaluation tasks produce meaningful data that can be leveraged across a variety of contexts.
  • Analytics skills and ability to think about data.
  • Data-querying skills (SQL - be able to use data to evaluate task designs)
  • Experience with a scripting language for data processing and data analysis (e.g., Python, or R)

Description

The Siri Data Organization is seeking a talented Evaluation Scientist to drive methodologies for measuring how Siri is performing for our users as part of our evaluation program. Your will help curate data sets and metrics that will be used across Siri, and will impact key decisions on the Siri product!

You will develop and own various evaluation tasks for curating data sets from our human evaluators.
This will include:
- working with engineering teams to bring the tasks to life
- ensuring that the tasks are designed to elicit valid and reliable labels
- partnering cross-functionally to ensure that the tasks are providing data that meets the Siri organization's needs
- Author and collaborate on evaluation guidelines to ensure the human evaluators understand how to apply the guidelines and interact with the evaluation workflows
- Conduct analyses to inform evaluation task designs and guidelines improvements

Education & Experience

Masters Degree plus 2+ years industry experience or a Bachelor's degree plus 5+ years of industry experience.
Academic background in a social quantitative field such as Psychology, Cognitive Science, Sociology, Political Science, Quantitative Linguistics, or other related fields.

Additional Requirements

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

AI/ML - Evaluation Scientist, Siri Data

Apple, Inc.
Cupertino, CA 95014

Join us to start saving your Favorite Jobs!

Sign In Create Account