Director, Annotation
Location
Mountain View, CA
Level
Senior
Department
AI Coach
Type
Full - Time
Salary
Job Description
Posted on:
January 14, 2023
The Director, Annotation will provide leadership to a team or teams that label enterprise ticket and in-bot written textual utterance data for neural network training. A successful candidate will have high attention to detail, an appreciation for data integrity, and the people management and leadership skills to drive efficiency for large teams of annotators.
Responsibilities
- Own the data annotation pipeline at various stages of lifecycle from presales to maintaining existing customers while ensuring quality of data annotation for the entire team.
- Implement and improve quality assurance processes for increasingly complex data labeling, driving the optimization of annotation processes.
- Organize sequential and parallel data annotation processes and load balance team members across multiple projects of competing priority.
- Ensure the team follows the correct technical procedures and guidelines.
- Own the data annotation guidelines and ensure appropriate change management for modifying the guidelines when appropriate.
- Manage the team projects through development, pilot and implementation phases, and review processes.
- Provide technical training and mentorship. Ensure the team works as a cohesive unit.
- Work directly with engineers and, as needed, with Product, UX, Customer Success, and AI insights leadership. Own annotation policies that require an understanding of both technical and operational constraints.
- Build metrics that identify key performers, enabling better ownership and career trajectory for top annotators.
- Develop and maintain a capacity model for the entire annotation team
- Provide cost-of-task visibility to engineering
- Be the point person to make tradeoffs and prioritization decisions when the resource constraint dictates that some of requested tasks not be fulfilled
Job Requirements
- Commitment to data accuracy and throughput
- Strong problem solving skills, with an aptitude for quickly learning systems
- Proven track record executing and meeting sensitive deadlines
- Experience managing multiple competing projects with limited resources
- Understand when to delegate and allocate responsibilities to manage projects end-to-end
- Desire to automate out inefficiencies when applicable
- Ability to articulate and justify tooling additions requirements, serving as a liaison between the annotation teams and the tooling engineers
- Ability to match project tasks with opportunities for employee development
- Strong communication and presentation skills - can explain complicated topics and ideas in a clear and concise manner
- Proficiency in standard workflow software (JIRA, Excel, etc.)