Human Data Project Lead
- Company
- Protege
- Location
- New York City, NY
- Work type
- Full Time
- Posted
- 2026-10-26
Job description
About the team
The Human Data team produces the human judgments that make AI training data trustworthy. That includes annotating data, verifying model output, and the human review that makes healthcare data safe to use. We work with an in-house team of annotators and a network of partners ranging from annotation platforms to practicing physicians. Our work feeds the research teams building AI evaluations and training environments, and the AI labs that license our data.
About the role
You'll own human data projects from start to finish. A project might be 800 scanned medical documents reviewed by five annotators over three weeks, or thousands of physician ratings on model output for a research team. You scope it with the team that needs it, design and test the task, staff it, keep it on schedule and on budget, catch quality problems before they spread, and deliver results the customer can trust. This role is built for someone early in their career who wants to own outcomes quickly, and who is as comfortable in a spreadsheet as in a room with researchers.
What You'll Do
Own human data projects end to end: scope them with the requesting team, set timelines and budgets, and deliver on them.
Design tasks and instructions with the requesting team, and pilot them on a small batch before anything scales.
Staff projects across our in-house annotators and partner vendors, and manage those vendor relationships day to day.
Run calibration before production. Every annotator passes a known-answer test before their work counts, and anyone who doesn't gets feedback and a retry.
Track throughput, cost and quality by annotator and vendor, and report progress to leadership every week.
Catch problems early (an annotator drifting, a vendor slipping, an instruction that isn't working) and fix them before they reach the customer.
Handle sensitive data, including protected health information, under strict security procedures.
Travel on short notice when a project needs someone on site.
Make every project easier to run than the last one.
What Success Looks Like
30 days: learn the machine. You've shadowed a live project, taken over its roster and returns intake, and met each of our annotation partners.
60 days: own a project. You've taken one project from scoping to delivery on your own, and every open project is tracked in one place.
90 days: own the portfolio. You're running two to three projects at once, you've scoped a new one directly with a requesting team, and you've delivered our first cost-and-quality comparison across vendors.
What You Bring
Must have
One to three years in consulting, finance, a startup, or another operationally intense environment where you owned complex projects with real stakes. Strong new grads with that kind of experience are welcome.
Strong analytical skills, with instincts for statistics
Comfort and experience with agentic coding/work tools, willingness to learn new ones
Comfort working directly with demanding stakeholders, including researchers and outside partners.
The judgment to notice when something is off and the instinct to raise it early rather than guess.
Clear, fast written communication.
Authorization to work in the U.S. as a W2 employee (required for access to protected health information).
Nice to have
Experience running data annotation, labeling or AI training data projects
Healthcare, clinical research or HIPAA-regulated work
Experience managing contractors, vendors or freelancers
Python or SQL