Legal Expert Lead, Gemini Data, DeepMind

Minimum qualifications:
Juris Doctor (JD) degree, Master of Laws (LLM), or equivalent practical experience.
Experience measuring the performance of Large Language Models (LLMs) or generative AI tools and optimizing them for legal use cases.

Preferred qualifications:
Experience prioritizing AI capabilities and collaborating closely with cross-functional technical partners to unlock value for legal professionals.
Experience breaking down legal workflows into concrete tasks, evaluation rubrics, and quality standards.
Demonstrated dual expertise bridging law and AI, such as transitioning from a legal practitioner into a technical machine learning role, or vice versa.
Strong technical capabilities, including programming proficiency, evaluating machine learning models, and applying Large Language Models to complex legal workflows.

Responsibilities
Set the Legal data strategy by mapping workflows into a use-case taxonomy, and partnering with research to prioritize capabilities, data coverage, and sourcing pipelines.
Create high-quality Legal data by designing tasks and rubrics, and QA-ing datasets from vendors, customers, and synthetic pipelines for correctness, realism, and coverage.
Steer data priorities using model failures by reproducing reported losses and stress-testing checkpoints on real workflows to identify missing legal knowledge and false assumptions.
Build Legal evaluations by assessing industry benchmarks to measure data strategy success, and advise cross-team evaluation efforts on legal coverage and realism.
Scale expertise by recruiting and guiding Subject Matter Expert (SMEs), acting as the domain authority to enforce vendor quality, and converting expert contributions into usable training data.