Senior Data Scientist

Job description

Responsibilities: Provide strategic insights and recommendations to senior leadership through in-depth statistical analysis and modeling. Design, build, and maintain the metrics, models, and reporting that track agent quality, reliability, adoption, and unit economics for stakeholders across the Agents and Operations teams. Build evaluation and experimentation as a first-class discipline—datasets from production traces, rubrics, automated graders, regression suites, and the experiment design and analysis that prove causation—so every agent improvement we ship is backed by evidence. Identify key business challenges and opportunities—including agent failure modes, tool-use patterns, and cost and latency—and build statistical and machine-learning models to drive product improvements and growth initiatives. Architect scalable analytics and modeling infrastructure to ensure data integrity, governance, and accessibility for both human and agent consumers. Oversee the development and maintenance of Traba’s data warehouse to ensure data availability and governance. Work closely with the Agents team and Operations leadership to understand their data needs and provide actionable, statistically grounded insights that drive continuous process improvement and operational efficiency. Provide Operations teams with models, self-service analytics, and advanced technologies—including AI-assisted tools—enabling them to independently analyze operational data and optimize their daily activities. Mentor the scientists and analysts who build alongside you, and set the standards that define what “good” looks like for measurement, modeling, and experimentation at Traba. Qualifications: Experience: 4-8 years in data science, machine learning, applied statistics, or quantitative research, with 2+ years of hands-on work modeling or measuring LLM- or agent-based systems in production. Education: BS/MS/PhD in data science, statistics, machine learning, computer science, mathematics, economics, or a related quantitative field (or equivalent work experience). Technical Skills: Strong proficiency in Python and common ML and statistics libraries (e.g., scikit-learn, PyTorch, pandas, statsmodels). Strong proficiency in SQL. Experience designing and analyzing experiments (A/B testing) and applying statistical inference or causal methods. Experience with LLM evaluation and observability tools like Langfuse, Braintrust, or internal harnesses, and with building automated evaluators. Communication Skills: Excellent data storytelling skills to effectively engage with stakeholders. Collaboration Skills: Strong ability to work across departments, identifying and prioritizing analytics problems to deliver actionable insights. Curiosity and Initiative: Intense curiosity to ask “why?” and use data to find answers, combined with a “no task too small” mentality. Self-Motivation: Ability to work independently and as part of a team in a fast-paced startup environment.