Lead Data Engineer

What You Will Do

Lead and optimize the GP data platform (Enterprise Data Lake, Ingestion Framework, Databricks), ensuring reliability, integrity, and cost efficiency data and data infrastructure
Collaborate with cross-functional teams to deliver features, maintain a healthy agile process, and enhance platform accessibility for customers
Drive automation, operational excellence, and continuous improvement through performance monitoring, issue resolution, and development of self-service tools

Who You Are (Basic Qualifications)

Experience designing, building, and maintaining large-scale distributed systems
Experience serving as a senior technical leader overseeing multiple data engineering teams
Experience in architectural design and systems integration for multi-stakeholder, enterprise-grade platforms
Experience leading operations for highly available data platforms and data lakes
Demonstrated experience in data infrastructure, including proficiency with Infrastructure as Code practices and skilled in GIT and CI/CD design patterns
Must have legal authorization to work permanently in the United States for any employer without requiring a visa transfer or visa sponsorship

What Will Put You Ahead

Delivered innovative, scalable data solutions using Python, SQL, Spark, and Delta Lake, while optimizing data accessibility with open table formats (Delta, Hudi, Iceberg)
Enhanced system efficiency and reliability by leveraging Terraform, Git/CI-CD workflows, and DAG tools (Airflow, Step Functions, Databricks Declarative Pipelines)
Implemented and optimized the Databricks Data Platform and AWS services (Lambda, Glue, EventBridge, ECS, Kafka, etc.) to support business objectives