Principal Data Engineer/Architect

Primary Responsibilities:
As a senior member of the Cloud Data Engineering team, the candidate should provide:
The technical leadership for all the data engineering efforts on public cloud for the Optum Rx Data Estate scrum team.
Will design and build scalable and cost-effective public cloud (Azure) native solutions as per the business use cases
Should be hands-on engineer with experience in multiple data engineering technologies (Spark/Kafka/ADF/Databricks)
Standard dev-ops technologies and processes (CI/CD- Containers, Jenkins etc.)
As a tech lead, he should be responsible for the deliverables of the team working closely with the product owners and the delivery leads

You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.
Required Qualifications:
Engineering Degree in computer science or equivalent experience
10+ years of Data Engineering experience
5+ years of Data engineering experience in Spark
3+ years on Cloud native data engineering technologies
Experience using AI/ML
Technical skill sets:
Experience in Dev Ops and CI/CD - Jenkins, Terraform (Azure) and containers (Docker)
Work experience on Proof of Concepts for AI/ML
Hands-on experience on building data engineering pipelines on Spark/ Data bricks (Azure)
Hands-on experience with stream data ingestion /processing with Spark
Hands-on experience on other Azure native data technology stack - Azure Data factory, Synapse, Snowflake
Exposure to providing ETL solutions for Azure native stack and in Spark
Good exposure on Enterprise Data Lake (Delta Lake) and Data Warehouse concepts

Preferred Qualifications:
Experience working with business partners for requirement gathering, scoping, estimation, and execution plan for various capabilities
Experience working in an Agile environment with multiple Scrum and Kanban teams

Skills Required
10+ years of Data Engineering experience
5+ years of Data engineering experience in Spark
3+ years on Cloud native data engineering technologies
Engineering Degree in computer science or equivalent experience
Experience using AI/ML