EFS FCRM 169474

Key Responsibilities: Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Required Skills: Experience in data engineering with a strong focus on Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Hands-on experience with cloud platforms Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning. Experience with ...

Ready to Apply?

Submit your application today and join our talented team at ADP.

Submit Application