Note: The job is a remote job and is open to candidates in USA. ARK Solutions, Inc. is seeking a Databricks Engineer to design, develop, maintain, and optimize scalable data pipelines and ETL processes. This role is crucial for modernizing data platforms and supporting enterprise data initiatives.

Responsibilities

• Create and maintain ETL pipelines and/or migrate existing legacy pipelines (e.g., synapse) to Databricks
• Develop scalable data processing solutions using Python, PySpark, and SQL
• Perform data transformation, cleansing, and validation to ensure data quality and consistency
• Optimize data workflows and Spark jobs for performance and efficiency
• Collaborate with business stakeholders, data architects, and development teams to understand requirements and deliver effective solutions
• Monitor, troubleshoot, and resolve data pipeline issues

Skills

• Hands-on experience with Databricks
• Strong programming skills in Python and PySpark
• Strong SQL development and query optimization experience
• Experience designing, developing, and maintaining ETL pipelines
• Experience working with large-scale data processing and analytics solutions
• Strong analytical, troubleshooting, and problem-solving skills
• Excellent communication and collaboration abilities
• Databricks
• Python
• PySpark
• SQL
• Bachelor's Degree

Company Overview

Company H1B Sponsorship