Roles:
- Data Engineer with 2 to 3 years of experience in Building and Supporting Cloud Based Data Pipelines.
- Developing reliable data flows, enabling analytics and reporting and maintaining an efficient data processing System using Azure Databricks.
Responsibilities:
- Build and Maintain Automated Batch and near-real-time data pipelines
- Clean, transform and vailidate data using PySpark and SQL.
- Organizing Data for reporting and analytics consumption
- Implement basic real time/streaming data procwessing
- Troubleshoot pipelines issue and optimize performance
Requirements:
- Azure Databricks
- PySpark, Python, SQL
- Azure Datalake Storage (ADLS) and Unity Catalog
- Strong scripting skills in Python, PySpark and SQL
