- 6+ years of experience in data engineering or big data technologies.
- Strong hands-on experience with:
- Azure Databricks
- Azure Data Factory (ADF)
- Apache Spark (PySpark/Scala)
- Good understanding of distributed computing and data processing models.
- Experience with Azure Data Lake Storage (ADLS Gen2).
- Strong SQL skills and experience with data modeling.
- Experience in building scalable ETL/ELT pipelines.
- Knowledge of data partitioning, performance tuning, and optimization techniques.
- Familiarity with CI/CD pipelines (Azure DevOps).
Preferred Skills
- Experience with Delta Lake and Lakehouse architecture.
- Knowledge of streaming data processing (Structured Streaming, Kafka, Event Hub).
- Experience with Power BI or other visualization tools.
- Familiarity with Python, Scala, or SQL-based programming.
- Understanding of data governance and security frameworks.
Soft Skills
- Strong analytical and problem-solving skills.
- Good communication and stakeholder management abilities.
Ability to work independently and in a collaborative team environment.
Preferred Qualifications:
- BS degree in Computer Science or Engineering or equivalent experience