Azure Data Engineer | ADF, Databricks, PySpark | Building Scalable & High-Performance Data Pipelines
I am a Senior Data Engineer with 11+ years of IT experience, currently working at Big 4 company, specializing in designing and building scalable, high-performance data platforms on Microsoft Azure.
Over the past 4+ years, I have focused on Data Engineering, working extensively with Azure Data Factory (ADF), Azure Databricks, ADLS Gen2, PySpark, Python, and SQL. I have successfully built end-to-end data pipelines, optimized large-scale data processing systems, and implemented robust data models for analytics and reporting.
My core strengths include:
• Designing efficient ETL/ELT pipelines using ADF and Databricks
• Developing scalable data processing solutions using PySpark
• Performance tuning and cost optimization of big data workloads
• Data modeling for analytics (star schema, dimensional modeling)
• Working with structured and semi-structured data (JSON, Parquet, Delta)
• Implementing data quality, validation, and monitoring frameworks
Before transitioning into Data Engineering, I spent 7+ years in Automation Testing, which gives me a strong edge in building reliable, well-tested, and production-ready data solutions.
I am highly focused on delivering clean, optimized, and business-driven solutions. I believe in clear communication, timely delivery, and long-term client relationships.
Let’s work together to turn your data into actionable insights.
Work Terms
• Availability: 10-15 hours per week (Flexible based on project needs)
• Time Zone: IST (Available for overlap with US/UK time zones)
• Communication: Slack, Microsoft Teams, Zoom, e-mailing
• Project Updates: Daily/Weekly status reports with progress status
• Payment Terms: Hourly or Fixed Price (Milestone-based preferred for long-term projects)
• Response Time: Within 12–24 hours
I prioritize transparency, quality delivery, and long-term collaboration.