I am a reliable data engineer with 10+ years of proven industry experience in data lake development, data analytics, real-time streaming, and back-end application development. I have built exceptionally stable solutions for high-traffic, high-visibility projects, and understand what it takes to ensure products are robust and dependable. I also have expertise in the Apache Spark ecosystem, Elastic Search, ETL, Databricks, AWS Glue, DMS, Athena, EMR, Data Lake, AWS Big Data, Apache Kafka, Python, Java, SQL, NoSQL, etc.
Work Experiences in the below technologies:
- Big data processing using Spark Scala
- Building large-scale ETL and Data Transformation
- Databricks (Unity Catalog, ETL, Delta Live Table, Orchestration, Streaming, etc.)
- Apache Spark
- Search Engine solutions using Elasticsearch
- Distributed platform development
- Machine learning
- Python Programming
- Algorithm Development
- AWS glue - Pyspark
- Data Conversion (Excel to CSV, PDF to Excel, CSV to Excel, Audio)
- Data Mining
- Data extraction
- Data Cleansing
- Linux Server Administration
- Website & Data Migrations
- DevOps (AWS, AZURE,GCP) and Cloud Server Management