Senior Data Engineer at Amazon with 5+ years of experience in big data and Worked in various cloud platforms like AWS, GCP, Azure. Developed and scheduled highly efficient production ETL pipelines capable of handling on Terabytes of data.
I'll do data modelling, ETL pipeline design and development, data engineering using spark, pandas and other data engineering activities
Programming Skills
Big Data Skills
- Apache Spark (DataFrame and Datasets)
- Apache Hive
- Apache Airflow
- Hadoop
- PySpark
- Apache Kafka
- Delta lake
- Databricks
- HDFS
Cloud Skills
- AWS Redshift
- AWS S3
- AWS Lambda
- EMR
- EC2
- Kinesis
- RDS
- GCP Compute cloud
- Big query
Other Skills
- Pandas (python-pandas)
- MySql
- Postgres
- RestAPI
If you're looking for someone to build pipeline from scratch or enhance or improve performance of existing pipelines, you're just a ping away from accomplishing it.