I have over 9 years of experience in the field of data engineering and have worked extensively with Big Data, Cloud, and Analytics. My professional expertise lies in developing and implementing data-intensive applications and decentralized solutions.

In my most recent role as a Technical Lead at Encora Innovation Labs, I have worked with a variety of tools including Hadoop, Spark, Hive, and GCP. My accomplishments include identifying Nifi as a data ingestion tool for the current data pipeline, improving performance in complex Hive queries, and enabling the data pipeline in Google Cloud Platform.

Prior to that, I worked as a Software Engineer at Optum Insights India where I migrated the data pipeline from Oracle to Hadoop ecosystem using Apache Spark, built data pipelines in AWS environment, and identified performance bottlenecks in Spark applications during migration and improved them.

As a Big Data Engineer at Comcast India Engineering Center LLP, I developed a Spark job to extract insights from the Splunk server, identified behavioral patterns for critical issues in GA image leading to root cause analysis, and integrated AWS with Hadoop to increase the efficiency of Spark applications.

In my early career, I worked as a Data Associate at Amazon Development Center where I created Spark Jobs for parsing input data from cloud (S3, Redshift) and processing it for improving customer recommendations. Additionally, I used the imported data to do failure analysis on transactions with optimized Hive queries and created custom UDF for analytical insight based on the requirement.