Senior Data & Backend Engineer | Scalable ETL Pipelines | Python & Big Data Expert
Hello! I am Atul Bisht, a Senior Software Engineer with a deep specialization in Data Engineering, Generative AI, and Backend Development. Over the past 3.5+ years, I have built and optimized large-scale data systems for rapidly scaling logistics and tech platforms.
My expertise lies at the intersection of high-performance data processing and robust cloud infrastructure. Whether it’s migrating 500+ complex pipelines from GCP to AWS with zero data loss or tuning Spark applications to reduce runtimes by 75%, my goal is always to build systems that are not just functional, but highly efficient and cost-effective.
What drives me: I believe that data is only as good as the infrastructure that supports it. This is why I focus heavily on "Right-Sizing" compute resources, ensuring data quality through Unified Frameworks, and building self-healing systems with Airflow and Kubernetes. I don't just "move data"—I architect scalable assets that power business decisions.
Technical Core:
Big Data: Apache Spark (PySpark), Kafka, Trino, Delta Lake, HDFS.
Cloud & DevOps: AWS, GCP, Huawei Cloud, Kubernetes, Terraform, Ansible.
Orchestration & Observability: Airflow, Grafana, Metabase.
Backend: Python, SQL, PostgreSQL, React, Node.js.
I prioritize clean code, comprehensive documentation, and proactive communication. Outside of engineering, I am an avid open-source contributor and enjoy mentoring aspiring developers. Let’s collaborate to build something that scales!
Work Terms
Availability & Communication:
Hours: 20-30 hours per week. I am generally available for communication Monday through Friday, with flexibility for urgent weekend milestones.
Channels: I am comfortable communicating via Guru Messenger, Slack, or video calls for initial project scoping and weekly alignment.
Project Management & Payment:
Milestones: I prefer working with clearly defined milestones and Guru’s SafePay system to ensure transparency and security for both parties.
Engagement: I am open to both short-term optimizations (e.g., Spark tuning, pipeline fixes) and long-term architectural builds.
Commitment: I provide weekly progress reports and am committed to rigorous testing and observability. For every project, I ensure a smooth hand-off with clean documentation so your internal team can manage the systems I build with ease.