Experience in developing pipelines with:
- Azure data factory
- Databricks (Spark)
- Azure Data Lake Service
- Azure Event Hub
- Azure Logic Apps
- Google Dataproc
- Google Pub/Sub.
- Google Cloud Storage
- Orchestration with Airflow, Dagster, Perfect.
- Google Cloud Functions
- Managing web services for event based processing.
Experienced in both Scala and Python.
Also experienced in Data Modelling, Data Warehousing and Data Lake Architectures