Track record

Responsible for the greenfield implementation of Databricks and the development of an AI-driven prospecting and ICP scoring pipeline at a SaaS company.

• DevOps setup: Git repositories, CI/CD pipelines and Databricks Asset Bundles
• Ingestion pipelines for semi-structured application telemetry using Databricks streaming tables, for product analytics
• Used and implemented dbt for dimensional data modelling
• Prospecting / ICP scoring pipeline enriching and transforming semi-structured data into usable ICP scores
• Fivetran for standardised, low-maintenance ingestion from multiple SaaS sources

Worked as a data engineer building a data platform using Azure Synapse, Azure Data Factory, Azure Databricks and Azure Analysis Services.

• Azure Databricks infrastructure of 15+ workspaces plus Unity Catalog, set up with Terraform
• AI-driven migration of 400+ Synapse mapping data flows to PySpark notebooks
• DevOps setup: Git repositories, CI/CD pipelines and Databricks Asset Bundles
• Ingestion pipelines for a variety of source systems, structured and unstructured
• Metadata-driven historical data layer (SCD type 2) using Databricks Delta
• Exploration of a project to collect private-lease competitor data
• Set up a dbt project for BI workloads

Worked as a solution architect to design and implement a new solution for operational reporting on multiple sources.

• Analysis of existing reporting solutions in SSRS and Excel using a replica of Navision
• Design of the reporting platform using Azure Synapse: pipelines, Spark notebooks, serverless SQL and Delta
• Design of the ingestion for SAP and Navision data
• Advice on the types of data products needed

Worked as a data modeler on the migration from Teradata combined with SAS CI to Microsoft Azure combined with Selligent for marketing automation.

• Created conceptual, logical and physical data models
• Analysed and translated definitions from the old environment to the new environment
• Data validation between Microsoft Azure and Teradata