Azure Data Factory : Implement SCD Type 1

This project will teach you how to implement the most common concept in real-world projects, namely. Slowly Changing Dimension Type 1, using Azure Data Factory. Prerequisites: Azure subscription Azure data factory knowledge (Basic). The following tasks are covered in this project. Task 1: Understand Slowly Changing Dimension Type 1 We will attempt to understand the concept and the different types of Slowly Changing Dimension. However, we will be focusing on Type 1 by using a simple example. Task 2: Create Azure Services like Azure Data Factory, Azure Database. In this task, you will create the azure services such as azure SQL database and azure data factories that will be used in the later tasks. Azure sql database will contain the staging, dimension and data tables, while azure SQL factory will be used to create the data pipe. Task 3: Create Staging & Dimension Table in Azure SQL Database. In this task we will create the dimension and staging tables in azure SQL database. We will also insert some dummy data into the staging table. Task 4: Create an ADF pipeline to implement SCD Type 1 - Insert Logic. In this task, you will create the pipeline in azuredata factory and implement the logic for inserting new records that exist in the dimension but not in the staging table. This is one use case for SCD Type 1. Task 5: Create an ADF pipeline to implement SCD Type 1 Update Logic. In this task, you will create the pipeline in azuredata factory and implement the logic for updating records in the staging table and in dimension. This is another scenario of SCD Type 1. Task 6: Demonstration of ADF pipeline. This is the final task. !