Monday 13 May 2013

How to use Slowly Changing Dimensions (SCD) in pentaho

What is SCD ?

Slowly changing dimensions are the dimensions in which the data changes slowly, rather than changing regularly on a time basis.

SCD Type 1: Overwrite
---------------------------------------

A type 1 slowly changing dimension is the most basic one and doesn’t
require any special modeling or additional fields.
SCD type 1 columns just get overwritten with new values when they come into the data warehouse.
Example
--------------------------------------------------------------------------------------------------------------------------
Customer _key        Customer _id     Customer _Name       Customer _City
    1                               22321                Abhishek                      Delhi
--------------------------------------------------------------------------------------------------------------------------

After applying SCD Type 1

---------------------------------------------------------------------------------------------------------------------------
 Customer _key          Customer _id      Customer _Name      Customer _City
       1                                22321                Abhishek                Banglore
-----------------------------------------------------------------------------------------------------------------------------


SCD Type 2: Add Row
---------------------------------------
keep historical Data as well as current data by adding one new row.

This method tracks historical data by creating multiple records for a given natural key in the dimensional tables with separate surrogate keys and/or different version numbers.
Unlimited history is preserved for each insert.
 -----------------------------------------------------------------------------------------------------------------------
Supplier_Key  Supplier_Code Supplier_Name Supplier_State Version.
123   ABC Acme Supply Co CA 0
124    ABC Acme Supply Co IL 1
 ------------------------------------------------------------------------------------------------------------------------

SCD Type 3: Add Column
----------------------------------------------

This method tracks changes using separate columns and preserves limited history. 
The Type II preserves unlimited history as it's limited to the number of columns designated for storing historical data.
The original table structure in Type I and Type II is the same but Type III adds additional columns. In the following example, an additional column has been added to the to record the supplier's original state -
only the previous history is stored.

 -------------------------------------------------------------------------------------------------------------------------------

Supplier_Key Supplier_Code Supplier_Name    Original_Supplier_State Effective_Date    Current_Supplier_State
123 ABC Acme Supply Co          CA 22-Dec-2004              IL

 ---------------------------------------------------------------------------------------------------------------------------------






No comments:

Post a Comment