DimensionMergeSCD Wiki & Documentation Rss Feed

New Comment on "FAQ"

benschmucker — Thu, 23 Aug 2012 20:31:24 GMT

I've been using the component and it is working great. I'm currently at a client that is moving from SQL Server 2008R2 to SS2012. I cannot get the component to show up in the toolkit (I understand you have to add it manually in 2008/2008r2, but in 2012 it adds these automatically and I cannot see it)... Any ideas?

New Comment on "FAQ"

bcruz03 — Wed, 28 Dec 2011 15:56:16 GMT

I installed the component on a VSS Team Sytem 2008 and when selecting to add custom toolbox items the VSS closes. Tried re-intsalling the product with same result. Do you have a solution to the problem? Thanks

New Comment on "FAQ"

Codersvalley — Mon, 18 Jul 2011 15:17:50 GMT

Q: Is the Component intended for dimension tables only, is not is there any other example to load fact tables using kimbal's method.

Updated Wiki: Home

toddmcdermid — Tue, 10 May 2011 22:07:50 GMT

Project Description

A custom Data Flow component for SQL Server Integration Services (SSIS) that replaces the standard SCD Wizard with a superior experience, from the configuration UI to runtime performance. Performs 100x faster than the standard component, and edits are non-destructive.

The Slowly Changing Dimension Wizard in SQL Server Integration Services is a very useful tool for setting up and handling slowly changing dimension processing, which is a very common requirement of Data Warehousing. The SCD Wizard is easy to configure and works as advertised.
However, there are some significant issues with it that have been experienced by many:

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

Pragmatic Works has recently committed to supporting the Dimension Merge SCD! They have incorporated the code into their Task Factory suite of extensions for Integration Services. If you can't use an "unsupported" open-source extension, check out Pragmatic Works - I think you'll find it too compelling to ignore.

New walkthrough videos of the v1.6 release are now available on Youtube, and in full-size/full-quality on my SkyDrive:

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with the Dimension Merge SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.

New Comment on "Upgrading Instructions"

dave_lumley — Mon, 18 Apr 2011 14:14:54 GMT

Todd, Tried to install the 1.6 version on my test server and got the following error when trying to bring the component on the dataflow area. Could not load type: MouldingAndMillwork.SSIS.DimensionMergeSCDEditorInterface,DimensionMergeSCD100,Version=1.0.0.0,Culture=neutral,PublicKeyToken=8b0551303405e96c. Verify that the component editor is installed properly. (Microsoft Visual Studio) FYI I did stop and restart both SSIS service and BIDS. How can I fix the issue?

Updated Wiki: Documentation

toddmcdermid — Wed, 13 Apr 2011 19:47:34 GMT

What you should do:

If you have any questions, review the FAQ, then post your question or problem in the discussions.

Updated Wiki: Documentation

toddmcdermid — Wed, 13 Apr 2011 19:47:21 GMT

What you should do:

If you have any questions, review the FAQ, then post your question or problem in the discussions.

Updated Wiki: Home

toddmcdermid — Wed, 13 Apr 2011 19:47:06 GMT

Project Description

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

This component provides an improved design-time, maintenance, and execution experience for processing slowly changing dimensions. It provides more options to support complex dimension table processing while executing over 100x faster than the Wizard.

New walkthrough videos of the v1.6 release are now available on Youtube, and in full-size/full-quality on my SkyDrive:

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with the Dimension Merge SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.

Updated Wiki: Home

toddmcdermid — Wed, 13 Apr 2011 19:37:43 GMT

Project Description

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with the Dimension Merge SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.

Updated Wiki: Documentation

toddmcdermid — Wed, 13 Apr 2011 19:35:20 GMT

What you should do:

If you have any questions, review the FAQ, then post your question or problem in the discussions.

Updated Wiki: v1.6 Component Optimization

toddmcdermid — Wed, 13 Apr 2011 19:35:13 GMT

Dimension Merge SCD Optimization

There are a few techniques that you can use to improve performance of the component. Some of these techniques will also work for other ways of processing SCDs.

Basic Optimization

The component itself will suggest that you do certain things to improve performance - sort the inputs according to the Business Key, and provide it with a guaranteed seed value for the Surrogate Key (should you configure the component to manage it).

Sorting Inputs

The component should work best when the inputs are sorted according to the business key. The sort order does not have to be exactly the same as the business key columns - but of course, making it that way is the simplest option. As always, it is a best practice to avoid using the SSIS Sort component - try to read in the flows in optimal sorted order.
In those cases where you can retrieve the data in some form of order - but that order does not exactly match the business keys, you may still be able to use it. If columns "A" and "C" are (in that order) the Business Key of the flow, having the flow sorted by columns "A, B, and C" is acceptable. However, the sort specification of one input must also be a subset of the sort specification of the other input. This means that if one input is sorted "A, B, C", then it's OK for the other input to be sorted "A, C" or "A, C, D", but it is not OK for the other input to be sorted "C, A".

Providing a Guaranteed Surrogate Key Seed

If the component is managing Surrogate Keys for you, then it will likely need to assign new surrogate keys - to all rows coming out of the New and New SCD2 outputs. In order to do so, it needs to know what values to use. To determine what seed value to use, it starts with the value you supply in the editor, then watches all the rows that enter the component from the Existing Dimension input, incrementing the seed you supplied if necessary. Since the component can't know if the seed it currently has won't result in a key collision unless it examines all of the rows on the Existing Dimension input - it holds up all processing until it's seen every row. This can result in very high memory usage, as all of the Existing Dimension input rows are cached.
In order to optimize that scenario, it's suggested that you determine the "next" Surrogate Key value through another Task prior to the Data Flow Task that contains the Dimension Merge SCD. An Execute SQL Task with a SELECT MAX statement usually does the trick, but a Data Flow Task to extract the highest key value used may be necessary (or lowest key - depending on your keying mechanism). Store that "next" key value in a variable, supply it to the Dimension Merge SCD component, and tell the component it's guaranteed. This frees the component from ensuring that surrogate key collisions won't happen, and allows it to process rows immediately.

Advanced Optimization

The following techniques require an intimate understanding of the SCD process, your business needs for the dimension table, the technical capabilities of your source system, and SSIS' Data Flow architecture. These optimization techniques are not guaranteed to work for you. If this stuff was easy, the tool(s) would already do it for you - it's not, so they can't.

Only Use Current Dimension Rows

Typical usage involves sending ALL rows from your dimension table through the Dimension Merge SCD to make persisting SCD type 1 changes easy by using the surrogate key. However, this can add unnecessary overhead. If only "current" records are retrieved from your warehouse, you can perform the type 1 updates by using the business key.

Use CDC

If your source system is on SQL Server 2008 or later, or you have a CDC (change data capture) technology or technique for your source system, you can use that to your advantage. (Maintaining a "last updated" column on OLTP tables is one such technique.)
If you supply the component with ONLY the changed rows from the source system, it will have (hopefully) much fewer rows to process. However, if configured normally, this will result in the otherwise unchanged dimension rows being sent out the "Expired" or "Deleted" outputs. There is no possible mitigation for this result without performing lookups on the source system, which defeats the purpose of CDC. Your usage of the component - and utility of the resulting dimension table - will be diminished because you can no longer expire records when they have truly been removed from the source system. (Unless you have a method of tracking this change separately!)

Hashing Business Keys

If your business key is large and/or has multiple columns, then the Dimension Merge SCD component has a lot of memory to move around when comparing rows to each other. Hashing business keys (with the Multiple Hash component, for example) may improve this comparison performance significantly.

Buffer Sizes, etc.

The typical optimzation advice to increase or reduce the buffer sizes may or may not have any effect on the Dimension Merge SCD component. Internally, the component does not use SSIS buffers for caching information, and does not adhere to any of those settings. However, the component's interaction with SSIS buffers are governed by these settings, and adjusting them up or down may improve memory usage or performance - slightly. I would not expect changes to have any significant effect.

Coordinating Input Flow Arrival

The largest opportunity for improving data flow performance is in coordinating when rows arrive to the component. The effect of any adjustments here depend on many factors - here are some guidelines:

If either your dimension table OR your source system flow has significantly fewer rows than the other, attempt to arrange for the smaller flow to completely arrive at the component as soon as possible. Once one of the inputs has completely arrived at the component (the immediately upstream component turns green), then the Dimension Merge SCD can optimize cache handling even more than it does normally. How do you do this? Make sure your smaller flow is as direct and high-performance as possible. For example, place any transformation of that flow into another Data Flow Task, and write to a RAW file. Read directly from the RAW file into the Dimension Merge.
Use the FlowSync component from SQLBI.EU to feed the component rows proportionally. The Dimension Merge SCD has to cache rows that it can't find matches for. If it receives rows (from sorted inputs) at about the same rate, then it's likely this caching can be reduced.

Updated Wiki: v1.6 Usage Instructions

toddmcdermid — Wed, 13 Apr 2011 19:15:24 GMT

Usage Instructions

Before you start attempting to use the Dimension Merge SSIS Slowly Changing Dimension component or the SCD Wizard, please do your background learning on dimensional modeling and what a slowly changing dimension is. Here are some resources:

Wikipedia's entry on Slowly Changing Dimensions.
Ralph Kimball on Slowly Changing Dimensions: Part One and Part Two.

If you happen to be familiar with the SCD Wizard's terminology and not Kimball's - all you need to know is that a "Changing" attribute is a Type 1, and a "Historical" attribute is a Type 2.

Overview

For a quick runthrough of how to use the component, watch the Overview video (on YouTube, or download the full-quality WMV). See the Documentation page for the full series of videos.

Inputs

Existing Dimension

The flow you connect to this input should be from your data warehouse's dimension table. Use whatever connection manager and source component you wish - the input connection is a data flow, not a connection manager. You will want to provide all rows, and likely all columns from your dimension table to the component - especially the "housekeeping" columns, such as the SCD2 effective and expiry dates.

Source System

The flow you connect to this input should be from your source OLTP system. It's likely you will have transformed the information coming from that source somewhat, in order to generate the attributes you'll need to compare and either insert or update in your data warehouse's dimension table. The format of the source system flow should match the data types of the existing dimension flow.

Special Members

This flow is entirely optional, but provides a clean way of specifying and maintaining any possible "special" members in your data warehouse dimension table. In typical dimensions, there is an "unknown" member that you can use to link fact table rows to when the fact table row refers to a NULL or otherwise unknown dimension value. In some cases, you may have several of these artificial special members in your dimension.
It is completely acceptable to preload your dimension table manually with these members, and/or manually add them to your source system on every load. However, this method does result in those special members acting identically to any other source row when you may not want them to behave in that manner. Typically, all changes to special member rows should be treated as Type 1 - fixing a mistake. The reason for this is that your "unknown" member is "special" and always represents "unknown". If an attribute of an "unknown" member needs to be changed, it's highly unlikely that you would want two versions of "unknown". It's much more likely that you are changing the "unknown" member to clear up confusion or improve its behaviour - which means you would want all "versions" of the member to be revised.

Outputs

The component does not perform any lookups on your dimension table, or persist any changes itself. Instead of performing lookups on the Dimension table, it compares the data you've provided it via the Existing Dimension input. Just like the SCD Wizard, it does not persist any changes. However, unlike the SCD Wizard, it will not create the data flow transformations and destinations needed to persist those changes - that's up to you. (Since the component doesn't know where your dimension table is.)
You may not need to use all of the outputs listed below. The outputs need to be treated differently - please make sure you understand SCD processing, and have watched the video series to understand (in general) what the outputs are used for.

Unchanged

All rows from your dimension table that haven't been changed by the source system will be output here. The rows on this output are typically discarded, but you may want to do some auditing/troubleshooting work with them.

New

Any row from your source system that didn't exist in your dimension table will be output here. The rows on this output are typically inserted directly into your dimension table as the "first version" of this business entity. This output does not include new versions of rows that have SCD changes.

Deleted

Any row from your dimension table that no longer exists in your source system will be output here - but only if your SCD process isn't tracking type 2 changes. If your SCD processing is only tracking type 1 changes, then deleted records will be output here. It's really undefined what you should do with these records... your data warehouse shouldn't be deleting data that's historically relevant... but since you're not tracking type 2 changes, there's no way of marking this record as retired in your warehouse. Rows coming out here should indicate an architecture problem with your data warehouse.

Expired SCD2 with SCD1 Changes

Any row from your dimension table that has SCD type 2 columns changed by the source system will generate two rows from this component. One of those rows will be sent out of this output - the "old version" of the row. All SCD type 1 columns will be set to the "new" values, and the expiry column will be set. It's typical to attach an OLE DB Command transformation to this output... although it's recommended that you send rows to a temporary table instead, and use a set-based UPDATE statement in an Execute SQL Task to perform the update.

New SCD2

Any row from your dimension table that has SCD type 2 columns changed by the source system will generate two rows from this component. One of those rows will be sent out of this output - the "new version" of the row. It's typical to send all these rows to an OLE DB Destination component for direct insertion into the dimension table.

SCD1 Updated

Any rows that have SCD type 1 changes to them (retroactive history changes) will be output here - except for "old versions" of rows that had SCD type 2 changes made. It's typical to attach an OLE DB Command transformation to this output... although it's recommended that you send rows to a temporary table instead, and use a set-based UPDATE statement in an Execute SQL Task to perform the update.

Invalid Input

Any rows that the component detects problems with - NULL business keys, NULL values where you've said there shouldn't be any, etc - are sent out this output. It's imperative that you record this information somewhere. Failure to do so will undoubtedly mean your SCD process is unreliable, and therefore your dimension table contents are suspect.

Auditing

Auditing information - input and output row counts, start and end times, average and maximum cache levels, etc - are sent on one row from this output. Best practice says you should be using some of this information for auditing purposes, but you may not need to use it all. It's typical practice to send this row to an OLE DB Command transform to update a row in an ETL audit table.

Statistics

This output will get sent performance information that's accumulated during the component operation. For package diagnosis, it's very informative to hook this up to a CSV or database table destination for later analysis. Be aware that rows are sent out this output in batches (due to the buffer structure of the data flow), and aren't written to the destination in real-time.

Updated Wiki: v1.6 Installation Instructions

toddmcdermid — Wed, 13 Apr 2011 18:53:39 GMT

Installing the Dimension Merge Slowly Changing Dimension Component

See Upgrading Instructions in order to upgrade from prior versions.
The installation instructions are the same for installing this component in Integration Services 2005 or 2008. Please note that you must run the appropriate installer for the version of SQL Server Integration Services that you have installed on your system, and that packages created for Integration Services 2005 will not run on SSIS 2008 and vice versa.

Development Machine Instructions

Follow the instructions below to install and use the component on any machine you intend to develop Integration Services packages on.

Extract the appropriate installer (2005 or 2008, x86 or x64) from the ZIP file you downloaded from the releases page.
Double-click on the MSI to launch it, and complete the installation.
Open BIDS (Business Intelligence Development Studio, AKA Visual Studio).
Open or create an Integration Services project.
Add a Data Flow Task to your package and double-click on it.
Right-click on the Toolbox and select “Choose Items...”
Select the “Data Flow Components” tab, and place a checkmark next to “Dimension Merge Slowly Changing Dimension”.

Server Machine Instructions

Follow the instructions below to install the component on any machine you intend to run packages containing the component.

Extract the appropriate installer (2005 or 2008, x86 or x64) from the ZIP file you downloaded from the releases page.
Double-click on the MSI to launch it, and complete the installation.

Installation Troubleshooting

If you have any difficulties with the above instructions, you can:

Continue reading to understand what the installer does.
View the Troubleshooting FAQ.
Post a message in the Discussions to ask for help.

Installer Details

The installer will proceed to place two .Net DLLs in the GAC (Global Assembly Cache) – you will require Administrator access on Vista/Win7 to permit this, and UAC will probably pop up to confirm you trust the installer’s actions. It will also place a copy of the DLLs in the default location for SQL Server Integration Services Data Flow Components:

For SSIS 2005 – <ProgramFilesFolder>\Microsoft SQL Server\90\DTS\PipelineComponents
For SSIS 2008 – <ProgramFilesFolder>\Microsoft SQL Server\100\DTS\PipelineComponents
For SSIS 2005 and 2008, if you run the 64-bit installer on a 64-bit system, the DLLs will be placed in BOTH 32-bit and 64-bit Program Files folders.

Uninstalling the Component

Use the Add/Remove Programs function in Control Panel to remove the component.
Open an Integration Services project in BIDS.
Right-click on the Dimension Merge Slowly Changing Dimension item in the Toolbox, and select Delete.

Manual Installation Instructions

If using the included installer isn't something you want to do, or it's not working for you, the component can be installed manually.

Download and build the source code, or copy the DLLs from the failed installation.
Install DimensionMergeSCDxxx.DLL and SSISComponentUtilitiesxxx.DLL into the GAC (where xxx is "100" or "90" depending on your SSIS version). You will need GACUtil in order to accomplish this - it is not included in standard Windows installations. You must install a .Net SDK or Windows Platform SDK in order to get this utility.
Copy the DimensionMergeSCDxxx.DLL to the PipelineComponents folder identified above for your version of SQL Server.

If you alter the source code, you must re-GAC and re-copy all of the binaries to the above locations. If you do not, then you will experience inconsistent behaviour with the component!

New Comment on "v1.5 Installation Instructions"

MikeHoney — Mon, 11 Apr 2011 00:19:46 GMT

Using SQL Server 2008 R2, step 7 under Development Machine Instructions is slightly different. You need to choose the "SSIS Data Flow Items" tab.

New Comment on "v1.5 Installation Instructions"

MikeHoney — Mon, 11 Apr 2011 00:17:03 GMT

This page appears to be out-of-date for release 1.6 - the component name change from v1.5 has affected several specific object names e.g. the DLL name and component name. Generally if you replace "Kimball Method" with "Dimension Merge" you can figure it out.

Updated Wiki: Upgrading Instructions

toddmcdermid — Fri, 08 Apr 2011 17:27:53 GMT

Upgrading Instructions

From v1.5 to v1.6

Download and install the appropriate release for your system.
For each DTSX package that uses the component, perform the following manual steps:
1. Back up each package
2. Open the DTSX file in a text editor, such as Notepad.
3. Search and replace occurrences of the string "KimballMethodSCD" with "DimensionMergeSCD".
4. Save and close the DTSX file.

Apologies for requiring the manual steps. It was a requirement of following the Kimball Group's request to remove their name from the component.

From v1.4 and earlier to v1.5

Uninstall previous versions of the component.
Follow the v1.5 Installation Instructions to install v1.5 of the component.
For each DTSX package that uses the component, perform the following steps:
1. Back up the package.
2. Open the DTSX file in a text editor, such as Notepad.
3. Search and replace occurrences of the string "SSIS.KimballMethodSCD, KimballMethodSCD, Version=1.0.0.0" to "SSIS.KimballMethodSCD, KimballMethodSCD100, Version=1.0.0.0"
4. Save and close the DTSX file.
5. Open your project in BIDS to complete the upgrade.

Apologies for the manual nature of this upgrade process (the editing in Notepad) - it was required to facilitate the future inclusion of feature 2389: Support 2005 and 2008 Side-by-Side (Same PC) Installation.

Updated Wiki: Upgrading Instructions

toddmcdermid — Fri, 08 Apr 2011 16:30:27 GMT

Upgrading Instructions

From v1.5 to v1.6

Download and install the appropriate release for your system.
For each DTSX package that uses the component, perform the following manual steps:
1. Back up each package
2. Open the DTSX file in a text editor, such as Notepad.
3. Search and replace occurrences of the string "KimballMethodSCD" with "DimensionMergeSCD".
4. Save and close the DTSX file.

Apologies for requiring the manual steps. It was a requirement of following the Kimball Group's request to remove their name from the component.

From v1.4 and earlier to v1.5

Uninstall previous versions of the component.
Follow the v1.5 Installation Instructions to install v1.5 of the component.
For each DTSX package that uses the component, perform the following steps:
1. Back up the package.
2. Open the DTSX file in a text editor, such as Notepad.
3. Search and replace occurrences of the string "SSIS.KimballMethodSCD, KimballMethodSCD, Version=1.0.0.0" to "SSIS.KimballMethodSCD, KimballMethodSCD100, Version=1.0.0.0"
4. Save and close the DTSX file.
5. Open your project in BIDS to complete the upgrade.

Updated Wiki: Home

toddmcdermid — Wed, 09 Mar 2011 17:07:24 GMT

Project Description

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

This component provides an improved design-time, maintenance, and execution experience for processing slowly changing dimensions. It provides more options to support complex dimension table processing while executing over 100x faster than the Wizard.

New walkthrough videos of the upcoming v1.6 release are now available on Youtube, and in full-size/full-quality on my SkyDrive:

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with the Dimension Merge SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.

Updated Wiki: Home

toddmcdermid — Wed, 23 Feb 2011 19:33:36 GMT

Project Description

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

This component provides an improved design-time, maintenance, and execution experience for processing slowly changing dimensions. It provides more options to support complex dimension table processing while executing over 100x faster than the Wizard.

New walkthrough videos of the upcoming v1.6 release are now available on Youtube, and in full-size/full-quality on my SkyDrive:

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with this SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.

Updated Wiki: Home

toddmcdermid — Thu, 30 Dec 2010 23:36:35 GMT

Project Description

A custom Data Flow component for SQL Server Integration Services (SSIS) that replaces the standard SCD Wizard with a superior experience, from the configuration UI to runtime performance. Performs 100x faster than the standard component, and edits are non-destructive.

(Not associated or endorsed by Ralph Kimball or the Kimball Group - but read their books and go take their courses, they're excellent!)

The Slowly Changing Dimension Wizard in SQL Server Integration Services is a very useful tool for setting up and handling slowly changing dimension processing, which is a very common requirement of Data Warehousing. The SCD Wizard is easy to configure and works as advertised.
However, there are some significant issues with it that have been experienced by many:

It can be very slow.
Re-running the Wizard destroys parts of the data flow.
It can’t be used against all data providers.
It can be hard to troubleshoot why it makes the decisions it does. Click here for more details on these and other disadvantages of the SCD Wizard

This component provides an improved design-time, maintenance, and execution experience for processing slowly changing dimensions. It provides more options to support complex dimension table processing while executing over 100x faster than the Wizard.

New walkthrough videos of the upcoming v1.6 release are now available on Youtube, and in full-size/full-quality on my SkyDrive:

Part One	Introducing the sample data	On YouTube	Download
Part Two	Introducing the sample SSIS package	On YouTube	Download
Part Three	Loading the dimension table with the SCD Wizard	On YouTube	Download
Part Four	Loading the dimension table with other SSIS components (Lookup/Conditional Split)	On YouTube	Download
Part Five	Loading the dimension table with the T-SQL MERGE command	On YouTube	Download
Part Six	Loading the dimension table with the Kimball Method SCD component	On YouTube	Download

You can download the Sample Data and Package used in the videos.

What you should do next:

If you have any questions, review the FAQ, then post your question or problem in the discussions or send a tweet to @Todd_McDermid.