Production Release

Topics: Questions About Best Practices
Mar 22, 2011 at 3:58 AM
Edited Mar 22, 2011 at 3:58 AM

Hi All,

We are looking to implement the Kimball SCD into a production data mart environment. Would be great to get some feedback about reliability, scalability  and/or any other challenges using the Kimball SCD? The other options of the standard SSIS SCD or fully custom coding don't appear too desirable. Any info is appreciated. Thanks in advance!!

Mar 22, 2011 at 10:05 AM

Hi

I have implemented this in a large data warehouse project and we have mixed feelings about it.  Under the right circumstances the KSCD is perfect as it speeds up development, performs well and is a vast improvement on the SSIS component. 

However, we have had issues are when running more than one KSCD in parrallel, it errors randomly.  And more alarmingly we seem to have situations where we lose records, it has to be said that this only occurs when processing large and wide datasets - approx 25 million records - and we never seem to be able to reproduce the problem consistently.  We are worried about it's ability to multithread correctly, the errors when running in parrallel indicate some threading issue.  We run our SSIS packages on virtualised 64bit servers.

Having said the above, it is great for small dimension loads running serially.  The developer has invested a lot of time into this and it's free - a lot of credit must go to him for his efforts.

Thanks