SCD 1.5 Hanging under 2 conditions

May 5, 2010 at 10:27 PM

I turned to KSCD 3 days ago in hope of faster SCD transforms, but i've ran into snags using this component.  The data flow task locks up, seemingly not reading in any rows from either Source DB table, or Existing Dimension table.  I chose to presort the tables in the two DBs via views, and set OLE DB source tasks to IsSorted = true, and set SortKeyPosition accordingly (to match BK's).  My surrogate key is generated by the existing dimension / destination DB, and therefore suppressing surrogate key warnings.

The task runs fine when the existing dimension table is empty and source table is small (e.g. 200 rows).

Two scenarios when the IDE locks up are:

- Existing Dimension table is not empty (having successfully ran KSCD once prior / populating table)

- Large amount of rows in source table (6.7M rows)

Any help would be appreciated, I'd really like to get this component running with it touted greater speeds, thanks, dave

 

 

May 6, 2010 at 6:18 PM
I have locked up KSCD 1.5 on a dimension of 1.1 million rows with a 7 part business key. Acc/to Todd performance is affected by the overall size of the business key. And, perhaps the hanging scenario is related to business key size also. In my case I was able to change the larger business key components into numeric fields thereby shrinking the business key size. After that change, I no longer experienced the hang. If you also have a large composite business key, you may want to consider trying alternatives to shrink the keys possibly even using a hash routine to create a single, smaller business key. Tim