We have just begun to test your component with a number of large data syncronisation tasks we currently perform via various combersome (and slow) methods. We have seen how good the component is, as we are merging 26 source databases into a consolidated database with daily data changes.
 
We are experiencing a number of issues, these manifest themselves both as an apparent package hang or as an internal error. We have tried to determine if this is volume of data or specific data issues but have been unable to establish any consistency.
 
We have had this happen with volumes as low as 150 records and as high as 80,000 records; in some situations we have found volumes that worked on a previous run fail on subsequent runs and vice versa.
 
We have enable debugging which shows that the application hang is actually in infinate loop (issue number 5400). Also when we get an internal error this appears to happen prior to where the infinate loop occurs and would be similar to issue 9262. As we appear to experience both previous issues, and one is marked as fixed I have opted to add a new issue.
 
The source and destination database servers have been checked and show no issues. The issue happens both within Visual Studio IDE and from the command line using dtexec.
 
Our development environment we have is;
 
Source server
Windows 2003 64bit
SQL Server 2005 Std 64bit (with SP4)
 
Destination server
Windows 2008 R2 64 bit
SQL Server 2008 R2 Developer 64bit
 
Development machine running the package
Windows 7 Professional 64bit (with SP1)
SQL Server 2008 R2 Developer 64 bit
Visual Studio 2008 (with SP1)
 
We are using the latest 1.6 version.
 
This issue feels like a thread management in the component, but even after disabling threading in the component options we still get this error; although lookng at the debug output even with threading disabled it appears that 2 threads are working together to process and mange the work queue.
 
I am reviewing other similar SSIS components, but so far prefer yours because of the extra output information we can log around auditing and statistics data and handling invalid input data, all of which are very important to us.
 
For information:
Debug output for infinate loop
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: Thread 'ProcessKey (12)' (in ProcessKey): Sleeping - no work.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: ED 0/21792(comp), SS 766/29556(comp) - 0 matched, 0 work units ready.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: 678 II, 21792 UC, 6998 NW, 0 DL, 0 S1, 0 S2E, 0 S2N
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: Thread 'MatchKeys (11)' (in MatchKeys): Sleeping - no work.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: ED 0/21792(comp), SS 766/29556(comp) - 0 matched, 0 work units ready.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: 678 II, 21792 UC, 6998 NW, 0 DL, 0 S1, 0 S2E, 0 S2N
 
Debug output for internal error:
Error: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: Internal error (Error building work units: Error getting next work unit - marking all keys matched: Internal error in RuntimeInputCache retrieving current key list.) in ProcessCache_Thread_MatchKeys.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: ED 0/0(comp), SS 15/200(comp) - 0 matched, 0 work units ready.
Information: 0x0 at Syncronise, Dimension Merge Slowly Changing Dimension: 7 II, 0 UC, 185 NW, 0 DL, 0 S1, 0 S2E, 0 S2N
SSIS package "ProdHdr.dtsx" finished: Canceled.