Nullable columns and output for Invalid Input in v1.6

Topics: Possible Bugs/Missing Features?
Jan 19, 2011 at 9:15 AM

In my ETL I would like to catch invalid rows due to inconsistency in source system and redirect them to a logging table. A typical indicator would be a lookup (before KSCD) failing and returning a NULL for some ID column.

It looks like using Nullable and activating output for Invalid Input should be a way to do this. Still, I get no rows on this output even when rows are containing NULLs for ID columns, and none of the columns in the Row Change Detection are checked as Nullable.

Isn’t this the way KSCD should work or is this functionality simply not implemented?



Feb 22, 2011 at 9:16 PM

Sounds like you're using it as expected.

What happens when you do have NULL values in columns that are configured as not Nullable?  Does the component generate an error?  Does the row get output somewhere?  What output does it go to, and what value is in the column in question?

Mar 3, 2011 at 9:23 AM

Columns configured as nullable behave the same as columns configured as not nullable. There are no errors and the output seems to go to updated SCD1. In my test existing rows have valid IDs while the source is manipulated returning some NULLs.

If column is configured as business key the row is output both as deleted and invalid so the logic seems to be working partly in that case (but my business keys are never NULL).

Apr 8, 2011 at 7:02 PM

I think there was some problems with how I was checking for NULLs.  Can you try v1.6 out to see if it still shows this odd behaviour.  As a side-note, BKs are never allowed to be NULL, regardless of settings on the comparison tab.  I should figure out how I can inform users of that...

Apr 11, 2011 at 5:46 AM

This is v1.6 behaviour (I only use v1.6).