Bluemix Data Connect does not apply the filter rows for the entire dataset, just for the first 1000 rows

63 Views Asked by At

The operation "filter rows" does not apply for the entire dataset, just for the first 1000 rows, that is the preview. This is weird, once other operations are applied onto the entire dataset with no problem.

Is it a bug or something that just work in the preview mode ?

Tks, Jean

1

There are 1 best solutions below

0
On

Data Connect currently takes a 1000 row sample from the source for the user to perform refine operations on. As you've seen, when performing a filter operation on a suitably large data set, it is possible that the sample will not contain the value you wish to filter on. We are aware of this and you can expect to see the current sampling method expanded in future updates to the service.

Regards,

Wesley - IBM Bluemix Data Connect Engineering