Here's my scenario -
Let's say there are different fruit_baskets that stores different number of apples in each one of them, and these fruit_baskets also have an expiry_date. All these information is stored in one of the tables in postgres db. The same information is also converted into orc (AFS) files. Now my use case is to first identify all the fruit_baskets that has expiry_date > CURRENT DATE and then once filtered, I need to compare the count of apples from each of the filtered fruit_baskets from postgres db against the same fruit_baskets in orc file. Mainly I want to make sure that the db data to orc file conversion is correct by comparing the count.
Could anyone please guide how I can achieve this?
Thanks in Advance!
I have been doing this manually, but I need a way to automate this and I'm not too much familiar with automation. Kindly guide.