Integrating / Transforming data from different / disparate sources without storing it

Question

I have a usecase. I want to Integrate / Transform data from different / disparate sources without storing it. Data sources are database(oracle,db2,etc), Webservice(Rest/Soap), Flat files(CSV, XML, JSON), MQ dumps, mainframe systems. I want to pull data from these sources and do some kind of intelligent transformation and integration and provide it our customers. It looks like typical ETL scenario, but my situation is different. I am not allowed to store the data given by the desperate sources, that means, for simple example, i pull data from oracle, soap and a rest, and do all my intelligent transformations and integrations on the fly.

I browsed through google and technical stuffs but could not get convincing solution to my problem.

If you guys can help me giving some valuable insight on this problem and give suggestion and probable approaches to it.

Note: Data size from these sources can sometime be really huge.

Thanks in Advance

Answer 1

Talend Open Studio y a great solution as well, I'm using it and it's great and easy to make the ETL workflow.

https://www.talend.com/products/data-integration/data-integration-manuals-release-notes/

You can see a lot of help videos: https://www.youtube.com/results?search_query=talend+studio

Answer 2

Take a look at htto://teiid.org

Thst is exactly what it does, and it is Open Source.