The source files structure will be changed on daily basis in informatica cloud

Question

Requirement is, The source files structure will be changed on daily basis / dynamically. how we can achieve in Informatica could:

For example,

Let's consider the source is a flat file with different formats like with header, without header, different metadata(today file with 4 columns and tomorrow its 7 different columns and day after tomorrow without header , another day file with count of records in file)

I need to consume all dynamically changed files in one informatica cloud mapping. could you please help me on this.

Answer 1

This is a tricky situation. I know its not a perfect solution but here is my idea-
create a source file structure having maximum number of columns of type text, say 50. Read file, apply filter to cleanup header data etc. Then use router to treat files as per their structure - may be filename can give you a hint what it contains. Once you identify the type of file, treat,convert columns according to their data type and load into correct target.
Mapping would look like Source -> SQ -> EXP -> FIL -> RTR -> TGT1, TGT2

There has to be a pattern to identify the dynamic file structure. HTH...

Answer 2

To summarise my understanding of the problem:

1. You have a random number of file formats
2. You don't know the file formats in advance
3. The files don't contain the necessary information to determine their format.

If this is correct then I don't believe this is a solvable problem in Informatica or in any other tool, coding language, etc. You don't have enough information available to enable you to define the solution.

The only solution is to change your source files. Possibilities include:

• a standard format (or one of a small number of standard formats with information in the file that allows you to programatically determine the format being used)
• a self-documenting file type such as JSON