Talend Read Dynamic File Structure

The data coming in text files are unpredictable. One day, we might get ten columns, the next day 12 columns, the other day columns in a completely different order, etc. If our source file has a dynamic structure or Muti structure, we can use the Talend tFileInputMSDelimited component to handle the same.

To demonstrate the Talend tFileInputMSDelimited to read dynamic file structure, we use the below file.

Talend Read Dynamic File Structure 1

From the above image, you can see, Canada has four columns, France has three columns, and the order of the columns of all the countries are different.

Talend read dynamic file structure Example

First, drag and drop the Talend tFileInputMSDelimited to read the multi-structure or dynamic file structure.

Talend Read Dynamic File Structure 2

Click on the Multi Schema Editor button.

Talend Read Dynamic File Structure 3

It is the place where we have to fetch the structures based on the filed separator. Most of the components are the same as the tFileInputDelimited.

Talend Read Dynamic File Structure 4

Please click on the Browse button and select the Dynamic Source text file from the file system.

Talend Read Dynamic File Structure 5

Next, change the Field Separator as per your need, and click the Preview button to see the data preview.

Talend Read Dynamic File Structure of 6

Please select or checkmark the Common column or say group by column to fetch the codes. By default, it will select the column; otherwise, you can select the required column. Next, click the Fetch Codes button.

Talend Read Multi Structure File 7

If there are multiple separators in the text file to separate the columns, then checkmark Use multiple separators option and place them in the below text box.

Talend Read Multi Structure File 8

Now you can see all the Connections under the Output columns.

Talend Read Multi Structure File 9

Once you right-click on the Talend tFileInputMSDelimited and select the Rows option, you can see all the outputs. For the Talend Read Dynamic File Structure example, we assigned each output to tLogRow with table preview.

Talend Read Multi Structure File 10

Please run the Talend Read Multi Structure File to see the output.

Talend Read Multi Structure File 11

You can also change the column names based on the requirements.