The data coming in text files are unpredictable. One day, we might get ten columns, the next day 12 columns, the other day columns in a completely different order, etc. If our source file has a dynamic structure or Muti structure, we can use the Talend tFileInputMSDelimited component to handle the same.
To demonstrate the Talend tFileInputMSDelimited to read dynamic file structure, we use the below file.
From the above image, you can see, Canada has four columns, France has three columns, and the order of the columns of all the countries are different.
Talend read dynamic file structure Example
First, drag and drop the Talend tFileInputMSDelimited to read the multi-structure or dynamic file structure.
Click on the Multi Schema Editor button.
It is the place where we have to fetch the structures based on the filed separator. Most of the components are the same as the tFileInputDelimited.
Please click on the Browse button and select the Dynamic Source text file from the file system.
Next, change the Field Separator as per your need, and click the Preview button to see the data preview.
Please select or checkmark the Common column or say group by column to fetch the codes. By default, it will select the column; otherwise, you can select the required column. Next, click the Fetch Codes button.
If there are multiple separators in the text file to separate the columns, then checkmark Use multiple separators option and place them in the below text box.
Now you can see all the Connections under the Output columns.
Once you right-click on the Talend tFileInputMSDelimited and select the Rows option, you can see all the outputs. For the Talend Read Dynamic File Structure example, we assigned each output to tLogRow with table preview.
Please run the Talend Read Multi Structure File to see the output.
You can also change the column names based on the requirements.