This section explains how to read a Text file in Talend and use the text file data in a Job. The following Talend image shows the data inside the text file that you want to read.
To read a Text file in Talend, you have to use the tFileInputDelimited field or create metadata under the File Delimited folder. And for this, right-click on the File Delimited and select the Create File Delimited option from the context menu. Or use the export item to export File metadata.
Talend Read Text File Example
Once you click on the Create File Delimited option, opens the following New Delimited File window. Please fill the File name, the purpose of this text file, and the brief description of the data inside the text file.
Here, click the Browse button to choose the text file that you want to load into Talend.
Here, we are selecting the EmpInfo.txt File.
Once you selected the text file, you can see the file data under the File Viewer. Use the format option to choose the operating system. Here, we kept it to the default UNIX, which will work mostly!
Within the next page, we have many options.
- Encoding: By default, Talend will select a suitable Encoding. However, you can use the drop-down button to select the one.
- Field Separator: Please choose the field that separates each column in your text file. If the desired separator is not available in the option, select the Custom and use Corresponding Character option to place the separator.
- Row Separator: Please choose the field that separates each row. I mean, a new line for each row = \n, etc.
- Rows To Skip: If there are any rows to skip, you can select the position (Header rows or Footer rows) and then specify the number of rows you want o skip. For instance, if the text file has the Column Names in the header, you have to skip the first row (Header = 1). If you have the file information like address details as the Footer, select the Footer, and skip those rows.
- Skip empty row: As the name suggests, it skips the empty rows in a text file.
- Escape Char settings: Please use these settings to select the Escape characters in the CSV file.
- Limit Of Rows: If your text file has tons of millions of records, and you want to limit them to 50,000, use this Limit option, and specify the number.
Please checkmark the Set heading row as column Names and click Refresh Preview button. We did this because our text file first column has column names. If you notice the Rows to Skip section, it automatically checkmark the Header option and assigned 1 to it.
Next, we changed the Field Separator to Comma because that one separates my columns.
Now, you can see the text file columns under the preview section.
Within the next page, assign the unique name to this File Delimited metadata. Next, check the Data type, data patterns (if any), length, and click the finish button.
Now, you can see the File metadata under the File Delimited folder.
There are two ways to use this text file data in the Talend job. The first one is using File Metadata that we created previously. So, drag and drop the EmpInfo from File Delimited folder into the Job design.
It will open the following Components window. Please select the tFileInputDelimited and click on the OK button.
Now, you can see the File options in the Component tab has filled with the text file metadata information.
The second approach to Read a Text File in Talend
The second option is to drag and drop the Talend tFileInputDelimited and select the File from the local system. Next, click on the Edit Schema, and you can see that it is empty. Here, you have to use the + button to add all the fields or column names, length, data type, precession, and data format.
If the Property Type is built-in, then you have to select the text or CSV file manually. This process is for reading excel file in Talend without creating metadata.
The other approach is to drag and drop the tFileInputDelimited. Next, change the Property type to Repository and select the metadata from Repository. Then change the schema to Repository and select the schema of EmpInfo metadata.
Let me display the text file output in Talend tLogRow. So, drag the tLogRow and select the Table preview for tLogRow.
Let us run the Talend read Text File job.
And you can see the result.
Please refer to load data from text file to Database article to load this text file into a Database.