SSIS Term Extraction Transformation

In this article We are going to show you, How to use the Exclusion Tab while extracting the Noun Phrases using SSIS Term Extraction Transformation. Before reading this article, Please refer Term Extraction in SSIS article for the definition and the functionality of this Transformation and Extract Noun Phrases Using Term Extraction Transformation in SSIS article to understand How to Extract Noun Phrases from the Source Data.

Below screenshot shows our source data

Let us see the Exclusion List as well

In this example, we are going to display all the Noun Phrases present in the source data except the above mention names.

Configure Exclusion Tab in SSIS Term Extraction Transformation

STEP 1: Open BIDS and Drag and drop the data flow task from the toolbox to control flow and rename it as Exclusion Tab in SSIS Term Extraction Transformation.

Double click on it and it will open the data flow tab.

STEP 2: Drag and drop OLE DB Source, Term Extraction Transformation and OLE DB Destination from toolbox to data flow region

STEP 3: Double click on OLE DB source in the data flow region will open the connection manager settings and provides space to write our SQL statement.

Here we selected the [SSIS Tutorials] Database as our source database and SQL Command we used in the above screenshot is:

USE [SSIS Tutorials]

SELECT [Player Information]

FROM [Term Extraction Transformation Source]

STEP 4: Click on columns tab to verify the columns. In this tab we can uncheck the unwanted columns also.

Drag the OLE DB source output arrow on to the Term Extraction Transformation to perform transformation on the source Data.

STEP 5: Double click on the Term Extraction Transformation will open the Term Extraction Editor to configure it. Within the Term Extraction tab, You simply need to choose the column you want to use for the Term Extraction from the available input columns. We left the output column names to default Term and Score.

Exclusion Tab: Here we want to exclude the country names (We shown in the earlier) from the Noun Phrases. For this reason, Let us configure OLE DB Connection Manager in SSIS and exclusion terms.

From the above screenshot you can observe that, We selected the [Exclusion List in Term Extraction] table from the [SSIS Tutorials] Database. We selected CountryName from this table because, it holds the exclusion terms such as country names.

STEP 6: Advanced tab of the Term Extraction Transformation Editor Dialog box is very important to select Term Type, Source Type and Frequency Threshold. In this example we are extracting Noun Phrases only. So, we selected Noun Phrases as term type and selecting Frequency Threshold as 1. Please refer Extract Nouns Using Term Extraction Transformation in SSIS article to understand, How to Extract Nouns from the Source Data.

From the below screenshot you can see, there is a warning symbol on the Term Extraction Transformation. And it is telling that error output is not connected. You can remove the warning symbol by configuring the error output of Term Extraction Transformation. So double-click on the Configure Error Output button will open new window to configure the error output.

The default configuration of a Term Extraction Transformation is to redirect error rows. You can get rid of this warning by connecting the error output, or by changing the default behavior to Ignore Failure or Fail Component. Let’s change to Ignore Failure

Click ok to finish configuring the Term Extraction Transformation.

STEP 7: Now we have to provide Server, database and table details of the destination. So double-click on the OLE DB Destination and provide the required information.

Here we selected [SSIS Tutorials] database as destination data source (localhost as server instance) and [Term Extraction Transformation Exclusion Tab] table as our destination table

STEP 8: Click on Mappings tab to check whether the source columns are exactly mapped to the destination columns. If not, Please assign them to appropriate destination column