Step 3. Map & Transform data
After selecting the Source and Destination Connectors for a Pipeline the next step is to transform the source data into a format that aligns with the Destination Connector's schema. You can clean up and restructure the data using training examples, column mapping, and formulas. If you want to watch an product expert demonstrate this step, click on the See it in action link above the destination table.
Data Transformation Overview

Source Table

The source table contains a sample of the source data from the Source Connector. Each column represents a source field, and the name of each field is shown in the column header. The remaining rows are all data records.
If you have more than 1,000 records of source data, only the first 1,000 records will be displayed in the source table. All source data will be sent to the Destination Connector when you run the Pipeline, and records with errors will be held within Osmos for you to review and correct.

Destination Table

The destination table is empty to start, but the field names and field types for the Destination Connector will be displayed in the column headers. To populate the destination table with the source data, you will need to provide training examples, column mappings, or formulas for each field.

Getting Started

Step 1: You will be prompted to start in the first row of the first column, but if preferred, you may start with any cell by clicking on that cell.
Step 2: Decide which of the following options is best for the specific column you want to populate:
  1. 1.
    Use column mapping to directly map data from one source field to one destination field,
  2. 2.
    Use formulas to map and transform data from one or many source fields to one destination field, or
  3. 3.
    Provide training examples to create a no-code program to map and transform data from one or many source fields to one destination field.
Note: Ensure that the output data in the destination columns match the field type noted above each field name.
Step 3: Once you have mapped and transformed the source data to all of the required destination fields (i.e. non-nullable fields), click Continue to proceed to the next step and set the rules for the Pipeline.
Last modified 1mo ago