What are Osmos Pipelines?
In this section you will learn the basics of how to setup Osmos Pipelines.

Contents

  1. 1.
    What is a Pipeline?
  2. 2.
    Connect to a Source
  3. 3.
    Connect to a Destination
  4. 4.
    Transform and Map your Data
  5. 5.
    Set Rules

What is a Pipeline?

Pipelines allow you to send data from a Source Connector to and Destination Connector. You can also cleanup and restructure the data before it is sent to the Destination Connector using training examples, column mapping, and formulas. Pipelines can be set up on a recurring schedule or manually triggered to run at any time.
Osmos Pipelines Overview

Connect to a Source

The first step in creating a Pipeline is selecting the Source Connector. This Connector provides access to the source data.
Check out the Source Connector page for more information.

Connect to a Destination

The second step in creating a Pipeline is selecting the Destination Connector. This Connector provides access to the system to where you want to send the source data after it's transformed.
Check out the Destination Connector page for more information.

Transform & Map Your Data

After selecting the Source and Destination Connectors for a Pipeline, the next step is to transform the source data into a format that aligns with the Destination Connector's schema. You can clean up and restructure the data using training examples, column mapping, and formulas.
Check out the Transformation step for more information.

Set Rules

After completing the transformation step, you will be taken to the Pipeline Rules page where you can set the rules. Currently, we only support one business rule, which allows you to set a schedule for the Pipeline, but more rules (such as PII detection, language translation and more) will be available soon.
Check out the Rules step for more information.
Continue to the next page to get started with creating a Pipeline!
Last modified 1mo ago