Osmos Documentation
Search…
Getting Started
👋
Welcome to Osmos!
Setting up your account
Terminology
Osmos Uploader
What is an Osmos Uploader?
Creating an Uploader
Embedding the Uploader in your Web App
Customizing the styling
Passing Parameterized Fields
Data Validators
Creating Dropdown-Controlled Fields
Dynamic Dropdown Options
Dropdown interaction with Validation functions
Osmos Pipelines
What are Osmos Pipelines?
Step 1. Select the Source
Step 2. Select a Destination
Step 3. Map & Transform data
Step 4. Schedule the pipeline
Step 5. Review & Confirm
Mapping & Transforming Data
Map & Transform data...
... Using Column Mapping
... Using Formulas
... Using Training Examples
Source Connectors
Source Connector Overview
Amazon S3
BigQuery
FTP
Google Drive
Snowflake
Google Sheets
HTTP Endpoint
Destination Connectors
Destination Connector Overview
Amazon S3
HTTP API
BigQuery
FTP
Google Drive
Google Sheets
MySQL
PostgreSQL
Snowflake
Security
Security, Privacy & Compliance
Support
Contact Us!
Back to Osmos.io
Powered By GitBook
What are Osmos Pipelines?
In this section you will learn the basics of how to setup Osmos Pipelines.

Contents

  1. 1.
    What is a Pipeline?
  2. 2.
    Connect to a Source
  3. 3.
    Connect to a Destination
  4. 4.
    Transform and Map your Data
  5. 5.
    Set Rules

What is a Pipeline?

Pipelines allow you to send data from a Source Connector to and Destination Connector. You can also cleanup and restructure the data before it is sent to the Destination Connector using training examples, column mapping, and formulas. Pipelines can be set up on a recurring schedule or manually triggered to run at any time.
Osmos Pipelines Overview

Connect to a Source

The first step in creating a Pipeline is selecting the Source Connector. This Connector provides access to the source data.
Check out the Source Connector page for more information.

Connect to a Destination

The second step in creating a Pipeline is selecting the Destination Connector. This Connector provides access to the system to where you want to send the source data after it's transformed.
Check out the Destination Connector page for more information.

Transform & Map Your Data

After selecting the Source and Destination Connectors for a Pipeline, the next step is to transform the source data into a format that aligns with the Destination Connector's schema. You can clean up and restructure the data using training examples, column mapping, and formulas.
Check out the Transformation step for more information.

Set Rules

After completing the transformation step, you will be taken to the Pipeline Rules page where you can set the rules. Currently, we only support one business rule, which allows you to set a schedule for the Pipeline, but more rules (such as PII detection, language translation and more) will be available soon.
Check out the Rules step for more information.
​
Continue to the next page to get started with creating a Pipeline!
Osmos Uploader - Previous
Dropdown interaction with Validation functions
Next - Osmos Pipelines
Step 1. Select the Source
Last modified 1mo ago
Copy link
Contents
Contents
What is a Pipeline?
Connect to a Source
Connect to a Destination
Transform & Map Your Data
Set Rules