🔠Datasets

In this section you will learn the basics about Osmos Datasets and Dataset Tables.

What are Osmos Datasets?

Datasets keep information organized, housing tables and their data. Datasets offers a clean logical entity to build and manage your data models. A Dataset serves as the container for all your tables where the actual data resides. Each of these tables can have Primary keys and reference other tables using Foreign keys, thereby helping you organize the elements of data and standardize how they relate to one another by enforcing referential integrity between your data elements.

You can create new datasets, duplicate from another project, and manage existing datasets.

Why use Osmos Datasets?

Datasets gives you the ability to achieve high fidelity Data Management in one place, without the need for SQL skills.

Let's say you are working on a migration project. As a part of the migration project, you need to import 10 types of data elements (tables) from your customer. These data elements are interconnected and reference one another using foreign keys. When migrating the data, you need to ensure that you are keeping the data integrity intact and perform Lookups and Joins at the time of data migration.

The cleanest way to accomplish such a project would be using Datasets. You would end up creating a migration Dataset, housing all the tables with their Foreign Key relationships. You would then use Uploaders or Pipelines to import data into these tables, using Lookups to ensure referential integrity. Once the data is loaded in Datasets, you would then use Pipelines to extract the clean data from Dataset into your Database and APIs.

A few features of Osmos Datasets

There are no limitations to the number of datasets or the tables you can create inside a dataset.
There is no cost to storing data inside Datasets. You only pay for the in-transit data processing costs to and from Datasets, per the service tier you are in.
You can write to and read from Datasets. You can use Uploaders or Pipelines to perform read and write operations on Datasets.
Depending upon your Osmos plan, you can leverage datasets to do lookups, joins, and aggregations.

PreviousParser Webhook for file based connectors NextOsmos Datasets

Last updated 5 months ago

What are Osmos Datasets?

You can create new datasets, duplicate from another project, and manage existing datasets.

Why use Osmos Datasets?

Datasets gives you the ability to achieve high fidelity Data Management in one place, without the need for SQL skills.

A few features of Osmos Datasets

There are no limitations to the number of datasets or the tables you can create inside a dataset.

There is no cost to storing data inside Datasets. You only pay for the in-transit data processing costs to and from Datasets, per the service tier you are in.

You can write to and read from Datasets. You can use Uploaders or Pipelines to perform read and write operations on Datasets.

Depending upon your Osmos plan, you can leverage datasets to do lookups, joins, and aggregations.