Projects
An Osmos Project is a collection of datasets, pipelines, uploaders, and connectors that are organized together to manage and automate data ingestion, preprocessing, transformation, and movement.
Last updated
Was this helpful?
An Osmos Project is a collection of datasets, pipelines, uploaders, and connectors that are organized together to manage and automate data ingestion, preprocessing, transformation, and movement.
Last updated
Was this helpful?
Here are some key points about Osmos Projects:
Collections: Projects act as containers for organizing your data resources. This includes datasets (tables), pipelines (ETL processes), uploaders (tools for uploading data files), and connectors (connections to source or destination systems).
Multiple Projects: Customers can have more than one project. Each project can be tailored to different use cases, teams, or departments within an organization.
Datasets: Within a project, datasets are managed databases that store customer data. Each dataset can contain multiple tables and can be used for various purposes such as a system of record, landing zone for incoming data, or source for clean data to be sent to other systems. Please refer to for more information.
Pipelines: Pipelines in a project automate the ETL (Extract, Transform, Load) process. They define mappings between source systems and destination systems, including any transformations or data cleaning that needs to be done. Please refer to for more information.
Uploaders: Projects include uploaders that allow users to upload files (CSV, Excel, etc.) into the system. Uploaders can be used directly within the Osmos UI or can be embedded into third-party websites to enable self-service data uploads. Please refer to for more information.
Connectors: Connectors within a project facilitate integration with other systems. They can act as sources from which data is pulled or as destinations where data is pushed. Connectors can be reused across multiple pipelines and uploaders within the project. Please refer to and for more information
Management: Projects provide a way to manage and monitor all associated resources, enabling users to see how data flows through their pipelines, validate data integrity, and troubleshoot any issues that may arise.
Creation and Support: Currently, new projects must be created by Osmos support upon request, but self-service project creation is a feature that is coming soon.
By organizing resources within projects, Osmos ensures that users can effectively manage complex data ingestion workflows, maintain data integrity, and meet their organization’s data processing needs comprehensively.
If you have any more specific questions about projects or need help setting up your projects in Osmos, please contact support@osmos.io.