LogoLogo
Back to OsmosDeveloper DocsOsmos BlogWhat's New
  • Welcome to Osmos
  • 👋Getting Started with Osmos
    • Terminology
  • 🎉What's New
  • 🧩Osmos API Reference
  • ⌨️Osmos Chat
  • 👩‍💻Developer Docs
    • Manage API Keys
    • Embedding an Osmos Uploader
    • Embedding Uploader Jobs Table
    • Turning on Advanced Mode Uploader
    • Customizing Uploader Styling
    • Passing Parameterized Fields
    • Configuring Uploader's "Recall" functionality
    • Optional Uploader Settings
    • Uploader Submission Callback
    • Configuring AutoClean for your Uploader
    • Uploader Client-Side Validation
      • Data Validators
      • Checking for Duplicate values in a field
      • Creating Dropdown-Controlled Fields
      • Dynamic Dropdown Options
      • Dropdown Interaction with Validation Functions
    • Validation and Transformation Webhooks
      • OpenAPI Validation Webhook Testing
    • Parser Webhook for file based connectors
  • 🔠Datasets
    • Osmos Datasets
      • Uploading Data to your Table
      • Creating Primary and Foreign keys
      • Osmos Dataset Destination Connector
      • Osmos Dataset Source Connector
      • Dataset Edits
    • Datasets Query Builder
      • Query Builder Metadata
    • Performing Look Ups
      • Performing Joins
        • Types of Joins
  • ⏏️Uploader
    • Creating an Osmos Uploader
      • Testing your Osmos Uploader
    • Uploader Validation Summary
    • Advanced Mode
      • Overview
      • Process
    • Standard Mode
      • Overview
      • AutoClean
      • Process
    • AI AutoMapping
    • Uploaders Page
    • Uploader Details Page
  • 🔀Pipelines
    • Step 1. Select the Source
    • Step 2. Select a Destination
    • Step 3. Map & Transform Data
    • Step 4. Schedule the Pipeline
    • Step 5. Review & Confirm
    • Pipelines Page
    • Pipeline Details Page
  • ⏩Data Transformations
    • AutoMap
    • Column Mapping & Data Cleanup Panel
    • QuickFixes
    • AI Value Mapping
    • AI AutoClean
    • Lookups
      • Performing Lookups
    • SmartFill
    • Formulas
      • Date & Time Formulas
        • DateTime Format Specifiers
        • Timezone specifiers
      • Math Formulas and Operators
      • Logical Formulas & Operators
        • True & False Casting
      • Text Formulas
      • Other Formulas
    • Deduplication
  • ↘️Source Connectors
    • Amazon S3
    • Azure Blob Storage
    • BigQuery
    • Email
    • FTP
    • Google Cloud Storage (GCS)
    • Google Drive
    • Google Sheets
    • HTTP API (Call an Osmos API)
    • HTTP API (Osmos Calls Your API)
    • Osmos Dataset
    • Snowflake
    • Accessing Sources behind firewall
  • ↖️Destination Connectors
    • Amazon S3
    • BigQuery
    • FTP
    • Google Cloud Storage (GCS)
    • Google Drive
    • Google Sheets
    • HTTP API (Call an Osmos API)
    • HTTP API (Osmos Calls Your API)
      • Passing Dynamic Tokens in the API Header
    • MySQL
    • Osmos Dataset
    • PostgreSQL
    • Snowflake
    • Accessing Destinations behind firewall
  • 🗂️Projects
  • ⚙️Administration
    • Email Notifications
  • 🔒Security
  • 📞Support
  • Back to Osmos.io
Powered by GitBook
On this page
  • Overview
  • Metadata Fields
  • Field Details

Was this helpful?

  1. Datasets
  2. Datasets Query Builder

Query Builder Metadata

PreviousDatasets Query BuilderNextPerforming Look Ups

Last updated 10 months ago

Was this helpful?

Overview

Query Builder includes a set of metadata that are available to add as columns to any Dataset Table. They are added when you select columns in the Query Builder.

Metadata Fields

Nine metadata fields are available. These include the Osmos internal ID, date create, date modified, user ID, dataflow ID, job ID, uploader ID, raw source data JSON and write index.

Field Details

  • Osmos Internal ID: A unique identifier is generated for each row in the table. Randomly generated the first time the row is inserted. Will not change in the case of upserts or edits.

  • Date Created: A timestamp of when the row was first inserted into the table. Stays the same through the life of that row.

  • Date Modified: A timestamp of the last time that any data in the row was edited, upserted, or changed in any way. Will be updated in the case of upserts even if no data changes in the row.

  • User ID: The internal ID of the user that initiated the insert/upload/job/edit/etc. updated the row to have its most recent value, or that created the row.

  • Dataflow ID: The ID of the dataflow which updated or inserted the row. If data is uploaded via Osmos Uploader, this value will be equivalent to the Uploader ID. For pipelines, this will represent Pipeline ID. Will be updated in the case of upserts even if no data changes in the row.

  • Job ID: The ID of the job which created or updated the row.

  • Uploader ID: The ID of the Uploader that created or updated the row.

  • Raw Source Data JSON: A JSON object containing the full data from the source side that was used to map data to this output, if this row was created or updated via an uploader or pipeline. The JSON is returned as a string.

  • Write Index: Every time data is inserted, upserted, or edited in a dataset table, a unique monotonically increasing write index is associated with it. For each row, the write index will be set to the write index of the insert/upsert/update operation that sets the most recent version of data in this row. This will be updated in the case of upserts even if no data in the row changes.

🔠