CosmosDB

Overview

Note: Please contact us at [email protected] to enable this Source Connector.

CosmosDB is a NoSQL document store which stores data in a JSON-like format without a specified schema. This Source Connector allows you to read documents from a single collection and ingest them into Osmos.

This Source Connector can pull from a CosmosDB table using your CosmosDB username and password.

We use the Azure Cosmos DB API for MongoDB. More information can be found here.

Prerequisites

You need to have a CosmosDB instance that will accept connection attempts from the following fixed IP addresses: 35.222.106.198/32, 35.232.180.174/32.

The collection must be non-empty as the first 10 documents in the collection are used to infer the schema.

You must have a connection string provided by the database from which to connect your DB.

Creating a CosmosDB Source Connector

Step 1: After selecting + New Connector, under the System prompt, click CosmosDB.

Step 2: Enter a Connector Name.

Step 3: Select Source Connector. Selecting this allows you to ingest data from CosmosDB.

Step 4: Provide the connection string, but leave out the username and password.

Example: mongodb://host1[:port1][,...hostN[:portN]][/[defaultauthdb][?options]]

For more information about formatting the connection string, visit: https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb/connect-mongodb-account

Step 5: Provide your username.

Step 6: Provide the password associated with the username above.

Step 7: Provide the name of the database that contains the collections you want to read from.

Step 8: Provide the name of the collection that you want to read from.

Tip: If you are trying to access a database that is behind a firewall, please ensure that the following Osmos IP addresses are allowed to access the database. Osmos uses these fixed IP addresses to access your sources and destinations:

35.222.106.198/32

35.232.180.174/32

Tip: The schema for this Source Connector is defined by the first 10 documents in the collection.

Advanced Options

Header Normalization

The source file may have characters at the start or end that includes spaces, tabs, carriage returns and line endings. You can choose to keep all characters from the source or remove all whitespace. Select one of the options:

1. Don't normalize headers. Use headers exactly as they appear in the source: If this option is selected, we will retain all characters from the source file.

2. Remove extra whitespace and other common untypable characters from headers: If this option is selected, we remove all whitespace (spaces, tabs, carriage returns, line endings) at start/end.