HTTP API (Osmos Calls Your API)

Overview

There are two modes of operation for this connector. Either Osmos will call your API, or Osmos can generate an API for you to call. Put another way, either you have an API endpoint created, or you want Osmos to create an API endpoint for you. These two modes will be referred to as Osmos Calls Your API and Call an Osmos API respectively. This page is for the former. To see documentation on the latter, see HTTP API (Call an Osmos API).

This Destination Connector will send data to a specified third party API.

Types of APIs Supported

This is not a Postman or curl replacement -- the scope of HTTP requests that Osmos will create are limited. Generalized authentication handshakes, forwarding data from a previous API request to the next one, and non-HTTP (TCP, FTP, Telnet) are not supported by this connector.

Example Use Cases

For example, let's say you have an API which is used to send notifications to your users. You'd like to notify all users every morning of all new records in a specific Google Sheet. You've created your Google Sheets source connector and now you're working on the destination connector. You have an authentication token that you use to prevent anybody from sending notifications to your users.

In this scenario, you would want to create an HTTP API connector. You'll use the POST method, most likely, to send data to this endpoint. In the headers, you'll put the authentication token. As the destination connector, whenever the pipeline it is associated with is run, the connector can either send every individual record as a separate API call; or, it can send the entire pipeline run as a single API request containing an array of records.

Creating an HTTP API Destination Connector

Step 1: After selecting + New Connector, under the System prompt, click HTTP API.

Step 2: Provide a Connector Name.

Step 3: Select Destination Connector.

Step 4: Select Osmos Calls Your API

Step 5: API URL

Specify the URL, including the port number if any, of the API that Osmos should be sending data to. For example, this could be http://example.com/api/v2:8824

Step 6: HTTP Method

Select the HTTP Method that Osmos should use when sending the data to this API.

Step 7: HTTP Headers

Specify the headers, if any, that Osmos should be using to send data to this API. Headers are often used for authentication, authorization, or declarations of what type of content you're sending. Headers have both a Name and a Value. For example, an authorization header might look like:

Name: Authorization

Value: qaX1SrppzSe1Y1qOOc08

Step 8: Write Options

Choose how data are sent to your external API. If "Batched as an array of records" is selected, we send up to 250,000 records in a json array as the request body. If "Send one record at a time" is selected, we send each record as JSON obj as its own request. If you select to call your api in batches, you can choose how we respond to a failed request to the API. By default, we mark all those records as failed in our product. If you check "Resend failed batches in smaller chunks", we will divvy up the batch into increasingly smaller batches and resend them to the API until the chunk either succeeds or it fails when the chunk is one message in size.

Step 9: Destination Schema

Design the output schema via two options, either import the schema or build it within Osmos.

Option 1: Schema Import

Upload or drag & drop the schema file.

Option 2: Building the Schema

Use the schema designer to build the output schema for this Destination Connector.

1. Click Add Field for each additional field required in the schema. 2. Select Create Schema once you have built the schema.

Connector Options

The connector can be deleted, edited and duplicated.

Duplication

To save time, the connector can be duplicated. This new connector needs to be named and can be edited, as needed.