HTTP API (Osmos Calls Your API)
Last updated
Last updated
There are two modes of operation for this connector. Either Osmos will call your API, or Osmos can generate an API for you to call. Put another way, either you have an API endpoint created, or you want Osmos to create an API endpoint for you. These two modes will be referred to as Osmos Calls Your API and Call an Osmos API respectively. This page is for the former. To see documentation on the latter, see HTTP API (Call an Osmos API).
This Destination Connector will send data to a specified third party API.
This is not a Postman or curl replacement -- the scope of HTTP requests that Osmos will create are limited. Generalized authentication handshakes, forwarding data from a previous API request to the next one, and non-HTTP (TCP, FTP, Telnet) are not supported by this connector.
For example, let's say you have an API which is used to send notifications to your users. You'd like to notify all users every morning of all new records in a specific Google Sheet. You've created your Google Sheets source connector and now you're working on the destination connector. You have an authentication token that you use to prevent anybody from sending notifications to your users.
In this scenario, you would want to create an HTTP API connector. You'll use the POST method, most likely, to send data to this endpoint. In the headers, you'll put the authentication token. As the destination connector, whenever the pipeline it is associated with is run, the connector can either send every individual record as a separate API call; or, it can send the entire pipeline run as a single API request containing an array of records.
Step 1: After selecting + New Connector, under the System prompt, click HTTP API.
Step 2: Provide a Connector Name.
Step 3: Select Destination Connector.
Step 4: Select Osmos Calls Your API
Specify the URL, including the port number if any, of the API that Osmos should be sending data to. For example, this could be http://example.com/api/v2:8824
Select the HTTP Method that Osmos should use when sending the data to this API.
Specify the headers, if any, that Osmos should be using to send data to this API. Headers are often used for authentication, authorization, or declarations of what type of content you're sending. Headers have both a Name and a Value. For example, an authorization header might look like:
Name: Authorization
Value: qaX1SrppzSe1Y1qOOc08
Choose how data are sent to your external API. If "Batched as an array of records" is selected, we send up to 250,000 records in a json array as the request body. If "Send one record at a time" is selected, we send each record as JSON obj as its own request. If you select to call your api in batches, you can choose how we respond to a failed request to the API. By default, we mark all those records as failed in our product. If you check "Resend failed batches in smaller chunks", we will divvy up the batch into increasingly smaller batches and resend them to the API until the chunk either succeeds or it fails when the chunk is one message in size.
Design the output schema via two options, either import the schema or build it within Osmos.
Upload or drag & drop the schema file.
Import a file with the headers along with one row of sample data. This data is used only in schema creation.
Use the schema designer to build the output schema for this Destination Connector.
Parameter
Description
Field Name
Provide a field name for the output fields. These names will be used as the column names for the source data you send to this API.
Type
Define the type of each field. The field types will be used to enforce rules when you send data to this API.
Nullable
Check this box if the field is nullable. If the field is not nullable, you will be required to provide values for this field when sending data to this API.
Delete
Deletes the Field
1. Click Add Field for each additional field required in the schema. 2. Select Create Schema once you have built the schema.
The connector can be deleted, edited and duplicated.
Duplication
To save time, the connector can be duplicated. This new connector needs to be named and can be edited, as needed.
