HTTP API (Osmos Calls Your API)
There are two modes of operation for this connector. Either Osmos will call your API, or Osmos can generate an API for you to call. Put another way, either you have an API endpoint created, or you want Osmos to create an API endpoint for you. These two modes will be referred to as Osmos Calls Your API and Call an Osmos API respectively. This page is for the former. To see documentation on the latter, see HTTP API (Call an Osmos API).
This Connector sends an HTTP request to a specified HTTP API and treats the returned data from that API as source data. This source data is then sent through an Osmos Pipeline to a Destination Connector.
This is not a Postman or
curl replacement -- the scope of HTTP requests that Osmos will create are limited. Generalized authentication handshakes, forwarding data from a previous API request to the next one, and non-HTTP (TCP, FTP, Telnet) are not supported by this connector.For example, let's say there's an API out there,
www.example.com/weather?city=seattle. Let's say that issuing a GET request to this endpoint returns an array of temperature measurements -- one measurement for every hour of the day. Something like this:[75, 75, 75, 74, 72, 72,
70, 69, 70, 71, 75, 74,
72, 72, 70, 69, 68, 67,
67, 66, 65, 65, 66, 65]
Issuing an HTTP
GET request from an Osmos source would result in 24 records, one for each of the above numbers, being fed into the pipeline. You may also choose to have these 24 numbers treated as one record with 24 fields in it. Information needed:
- An HTTP API that you would like to retrieve data from and send through an Osmos Pipeline.
Step 1: After selecting + New Connector, under the System prompt, click HTTP API
Step 2: Enter a Connector Name.
Step 3: Select Source Connector.
Step 4: Select Osmos Calls Your API
We will automatically detect the schema of the data returned by the specified HTTP API.
Specify the URL, including the port number if any, of the API that Osmos should be sending the request to. For example, this could be http://example.com/api/v2:8824
Select whether the API should return one record at a time, or an array of records. This setting turns on/off unpacking a top level array into a series of records.
Specify the headers, if any, that Osmos should be using to request data from this API. Headers are often used for authentication, authorization, or declarations of what type of content you're sending. Headers have both a Name and a Value. For example, an authorization header might look like:
Name: Authorization
Value: qaX1SrppzSe1Y1qOOc08
Osmos supports record level deduplication when pulling data from an API. In this mode, Osmos will store hashes of each record pulled from the source API, and will only forward new records.
Last modified 3mo ago