BigQuery
Last updated
Was this helpful?
Last updated
Was this helpful?
You can create a Google BigQuery Destination Connector to write to a Google BigQuery table.
To set up this Connector using a GCP Service Account Key, you will need a GCP Service Account that has access to the project(s) where the resources reside. To learn more about creating and managing service accounts within GCP, visit: https://cloud.google.com/iam/docs/creating-managing-service-accounts.
To set up this Destination Connector using Single Sign-On, your Google account will need access to the BigQuery table(s) you are connecting to.
The schema for this Destination Connector is defined by the BigQuery table schema.
Required information:
Single Sign-On credentials OR Service Account Key with the proper privileges
Project ID
Dataset
Table
Step 1: After selecting + New Connector, under the system prompt, click BigQuery.
Step 2: Provide a Connector Name.
Step 3: Select Destination Connector.
There are two options for authentication: Single Sign-On and Service Account. You can choose to authenticate through your Google account using single sign-on, or by providing a Google Cloud Platform service account JSON key.
Step 4a: To authenticate using Single Sign-On, you will be prompted to sign into your Google Account after entering the required information and clicking Grant Access
Step 4b: To authenticate using Service Account, you will be prompted to enter the Service Account Key. Provide the Service Account JSON key for the account you wish to connect to.
Service accounts associated with a BigQuery Destination connector will need Data Editor (read/write) privileges in order to successfully establish a connection.
Creating a Service Account Key in the Google Cloud Console
To create a Service Account JSON key, first navigate to the Service Accounts page in the Google Cloud Console.
Click the project dropdown in the top navigation bar to view all of your projects, choose the project you want to create a service account key for, and then click Open.
Find the row of the service account that you want to create a key for. In that row, click the More button, and then click Create key.
Select the JSON Key type and click Create
To find the Project ID, first select the project dropdown in the top navigation bar to view all of your projects.
2. Find the corresponding ID for the project, copy the Project ID and paste it into the Project ID field.
Provide the name of the dataset that contains the table you want to connect to.
To find the dataset information, first Navigate to the BigQuery page in the Cloud Console.
In the navigation menu, click SQL Workspace
In the Explorer panel, expand the project name to see the datasets in that project. Copy the dataset name and paste it into the Dataset field.
For more information about creating a new Dataset, visit: https://cloud.google.com/bigquery/docs/datasets
Step 1: Provide the name of the table you want to connect to. To find the table name, expand the dataset from the step above to see all of the associated tables, and copy the table name and paste it into the Table field.
We support the use of Validation Webhooks to prevent bad data from being written to your systems, adding another layer of protection to the built-in validations that Osmos provides. The Webhook URL can be posted here.
For more information on Validation Webhook configuration, see Server Side Validation Webhooks
The connector can be deleted, edited and duplicated.
Duplication
To save time, the connector can be duplicated. This new connector needs to be named and can be edited, as needed.
example-project-365122
