# BigQuery

## Overview

You can create a Google BigQuery Destination Connector to write to a Google BigQuery table.

To set up this Connector using a GCP Service Account Key, you will need a GCP Service Account that has access to the project(s) where the resources reside. To learn more about creating and managing service accounts within GCP, visit: <https://cloud.google.com/iam/docs/creating-managing-service-accounts>.

To set up this Destination Connector using Single Sign-On, your Google account will need access to the BigQuery table(s) you are connecting to.

The schema for this Destination Connector is defined by the BigQuery table schema.

## Prerequisites

Required information:

* Single Sign-On credentials OR Service Account Key with the proper privileges
* Project ID
* Dataset
* Table

## Creating an BigQuery **D**estination Connector

**Step 1:** After selecting **+ New Connector**, under the system prompt, click **BigQuery**.

![](https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FuNo0j1lqqR43gLqZ6EZ1%2FBigQuery.png?alt=media\&token=47c5422b-ca8d-4064-8185-20c4b31e0b0c)

**Step 2:** Provide a **Connector Name.**

**Step 3:** Select **Destination Connector.**

### &#x20;Authentication

There are two options for authentication: **Single Sign-On** and **Service Account.** You can choose to authenticate through your Google account using single sign-on, or by providing a Google Cloud Platform service account JSON key.&#x20;

* **Step 4a**: To authenticate using **Single Sign-On,** you will be prompted to sign into your Google Account after entering the required information and clicking **Grant Access**
* **Step 4b**: To authenticate using **Service Account,** you will be prompted to enter the **Service Account Key.** Provide the Service Account JSON key for the account you wish to connect to.
  * Service accounts associated with a BigQuery Destination connector will need **Data Editor (read/write) privileges** in order to successfully establish a connection.&#x20;

{% hint style="info" %}
**Creating a Service Account Key in the Google Cloud Console**

1. To create a Service Account JSON key, first **navigate to the Service Accounts page** in the Google Cloud Console.
2. **Click the project dropdown** in the top navigation bar to view all of your projects, **choose the project** you want to create a service account key for, and then click **Open**.
3. Find the row of the service account that you want to create a key for. In that row, click the **More** button, and then click **Create key**.
4. Select the **JSON Key type** and click **Create**
   {% endhint %}

### **Step 5: Project ID**

1. To find the Project ID, first **select the project dropdown** in the top navigation bar to view all of your projects.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FbbltwYLRiD1R6CLmZm8Z%2FScreen%20Shot%202022-10-10%20at%204.19.14%20PM.png?alt=media&#x26;token=934fea0c-3860-41bf-8636-73c4a0c9db90" alt=""><figcaption><p>The project dropdown can be found in the the upper left of Google Big Query page</p></figcaption></figure>

2\.  Find the corresponding ID for the project, copy the Project ID and paste it into the **Project ID** field.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2F9gIMEmyC9jmvFwosDUj6%2FScreen%20Shot%202022-10-10%20at%204.17.17%20PM.png?alt=media&#x26;token=d7cf6750-dde1-4daf-9e3b-ed7d685090ec" alt=""><figcaption><p>The Project ID in this example is <code>example-project-365122</code></p></figcaption></figure>

### Dataset

Provide the name of the dataset that contains the table you want to connect to.

1. To find the dataset information, first **Navigate to the BigQuery page** in the Cloud Console.
2. In the navigation menu, click **SQL Workspace**

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FOs3VLZA9FtQ80i8x0PAl%2FScreen%20Shot%202022-10-10%20at%204.55.51%20PM.png?alt=media&#x26;token=9ce9a991-5f27-4ada-9664-10cc2a23d1a8" alt=""><figcaption></figcaption></figure>

1. In the **Explorer** panel, expand the project name to see the datasets in that project. Copy the dataset name and paste it into the **Dataset** field.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FMu90JtXkGht56nryo4gb%2FScreen%20Shot%202022-10-10%20at%204.57.10%20PM.png?alt=media&#x26;token=1ef87cf2-a91a-49e4-ab1f-24eb8455bb97" alt=""><figcaption></figcaption></figure>

For more information about creating a new Dataset, visit: <https://cloud.google.com/bigquery/docs/datasets>

### Table

**Step 1:** Provide the name of the table you want to connect to. To find the table name, expand the dataset from the step above to see all of the associated tables, and copy the table name and paste it into the **Table** field.

<div align="center"><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MYrsDW6vGBTygB1qqSE%2F-Ma5jeGQFuseFrGwPuPr%2F-Ma5m7P23o9-pG-aBUMI%2Fimage.png?alt=media&#x26;token=01f5d0cb-02f1-466a-8e7b-5821387f6ab7" alt=""></div>

## Advanced Options

#### Validation Webhook

We support the use of Validation Webhooks to prevent bad data from being written to your systems, adding another layer of protection to the built-in validations that Osmos provides. The Webhook URL can be posted here.

{% hint style="info" %}
For more information on Validation Webhook configuration, see [Server Side Validation Webhooks](https://docs.osmos.io/developer-docs/validation-and-transformation-webhooks)
{% endhint %}

## Connector Options

The connector can be deleted, edited and duplicated.

Duplication

To save time, the connector can be duplicated.  This new connector needs to be named and can be edited, as needed.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FdD9lQaH14I0UAhyWtN29%2FCleanShot%202024-01-04%20at%2020.53.21%402x.png?alt=media&#x26;token=3911df70-3a0d-4cdb-ba73-c94ec5317cf0" alt="" width="563"><figcaption></figcaption></figure>