Data Validators

In this section you will learn how to add validations to the Osmos Uploader schema fields.

Ensure uploaded data is correctly formatted, mapped, and ready to use every time based on your destination's schema.

Custom validation functions can be added to the schema fields in the code snippet directly. They can be added under the validator key. They should be async functions that accept a 2D array of strings as their only argument. This corresponds to the array of all inputs for all rows + columns sent to destination.

Validation functions are called every time a user enters any input in the transform table even if the field that is being validated didn't change. This is done in order to facilitate contextual validation, where the validity of a field depends on the values of other fields.

You can throw both ERROR or WARNING post validation checks.

To throw an error, use theerrorMessage function

To throw a warning, use thewarningMessage function

Here's an example of a validation function that will generate an error if the third column in the schema is equal to "Bob":

{
  "name": "LastName",
  "displayName": "LastName",
  validator: async (rows) => {
    return rows.map(row => {
      if (row[2] === 'Bob') {
        return {
          errorMessage: 'Last name cannot be "Bob"'
        };
      }

      return true;
    })
  }
}

The index passed to the row corresponds to the index in the fields array as given in the uploader snippet found on the uploader description page. As a reminder, fields array is ordered and should not be changed from the ordering given in the snippet.

Note:

• The validation functions must return an array of validation results for each row. The available validation outputs are a boolean true or false or an object with an errorMessage prop. true means that the row is valid, false means that it’s invalid with no error message, and `{ errorMessage: "custom error msg" }` is how you can provide custom error messages. They will be shown to the user when they over over the cell with error.

• Validation functions should not reject/throw errors. We make a best-effort attempt to catch the error, and if we do we ignore the custom validation function entirely and mark all rows as valid. Please use error handling within your validation functions to ensure that they don’t throw/reject.

• Validation functions only validate a single field, even though they are passed all columns for each row as input. If you want to validate multiple fields, add a validation function to each of the fields that you want to validate.

• You can run any code you want in the validation function including making network requests. However, if the validation function takes a long time to run, users will be prevented from continuing on with their transformation of the data until it returns.

• For files over 100,000 rows, data is bucketed into server requests of no more than 100,000 rows each. These bucketed responses are then merged and applied to the data source. This is done to avoid overwhelming customer validation endpoints.