Configuring AutoClean for your Uploader

Function

AutoClean is a capability which allows end users clean data with one click. Your Osmos Uploader comes with prebuilt AutoClean capability for certain scenarios detailed below.

AutoClean is not available in Advanced Mode. For more detail on how to clean messy data in Advanced Mode, please see our Advanced Mode documentation.

In step 2 of an upload, Map and Transform Data, AutoClean is displayed as a toggle above the "our Field" pane.

AutoClean in Action

AutoClean Configuration Example

On your schema fields, you can add an attribute called autoCleanMode that will let you configure whether or not AutoClean is on or off by default, or disabled entirely, for a given field. The options are:

• 'auto': Toggle is off by default, can be turned on. This is the default setting if you don't specify a mode for a field.

• 'onForced': AutoClean is on by default and cannot be turned off.

• 'onButAllowDisable': AutoClean is on by default and can be turned off.

• 'disabled': AutoClean is off and cannot be turned on for this field.

 Osmos.configure({
        schema: {
          fields: [
            {
              name: 'date',
              displayName: 'date',
              description: '<Your field description here>',
              // Disabled for this field, cannot be turned on
              autoCleanMode: 'disabled',
            },
            {
              name: 'datetime',
              displayName: 'datetime',
              description: '<Your field description here>',
              // autoCleanMode defaults to 'auto' if you don't set it
            },
            {
              name: 'bool',
              displayName: 'bool',
              description: '<Your field description here>',
              // Toggle starts off, and you can turn it on
              autoCleanMode: 'auto',
            },
            {
              name: 'int',
              displayName: 'int',
              description: '<Your field description here>',
              // Toggle starts on, and you cannot turn it off
              autoCleanMode: 'onForced',
            },
            {
              name: 'double',
              displayName: 'double',
              description: '<Your field description here>',
              // Toggle starts on, and you can turn it off
              autoCleanMode: 'onButAllowDisable',
            },
          ],
        },
        token: 'your_token',
        uploadDescription:
          '<Include a description of your uploader upload here which will be shown to users that click the uploader>',
        hideUploadSchema: true,
        hideUploadDescription: true,
        disableAdvancedMode: true,
      });

AutoClean Operations

The table below describes what cleanup operations will be performed by AutoClean, depending on the data type of the destination field, and whether or not the field is required.

Destination Field Type	Nullable	Required
Integer	Strip non-numeric symbols Example: $8 -> 8 Round to the nearest whole number Example: 8.34 -> 8 Example: $8.79 -> 9	Strip non-numeric symbols Example: $8 -> 8 Round to the nearest whole number Example: 8.34 -> 8 Example: $8.79 -> 9 If no value is in the source data, enter 0 If source data is not parse-able as a number, it will remain unaltered and show an error
Float	Strip non-numeric symbols Example: $8.79 -> 8.79	Strip non-numeric symbols Example: $8.79 -> 8.79 If no value is in the source data, set to 0.0 If source data is not parse-able as a float, it will remain unaltered and show an error
Date	If data is not parse-able as a date, it will be set to null	If data is not parse-able as a date, it will remain unaltered and show an error
Datetime	If data is not parse-able as a date and time, it will be set to null	If data is not parse-able, it will remain unaltered and show an error
Boolean	0, F, False, N, No (case insensitive) will map to false 1, T, True, Y, Yes (case insensitive) will map to true If data is none of the above, the output will be set to null	0, F, False, N, No (case insensitive) will map to false 1, T, True, Y, Yes (case insensitive) will map to true If data is none of the above, it will remain unaltered and show an error
Text	Text will remain unaltered by Osmos AutoClean	Text will remain unaltered by Osmos AutoClean

Customizing AutoClean Output

You can override or augment AutoClean output data by setting up Server Side Validation Webhooks for the destination connector of your uploader.

By default, AutoClean will run, then send the resulting data to the webhook. The webhook can then return final writeback values, errors, or warnings to the Osmos data mapping page.

If you don't want AutoClean to be performed before data is sent to the webhook for a given field you can set that field's autoCleanMode to 'disabled' as mentioned in the AutoClean Configuration Example section above. In which case the input data will be sent to your webhook unaltered. When autoCleanMode is set to disabled, writeback values from the validation webhook will not be applied.