# AutoClean

## Contents

1. What is Auto Clean? 
2. Getting Started
3. AI Value Mapping (Enums)
4. Clean up operations

## What is AutoClean?

AutoClean is a capability that allows end users to clean data with one click. Your Osmos Uploader comes with prebuilt AutoClean capability for certain scenarios detailed below but also empowers developers with the ability to transform data on the fly by creating their AutoClean functions.  AutoClean also allows end users to quickly map a dropdown list for enums.

{% hint style="info" %}
AutoClean is a feature for the Standard Mode Uploader.
{% endhint %}

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2Fg0zFwD4RaAceDSII0vR0%2FCleanShot%202023-07-18%20at%2010.19.05.gif?alt=media&#x26;token=edea8f54-8383-4633-9b9f-5c01318e180a" alt=""><figcaption></figcaption></figure>

## Getting Started

**Step 1**: Once you have mapped the input (source) column(s) to the output (destination) column, you can activate the AutoClean toggle to on. &#x20;

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2F3dkY86Von2QWF3Q94KbL%2FCleanShot%202023-07-18%20at%2011.39.05.png?alt=media&#x26;token=8305052f-069e-4faa-afe8-ca78ec9a59aa" alt=""><figcaption></figcaption></figure>

**Step 2**: If there is data that can be cleaned, it will display a sample of the data to be cleaned.  Review the sample of the new values and hit Accept.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2Fb4UCXNXXY4yhS7k3cmOD%2FCleanShot%202023-07-18%20at%2011.40.07.png?alt=media&#x26;token=c598a40a-96d2-4c17-9342-ed4a4e1dfd94" alt=""><figcaption></figcaption></figure>

If there is no data to clean, you will receive a message that no data can be AutoCleaned.

**Step 3:**  The AutoCleaned data will maintain a purple visual to let you know it has been cleaned through the AutoClean process.  Any field can still be edited directly in the field itself.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2Flw6mKmiR1di0E7LCFZKE%2FCleanShot%202023-07-18%20at%2011.46.40.png?alt=media&#x26;token=3f93094a-f0eb-4f1b-bca5-660a1acdaf49" alt=""><figcaption></figcaption></figure>

Step 4:  To Review all of your AutoCleaned Records, you can select the Filter at the top.

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FSa0GJSi5bbBhMnNeGXsz%2FCleanShot%202023-07-18%20at%2012.25.10%402x.png?alt=media&#x26;token=a86ed88d-5234-4174-a04b-d6eaef8dcf5e" alt=""><figcaption></figcaption></figure>

### AI Value Mapping through AutoClean

{% hint style="info" %}
Note: For[ Uploader embedded deployments,](https://docs.osmos.io/developer-docs/configuring-autoclean-for-your-uploader) you can update the configuration schema to always have AutoClean on for a specific field(s). &#x20;
{% endhint %}

### AI Value Mapping AutoClean

Value Mapping is a capability that allows end users to map enumerations ("enums").  Enums are integrated as part of the Uploader Validation.  Users can now map a group of constants to a specific valid option.  For the Standard Mode, the list of valid options is configured in your validation and the mapping occurs in AutoClean.&#x20;

#### AI Value Mapping in Action

<figure><img src="https://353417064-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MYrsDW6vGBTygB1qqSE%2Fuploads%2FQaycuDcNGGxYY1xMhRMo%2FCleanShot%202023-09-15%20at%2007.32.05.gif?alt=media&#x26;token=639a6f42-4a52-4808-b7a8-2cd5bc8c9157" alt=""><figcaption></figcaption></figure>

### Osmos AutoClean Operations

The table below describes what cleanup operations will be performed by AutoClean, depending on the data type of the destination field, and whether or not the field is required.

| Destination Field Type | Nullable                                                                                                                                                                                                                   | Required                                                                                                                                                                                                                                                                                                                                                         |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Integer                | <ol><li><p>Strip non-numeric symbols</p><ul><li>Example: $8 -> 8</li></ul></li><li><p>Round to the nearest whole number</p><ul><li>Example: 8.34 -> 8</li><li>Example: $8.79 -> 9</li></ul></li></ol>                      | <p></p><ol><li><p>Strip non-numeric symbols</p><ul><li>Example: $8 -> 8</li></ul></li><li><p>Round to the nearest whole number</p><ul><li>Example: 8.34 -> 8</li><li>Example: $8.79 -> 9</li></ul></li><li>If no value is in the source data, enter 0</li><li>If source data is not parse-able as a number, it will remain unaltered and show an error</li></ol> |
| Float                  | <p></p><ol><li><p>Strip non-numeric symbols</p><ul><li>Example: $8.79 -> 8.79</li></ul></li></ol>                                                                                                                          | <ol><li><p>Strip non-numeric symbols</p><ul><li>Example: $8.79 -> 8.79</li></ul></li><li>If no value is in the source data, set to 0.0</li><li>If source data is not parse-able as a float, it will remain unaltered and show an error</li></ol>                                                                                                                 |
| Date                   | If data is not parse-able as a date, it will be set to null                                                                                                                                                                | If data is not parse-able as a date, it will remain unaltered and show an error                                                                                                                                                                                                                                                                                  |
| Datetime               | If data is not parse-able as a date and time, it will be set to null                                                                                                                                                       | If data is not parse-able, it will remain unaltered and show an error                                                                                                                                                                                                                                                                                            |
| Boolean                | <p></p><ul><li>0, F, False, N, No (case insensitive) will map to false</li><li>1, T, True, Y, Yes (case insensitive) will map to true<br><br></li></ul><p>If data is none of the above, the output will be set to null</p> | <p></p><ul><li>0, F, False, N, No (case insensitive) will map to false</li><li>1, T, True, Y, Yes (case insensitive) will map to true</li></ul><p><br></p><p>If data is none of the above, it will remain unaltered and show an error </p>                                                                                                                       |
| Text                   | Text will remain unaltered by Osmos AutoClean                                                                                                                                                                              | Text will remain unaltered by Osmos AutoClean                                                                                                                                                                                                                                                                                                                    |

{% hint style="info" %}
Note: You can also set up your own instances of AutoClean to perform custom cleanup operations on input data by setting up [Server Side Validation Webhooks](https://docs.osmos.io/developer-docs/validation-and-transformation-webhooks) for the destination connector of your uploader.  Please review the [AutoClean Developer Docs](https://docs.osmos.io/developer-docs/configuring-autoclean-for-your-uploader) for more info. &#x20;
{% endhint %}