AutoClean
Last updated
Last updated
What is Auto Clean?
Getting Started
AI Value Mapping (Enums)
Clean up operations
AutoClean is a capability that allows end users to clean data with one click. Your Osmos Uploader comes with prebuilt AutoClean capability for certain scenarios detailed below but also empowers developers with the ability to transform data on the fly by creating their AutoClean functions. AutoClean also allows end users to quickly map a dropdown list for enums.
AutoClean is a feature for the Standard Mode Uploader.
Step 1: Once you have mapped the input (source) column(s) to the output (destination) column, you can activate the AutoClean toggle to on.
Step 2: If there is data that can be cleaned, it will display a sample of the data to be cleaned. Review the sample of the new values and hit Accept.
If there is no data to clean, you will receive a message that no data can be AutoCleaned.
Step 3: The AutoCleaned data will maintain a purple visual to let you know it has been cleaned through the AutoClean process. Any field can still be edited directly in the field itself.
Step 4: To Review all of your AutoCleaned Records, you can select the Filter at the top.
Note: For Uploader embedded deployments, you can update the configuration schema to always have AutoClean on for a specific field(s).
Value Mapping is a capability that allows end users to map enumerations ("enums"). Enums are integrated as part of the Uploader Validation. Users can now map a group of constants to a specific valid option. For the Standard Mode, the list of valid options is configured in your validation and the mapping occurs in AutoClean.
The table below describes what cleanup operations will be performed by AutoClean, depending on the data type of the destination field, and whether or not the field is required.
Integer
Strip non-numeric symbols
Example: $8 -> 8
Round to the nearest whole number
Example: 8.34 -> 8
Example: $8.79 -> 9
Strip non-numeric symbols
Example: $8 -> 8
Round to the nearest whole number
Example: 8.34 -> 8
Example: $8.79 -> 9
If no value is in the source data, enter 0
If source data is not parse-able as a number, it will remain unaltered and show an error
Float
Strip non-numeric symbols
Example: $8.79 -> 8.79
Strip non-numeric symbols
Example: $8.79 -> 8.79
If no value is in the source data, set to 0.0
If source data is not parse-able as a float, it will remain unaltered and show an error
Date
If data is not parse-able as a date, it will be set to null
If data is not parse-able as a date, it will remain unaltered and show an error
Datetime
If data is not parse-able as a date and time, it will be set to null
If data is not parse-able, it will remain unaltered and show an error
Boolean
0, F, False, N, No (case insensitive) will map to false
1, T, True, Y, Yes (case insensitive) will map to true
If data is none of the above, the output will be set to null
0, F, False, N, No (case insensitive) will map to false
1, T, True, Y, Yes (case insensitive) will map to true
If data is none of the above, it will remain unaltered and show an error
Text
Text will remain unaltered by Osmos AutoClean
Text will remain unaltered by Osmos AutoClean
Note: You can also set up your own instances of AutoClean to perform custom cleanup operations on input data by setting up Server Side Validation Webhooks for the destination connector of your uploader. Please review the AutoClean Developer Docs for more info.
