AI Value Mapping
This feature is currently in Beta.
Last updated
Was this helpful?
This feature is currently in Beta.
Last updated
Was this helpful?
What is AI Value Mapping?
Getting Started
AI Value Mapping allows you to map end-user data into categories with your organization’s internal representation (commonly known as enumerations “enums”). Leveraging advanced Large Language Models (LLMs), this QuickFix automatically maps source data to its nearest semantic match in the destination schema. This saves your data operations teams hours of manual work while dynamically adapting to evolving source data without requiring updates to formulas or webhooks.
Value Mapping can be used throughout Osmos in both Uploaders and Pipelines. It is most commonly used with an Uploader where end users map enumerations ("enums"). Enums are integrated as part of the Uploader Validation. Users map a group of constants to a specific valid option. The list of valid options is configured in your validation and the mapping occurs in the Value Mapping QuickFix.
There are two primary types of configurations.
Type 1: Create a Dataset Table to hold your list of valid options. Then, set up a column in your destination Dataset Table. This configuration can be used for Pipelines and Uploaders. .
Type 2: With an Embedded Uploader, create a dropdown-controlled field that contains your list of valid options in the code snippet. Here is a link to this .
One-time setup items:
Create a valid options list in a Dataset table
Create a standard Uploader (with Destination Datasets) with a Foreign Key field to look up the valid option.
Customer ABC is ingesting employee data. The employee data includes employee type, which needs to be narrowed down to Customer ABC's list of valid options. In this scenario, see how to use Value Mapping to group a field to a specific list of options.
Datasets
Pipelines and/or Uploader
Create the Dataset to hold the valid options.
Best Practice is to use one Dataset to hold all of your lookup tables.
Within your new Dataset, create the first table called Employee_Category. This table will have one field called 'Employee Category', set to required field, and selected as a Primary key field. Additional fields may be included but will not be featured in this scenario. Save your table.
Either create a new Dataset or within the existing Dataset, create a second table called 'Employees'. This table will be the destination for your ingested data and will have all of our destination data fields.
AI Value Mapping requires a foreign key relationship on the destination field where the valid option will reside. To set up the relationship, select Add Foreign Key from the table creation page.
Enter local field name: 'Employee_Type',
Select the project where the valid options Dataset resides
Select the Dataset where the valid options reside
Select the Table
Select the Reference Field for the corresponding valid option.
Hit Add Foreign Key
At this point, we have established a relationship between the Employee_Category table and the Employees table. This relationship tells the system that only values that have been submitted to the 'Employee Category' field of the Employee_Categories table are valid options for the 'Employee_Type' field in the Employees table.
Step 1: Go to your newly created Uploader and upload your data. Once the AutoMapping is complete, select the field you wish to initiate the AI Value Mapping. In our scenario it is Employee_Type.
Step 2: However on the Column Mapping > Select Transform on the left-hand side column mapping field. In this scenario, it will be the destination Employee_Type field.
Step 3: Select the AI Value Mapping QuickFix.
AI Value Mapping will automatically run and map. You can then make adjustments and/or map any fields left unmapped.
Step 4. Hit Save when you are done to save the mapping logic.
Step 1: In the AI Value Mapping Window, select the box next to an unmapped field
Step 2: Scroll to the bottom and select the valid option to map the field to
Step 3: Hit Apply
Step 4: Hit Save Mapping