Pipeline Details Page
Last updated
Was this helpful?
Last updated
Was this helpful?
The Pipeline Details page allows you to review, download and edit important information about the pipeline and associated runs. You can also run the pipeline from the Details page.
The title of the pipeline displays in the top left.
The pipeline can be run, edit mapping or deleted by selecting the buttons on the top right.
The Source and Destination buttons allows you to drill into the individual connector.
The Errors button displays the current number of errors in the pipeline.
Under the Status tab, you can view and take action on the individual pipeline runs. See more detail under Pipeline Actions.
Under the Overview tab, you can view details about the pipeline overall such as date created, total records, schedule, last run and pipeline ID.
Osmos displays the last 30 days of pipeline runs. If you need to modify the number of days to display, please contact support@osmos.io
For each pipeline job, you can view the run date, run time and status.
Your pipeline may have a number of different status:
Complete - 100% successful run with no errors
Finished with Errors - The pipeline completed with one or more errors
Failure - Typically a failure means their is an error with the Source Connector that prevented the pipeline to successfully run
For each job, you can also view of the records are in a Pending, Successful, Skipped or Failed state.
If you have one or more failed records, you may chose to take an Action.
Pipeline Actions are present only when you have at least one failed record on your Pipeline.
Under Actions you can do one of three things:
Run a Download Error Report
Fix Errors
Ignore Errors.
Osmos allows you to download a CSV report of the errors associated with the individual pipeline job.
The report displays the following details:
The row number in the original file which has the errors
A field to show if the error is at the connector level (e.g. auth issues)
Every error row in the report will include all the “source columns for the same row” because often the source column names won’t match the destination column names. Showing the source columns helps the user understand and act on the errors better.
Each destination column will have an “error description” column right next to it, which will contain the error reason if that destination column has errors in it. If that destination column has no errors, the “error description” column will be empty
This option allows you to fix the errors specific to the pipeline run only.
When you select Fix Errors for the pipeline job you will be taken to the spreadsheet view to perform single-cell edit functionality to update the fields in error for the pipeline. Once the updates are complete, the pipeline can be re-run.
If there are 50+ errors in the pipeline, a box will pop up giving you the option to either edit the cells in the spreadsheet view or go back to the column mapping and update the transformations.
This option allows you to simply ignore the errors in the individual pipeline run. This means that the errors will not be carried over to subsequent runs. This is helpful when the error is a one-off and does not require an overall pipeline mapping change. If you select "Ignore Errors" on your pipeline, the Errors Count button on the Pipeline Detail page will reduce the number of errors ignored.
You can streamline your pipeline runs view to show only jobs with records. This will benefit pipelines running on an automated schedule where zero record rows run periodically.
