For the complete documentation index, see llms.txt. This page is also available as Markdown.

Pipeline and QC Configuration

Use Pipeline & QC Configuration to register custom pipelines, upload pipeline and QC configuration YAML files, and edit QC metric thresholds for built-in and custom pipelines.

Edit QC Metrics for a Pipeline

Configure the pipeline metrics that appear on the report and edit the default values for each metric as follows.

  1. On the top toolbar, select Configuration, and then select the General tab.

  2. In the left-side pane, select Pipeline & QC Configuration.

  3. Select the applicable pipeline. QC metric configurations display to the right of the selection.

  4. Select Edit.

  5. To add or remove metrics from the report, select the Add to Overview & Report toggle for each metric.

  6. To edit specification limits for applicable metrics, select the Lower Spec Limit or Upper Spec Limit fields and enter the new value. If you cannot edit a value, NA replaces the field.

  7. [Optional] To undo all of your changes, select Reset to Default Values.

  8. Select Save. The updated metrics appear in the Lab QC tab of the case along with the Pass/Fail designations.

Upload a Custom Pipeline Configuration

A custom pipeline is not recognized out of the box. A custom pipeline configuration YAML describes such a pipeline to Connected Insights in three respects:

  1. What the pipeline is — its display name, version, and (optionally) its Platform Core pipeline id for cloud import.

  2. Where each output file is — the path of every VCF, biomarker, QC-metrics, and visualization file, so Connected Insights can locate and ingest them.

  3. How QC is assessed — the QC metrics, their pass/fail thresholds, and which of them are up-leveled to the case Results and report.

Once the file is authored and uploaded, the custom pipeline behaves like a built-in one: Results, Visualize, and Lab QC are populated, and cases can be ingested by auto upload or manual upload.

  1. In Connected Insights, go to Configuration > General and open Pipeline & QC Configuration in the left navigation.

  2. Select + New to open the Upload Pipeline & QC Configuration panel.

  3. Select attached template to download a starter YAML (the same structure as the Sample Custom Pipeline YAML).

  4. Edit the template to describe the pipeline. For the complete YAML schema, refer to Pipeline and QC Configuration YAML. For file-layout sections (sampleSheet, jointFiles, sampleFiles), refer to Custom Pipeline Configuration.

  5. In the panel, Drag and drop the file or select Select a file, then select Upload. The file must be a single .yaml file no larger than 10 MB.

An existing user-uploaded configuration can be modified with Edit, retrieved with Download, or replaced with Re-upload.

Platform Core Linking and Ingestion Paths

Every uploaded custom pipeline supports local ingestion — manual upload and the Data Uploader (folder monitoring). Optionally supplying the Platform Core pipeline ids (icaPipelineId + icaUrn) also links the pipeline to Platform Core so analyses can be auto-imported or manually imported from there. These paths are not mutually exclusive: a pipeline with Platform Core ids can still be used via the Data Uploader. For details on the pipeline: block and the ICA-id pairing rule, refer to Pipeline and QC Configuration YAML.

In Connected Insights v5.5, the platform Connected Insights integrates with is presented to users as Platform Core (formerly Illumina Connected Analytics / ICA). The YAML field names icaPipelineId and icaUrn, and the urn:ilmn:ica:... identifier, are not renamed and must be kept exactly as shown. The Platform Core pipeline id is the same value previously referred to as the "ICA pipeline id".

Validation and Limitations

Connected Insights validates the file at upload. Uploads are rejected when:

  • The file is not a .yaml file, is empty, or is not valid YAML.

  • The pipeline: block is missing, or is missing displayName, name, or version.

  • Exactly one of icaPipelineId / icaUrn is set (they must be supplied together or not at all).

  • The threshold: section has no metrics.

  • A metric is missing its name, is missing its type, or uses a type other than RANGE / BOOLEAN.

Additional considerations:

  • Declare a variant file. Upload validation does not check the file layout, but a configuration with no jointFiles imports no variants and can cause the analysis import to fail. Always declare at least snvFiles. For file-layout details, refer to Custom Pipeline Configuration.

  • Unknown keys are ignored, so a typo in a key name does not raise an error — it silently drops that setting. Field spelling should be verified.

  • Effect on cases: a new or re-uploaded configuration applies to cases ingested after it takes effect; already-ingested cases retain their existing QC/pipeline association until re-ingested.

  • Test before production: there is no live schema editor — a new file should be validated in a non-production workgroup before rollout.

Last updated

Was this helpful?