Local Run Manager TruSight Tumor 15 Analysis Module v2.1
Archer Analysis v7 (VariantPlex and FusionPlex panels)
Archer VariantPlex Pipeline v7.3
Pillar Biosciences RUO PiVAT Pipeline v2024
Loading...
Loading...
Loading...
TMB JSON Minimal
TMB JSON Full
TMB CSV Minimal
TMB CSV Full
MSI JSON Minimal
MSI JSON Full
GIS JSON Minimal
Loading...
HRD CSV Full
HLA TSV Full
Loading...
Loading...
Loading...
Loading...
Loading...
Welcome
Connected Insights accepts data from various DNA and RNA oncology assays and variant callers, supports variants frequently identified in tumor samples (e.g., SNVs, indels, CNVs, SVs, fusions, splice variants) and genome-wide biomarkers (e.g., TMB, MSI, HRD), integrates multiple genomic databases to annotate data, providing insights into biological significance, and generates customized comprehensive summary reports.
Access Connected Insights
There are two ways to access Connected Insights:
Cloud
This deployment is hosted by Illumina.
🖊 Register your software
To register your software, refer to the Software Setup documentation.
During the software registration process, you will need to create a domain and workgroup. Domains and Workgroups are used by Illumina Software to control access to your data and assets. When logging into Connected Insights, you will be required to select a workgroup to establish the context for your work.
Make sure all users are added to the workgroup and that the workgroup has the necessary permissions to access the Connected Insights application. For more information, refer to the Workgroup documentation.
Access with a personal context
You can also access Connected Insights with a personal context. On your very first login, the platform automatically creates your domain and personal context. Select the Connected Insights tile from the Product Dashboard to start.
Once you've logged in, click on a section below to help you navigate Connected Insights.
Configure
Manage cases
Interpret and report
How to Upgrade (Cloud)
What version am I using?
There are a few ways to find out which version you are using:
Navigate to Cases Page and inspect the bottom right corner of the page.
Navigate to the workgroup selector by clicking your workgroup name in the top right corner of the page and click Change Workgroup.
Navigate to Configuration by clicking the ⚙ icon in the top right corner of the page and click the Software Update tab.
Software Update tab
How do I upgrade to the next version?
If you wish to upgrade, follow these steps:
Decide which workgroups you want to transition to the new version and when. You can upgrade one, several or all of workgroups as convenient for your organization.
Then, a lab director user needs to:
Click your workgroup name in the top right corner of the page.
Click Change Workgroup.
Click the workgroup that you want to upgrade.
Click Next.
Navigate to Configuration Page by clicking the ⚙ icon in the top right corner of the page
Click the Software Update tab.
Click Update. The workgroup will be upgraded to the next software version.
Repeat the above steps for each workgroup you want to upgrade.
Custom Case Data
The Custom Case Data page allows you to configure fields for case, subject, and sample information that accompanies cases in Connected Insights. The configuration covers field names, data type, restriction status, and the field category. When you upload the custom case data file, the fields in the file must be formatted to match the definition provided on this page. The information appears in the user interface on the Case Details screen and in the JSON and PDF reports in the corresponding sections.
To configure custom case data:
On the top toolbar, navigate to Configuration represented by the cog wheel icon.
Navigate to Case Data Definition.
In the right-side pane, select the Data Type from the drop-down list. This data type can be text, number, or a date. The type impacts the accepted inputs during data ingestion.
Enter a Custom Name. This name is the label that appears in the user interface and the PDF report.
Choose if you want this data to be Restricted from the drop-down list.
Select Yes if the custom case data are sensitive. When you view the Case Details, the data are hidden with asterisks as indicators, and only displays if you select Show/Hide.
If the data are non-sensitive, then select No.
Select Case, Subject, or Sample as the Associated Entity from the drop-down list. The corresponding data appears in the Case Details and on the PDF report.
Use the drag handles to change the order that data are displayed in Connected Insights and the final report.
Select Add to add the new custom case data definitions to the PDF report.
[Optional] Select the archive icon next to the custom case data definition to archive it so that it no longer appears on the final report.
[Optional] Select the X next to custom case data to remove an entry.
Preferred Transcripts
The Preferred Transcripts section allows you to upload a preferred transcript file that defines which transcripts for a gene are preferred. These transcripts are prioritized over other transcripts of the gene when the default transcript for a variant is determined. For more information, refer to Transcripts.
Upload the preferred transcript file as follows.
On the top toolbar, select Configuration, and then select the General tab.
Select Preferred Transcripts.
If there are no preferred transcripts, select Create One.
For Template File, select Upload and select your transcript file. You can also download a template to customize and upload as your preferred transcript file.
Select Open.
After the file displays, Select Save.
Select Download & View to download the transcript file.
[Optional] To upload an updated version of the file, select Upload New Version.
Variant Flag Groups
Variant flag groups allow you to organize variants later in the case interpretation. You can add flags for common tasks such as follow-up or to track review statuses.
Flag groups marked Provided by Illumina cannot be updated. Select Copy As New to copy one into your workgroup and edit the copy. See Illumina-Provided Default Resources.
On the top toolbar, select Configuration.
Select the Test Components tab.
In the left-hand pane, next to the Variant Flag Groups accordion, select New.
In the right-hand pane, enter a Variant Flag Group Name.
Enter a Flag Name.
Choose a flag color.
Select Add.
Create additional flags as needed.
Use the drag handles to change the order that data are displayed in Connected Insights and the final report.
Select Save. The variant flag group appears on the left-hand accordion menu. Select through the flag groups to display previews.
Select Copy As New to copy a flag group.
Select Archive to make the flag group unavailable when you create a test definition.
Actionability Classifications
Connected Insights provides default actionability classifications based on AMP/ASCO/CAP Li et al., 2017. You may use the predefined ESMO ESCAT, Mateo et al., 2018 definitions when selecting and/or creating new assertions by selecting ESMO ESCAT, Mateo et al., 2018 or you can create your own custom actionability classifications.
The default classifications are configured based on AMP/ASCO/CAP Li et al., 2017, shared with every workgroup, and cannot be updated. See Illumina-Provided Default Resources.
Custom actionability classifications can be set up in Configuration. Later, when you create therapeutic, prognostic, and diagnostic assertions, you can specify these classifications.
Create custom actionability classifications as follows.
On the top toolbar, select Configuration.
In the General tab, select Actionability Classification. The default actionability classifications are based on AMP/ASCO/CAP Li et al., 2017. If you want to specify your own, select I want to classify using my own classifications.
Provide a name for the actionability classifications.
Select Add.
Add a classification name (eg, Level 1).
[Optional] Add a description.
Choose a color.
Select Save.
Repeat steps 4–8 until you have your classification created.
Classifications can be deleted by selecting x on the created classification.
Classifications can be edited by selecting the pencil icon on the created classification.
Use the drag handles to map your classifications to the AMP/ASCO/CAP Li et al., 2017 guideline on the right side of the window. Place your classifications in order of their significance. After you create assertions, this order is used to prioritize variants and evidence.
❗ Whenever you edit these classifications, it does not modify existing assertions in your knowledge base or reports. When you encounter assertions using older classifications, they can be edited to use the latest classification.
Lab Information
The Laboratory Information page allows you to configure fields for your laboratory which will appear on your PDF report. The configurable information includes laboratory name, address, town/city, state/province/region, ZIP/postcode, and country. You may also upload a laboratory logo (jpeg or png, maximum 1MB), which will appear in your PDF report.
To configure Laboratory Information:
On the top toolbar, select Configuration.
Navigate to Lab Information
Select Edit to edit or add data and to upload a logo.
Enter the laboratory information you would like to appear on the PDF report.
gene num_alleles allele_1 reads_supporting_allele_1 EM_posterior_allele_1 allele_2 reads_supporting_allele_2 EM_posterior_allele_2 Notes
A 1 A*32:01 1227 1.000000 NA NA NA NA
B 1 B*15:01 1158 1.000000 NA NA NA NA
C 2 C*03:03 1361 0.770833 C*03:04 1310 0.229167 NA
DMA 1 DMA*01:01 1833 1.000000 NA NA NA NA
DMB 1 DMB*01:01 2247 0.975507 NA NA NA NA
DOA 1 DOA*01:01 1497 1.000000 NA NA NA NA
DOB 1 DOB*01:01 1769 1.000000 NA NA NA NA
DPA1 1 DPA1*01:03 3411 1.000000 NA NA NA NA
DPA2 2 DPA2*01:01 2560 0.562016 DPA2*02:01 2528 0.437984 NA
DPB1 2 DPB1*02:01 4360 0.443701 DPB1*31:01 3013 0.151119 NA
DPB2 2 DPB2*01:01 5734 0.749003 DPB2*03:01 3784 0.242016 NA
DQA1 2 DQA1*01:01 1869 0.740983 DQA1*06:01 603 0.225755 NA
DQA2 1 DQA2*01:01 2301 0.988328 NA NA NA NA
DQB1 2 DQB1*05:01 2129 0.751502 DQB1*03:01 719 0.248498 NA
DQB2 1 DQB2*01:01 2768 1.000000 NA NA NA NA
DRA 2 DRA*01:01 1876 0.814725 DRA*01:03 1098 0.185275 NA
DRB1 2 DRB1*01:01 779 0.731210 DRB1*08:03 290 0.171451 NA
DRB5 2 DRB5*02:02 49 0.608936 DRB5*02:21 33 0.291917 Allele1:LowSupportingReads Allele2:LowSupportingReads
E 2 E*01:03 1402 0.793814 E*01:01 1345 0.206186 NA
F 1 F*01:01 1420 1.000000 NA NA NA NA
G 1 G*01:01 1361 0.971827 NA NA NA NA
H 1 H*02:03 1386 1.000000 NA NA NA NA
J 1 J*01:01 1526 0.998765 NA NA NA NA
K 1 K*01:01 1403 0.990965 NA NA NA NA
L 1 L*01:01 1378 0.996270 NA NA NA NA
N 1 N*01:01 209 1.000000 NA NA NA NA
P 1 P*02:01 1192 0.999039 NA NA NA NA
S 2 S*01:04 314 0.809390 S*01:02 183 0.153795 NA
T 1 T*02:01 919 0.956639 NA NA NA NA
U 2 U*01:02 208 0.721489 U*01:03 71 0.215357 NA
V 1 V*01:01 794 1.000000 NA NA NA NA
W 2 W*04:01 1026 0.825142 W*02:01 664 0.168624 NA
MICA 1 MICA*010:01 5188 1.000000 NA NA NA NA
MICB 2 MICB*005:02 4603 0.630347 MICB*019:01 4283 0.298161 NA
TAP1 1 TAP1*01:01 3380 0.999999 NA NA NA NA
TAP2 2 TAP2*02:01 3481 0.544792 TAP2*01:01 3308 0.374378 NA
HFE 1 HFE*001:01 3 1.000000 NA NA NA Allele1:LowSupportingReads
DRB3 0 NA NA NA NA NA NA NA
DRB4 1 DRB4*01:183 20 0.125000 NA NA NA Allele1:LowSupportingReads
Y 0 NA NA NA NA NA NA NA
R 0 NA NA NA NA NA NA NA
Data Upload
This section describes how to upload data with Connected Insights and includes setup instructions. Connected Insights requires the following data types:
Secondary analysis output data — Connected Insights is compatible with a broad range of analysis pipeline outputs. To configure the input file format, select a compatible pipeline (for example, DRAGEN TruSight Oncology 500 v2.5.2) or configure a custom pipeline.
Case, subject, and sample data — Connected Insights will read case, subject, and sample data passed along with the analysis pipeline outputs when available from the following files:
&#xNAN;Samplesheet.csv - Will read Sample_ID, Tumor_Type, Pair_ID, Sample_Type, and Sex.
&#xNAN;Fastq-list.csv - Will read RGSM to determine Sample Id.
&#xNAN;Replay.json - Will determine Sample Id for tumor (and normal) sample(s).
For more information on custom case data such as to upload additional case metadata, refer to Custom Case Data Upload
Connected Insights - Cloud allows data upload from the following sources:
From computer via browser - Data ingestion supports upload of a single tumor-only VCF through the browser. For more information on manual upload from computer, refer to Data Upload from Computer.
Cloud storage on ICA and BSSH — Data ingestion can be configured through the Data Upload page or API. For more information on using the Data Upload tab or API, refer to Data Upload from ICA (Connected Insights - Cloud).
Before uploading, create the following items:
Custom case data (if applicable)
Test components
Test definitions (for more information on creating test components and definitions, refer to Test Definition Setup)
Custom Annotations
Custom Annotations allow you to annotate custom attributes (labels, scores, and population frequency) to variants or variants within a genomic region.
Create a new Custom Annotation file
Select the Configuration icon (gear icon) in the top right area of the screen.
The supported data categories for custom annotation in Connected Insights are as follows:
Filter: String labels
Score: Numeric values
AlleleFrequency: Allele frequency for population frequency data
Upload a new Custom Annotation file
Select the Configuration icon (gear icon) in the top right area of the screen.
Select the Test Components tab.
Navigate to Custom Annotations.
Select New.
Upload the file.
Download an existing Custom Annotation file
On the top toolbar, select Configuration, and then select the Test Components tab.
Select the Custom Annotations section.
Select the desired custom annotation.
Select Download.
Annotating Variants
After a custom annotation is added to a Test Definition and cases are ingested using the test definition, the variants will be annotated. The annotations can then be viewed and filtered on.
When matching annotations for large variants (structural variants and copy number variants), by default Annotation Overlap (measured as the total overlap divided by the length of the annotation) of >= 0.75 is used, but can be changed to Reciprocal Overlap (measured as the total overlap divided by either the length of the annotation or the length of the variant, whichever is larger) by specifying Reciprocal Overlap in the Description column for the annotation. For more information, refer to Custom Annotation File Format.
Report Templates
Connected Insights requires you to provide a report template for each test definition to ensure consistency of reports produced across cases. This section summarizes the actions that can be performed with report templates to customize the look and content of PDF reports to your specifications.
Create several report templates and use them in more than one test definition.
Use a default report template provided by Illumina, which is shared with every workgroup and cannot be updated, or select Copy As New to create your own copy and customize the report as needed. See Illumina-Provided Default Resources. Customizations can be done by uploading an edited report template or through ICI configuration, these customizations include:
– Changing the logo of the laboratory.
– Changing the report name.
– Reformatting the page and date.
– Changing text in the Supplemental Information section.
On the top toolbar, select Configuration, and then select the Test Components tab.
Navigate to Report Templates and select New.
Select Download the template.
Save the base ODT template.
Edit the file with an applicable document editor. The document editor must be able to load v1.3 ODT files. For information on making custom edits to the report, refer to Report Customizations.
Select Upload template to re-upload the template with the recently made changes. Uploaded files must in the ODT format.
Select Confirm Report Creation when the report preview displays. Download the original report in the ODT format at any time. You can also archive and unarchive report templates. Archiving a report template makes it unavailable for use when creating a test definition.
Download Report Template
On the top toolbar, select Configuration, and then select the Test Components tab.
Select Report Templates, and then select Pre-configured Demo Report Template or Pre-configured Demo Report Template (Version 2.3).
After the report loads, select Download as ODT.
Test Changes After Report Template Customization
On the top toolbar, select Configuration, and then select the Test Components tab.
Navigate to Report Templates and select New.
For Create your new report, select Upload template.
Using file explorer, navigate to the template with the changes and select Open.
Review the PDF preview to make sure that the modified report displays as intended.
Use Illumina Connected Insights with a personal context from the Product Dashboard.
A personal context provides an individual workspace in Illumina Connected Insights - Cloud. It is available from the Illumina BioSight Platform Product Dashboard.
Verify your email address using the verification email.
Sign in to the platform.
The platform creates an eight-digit domain and personal context.
Select either a trial or pay-as-you-go subscription.
After you select a subscription, the Product Dashboard appears.
Select the Illumina Connected Insights tile.
Trial access
A trial provides 30 days of application access. A further 30-day grace period follows. If you do not purchase a subscription, access ends after 60 days.
How personal contexts work
Connected Insights treats your personal context as a virtual workgroup. It is unique to you and uses your user ID as its workgroup ID.
Your virtual workgroup is selected when you open Connected Insights. Projects and access permissions are applied to this personal context. You can then work with cases as usual.
Personal contexts support individual work. Use a standard workgroup when you need to share data and assets with other users.
The Discover page lets you explore Connected Insights with demo cases prepared by Illumina, without uploading any data of your own. Use it to see how Connected Insights handles the applications it supports, from the case Results page through to the report.
An application is an application or workflow that Connected Insights supports. Illumina curates the applications, demo cases, and resources shown on Discover, and they can change over time. Some applications may not have demo cases yet.
Demo cases are read-only, and storing them does not consume credits. The Discover page is available in Connected Insights - Cloud only.
Open the Discover Page
In the header, select the application selector.
Select Discover.
If your workgroup has no cases, Connected Insights opens the Discover page instead of the Cases Page. To return to your cases, select Connected Insights in the header.
Browse Applications
The Discover page shows one card for each application. A card includes the application icon, a description, and the number of available demo cases.
Select an application card that has demo cases.
Review the application description and the list of demo cases.
To return to the full list of applications, select All applications.
Each demo case shows summary information, such as its title, description, and assay details. Each demo case also provides:
Report PDF — opens the case report in a new tab.
Configuration Preview — opens the test definition for the case, read-only, in a new tab.
Open a Demo Case
Select a demo case to open its Results page. Explore the case as you would any other case:
The Variant Filters section describes how to add, configure, copy, or archive variant filters.
Add and Configure a Variant Filter
On the top toolbar, select Configuration.
Select the Test Components tab.
Navigate to Variant Filters and select New.
Navigate to the Condition Group section and enter the applicable genes in the Genes field.
[Optional] Select the Include genes from diseases check box.
Select Apply.
Select Include or Exclude from the drop-down list for each gene.
For Variant Category, select the field to open up a drop-down list of variants that can be included or excluded.
Select the check boxes next to the applicable variant types.
Select Add Criteria and select the applicable criteria from the drop-down list (for example, Cancer Hotspot Samples).
Select Include or Exclude from the drop-down list for the criteria.
To include or exclude additional criteria, repeat steps 10 and 11.
For AND or OR, select Add Condition Group to add any other applicable condition groups.
[Optional] Select Reset to undo the changes to the condition groups.
Select Save.
The Genomic Region criteria is not available when creating a filter from configuration but can be included when adding/editing filters in Variant grid . For more information, refer to .
On the top toolbar, select Configuration.
Select the Test Components tab.
For Variant Filters, select a filter.
Select Edit.
On the top toolbar, select Configuration.
Select the Test Components tab.
For Variant Filters, select the applicable filter.
In the right-hand pane, select Copy As New.
On the top toolbar, select Configuration.
Select the Test Components tab.
For Variant Filters, select the applicable filter.
In the right-hand pane, select Archive.
Connected Insights provides the following default filters, marked Provided by Illumina, to assist in preliminary test implementation. Default filters cannot be updated; select Copy As New to copy one into your workgroup and adjust it. See .
All Demo
Filters for oncogenic, likely oncogenic, 1A, 1B, and 2C small variants, copy number variants, unidirectional gene fusions, and RNA splice variants.
Data Upload from BioInsight Platform Core (formerly Connected Analytics) or BSSH
Connected Insights - Cloud supports data uploads from BioInsight Platform Core (formerly Connected Analytics) and BaseSpace Sequence Hub (BSSH) for defined workflows.
Requirements
BioInsight Platform Core
You must be a member of the workgroup to which the data is uploaded. A project not linked to the workgroup must be given permissions at the level of Contributor in the Team tab of BioInsight Platform Core to be accessible from Connected Insights. User must have "Lab Director" role for Connected Insights to run the ici-uploader. For more information on adding users to a workgroup and adding permissions, refer to .
BSSH
You must be a member of the workgroup to which the data is uploaded in both Connected Insights and BSSH. The user must have permission at the level of Has Access in BSSH to be accessible from Connected Insights.
Configure Auto-pickup for a Supported BioInsight Platform Core Pipeline
This section provides instructions for adding a supported BioInsight Platform Core pipeline.
In Connected Insights, select the gear in the top-right corner to open the Configuration page.
Select the General tab.
Select Data Upload.
Select the From BioInsight Platform Core tab.
For Cloud Pipelines, select Add.
For Choose compatible pipeline from the catalog, select the applicable pipeline from the drop-down list, refer to .
For Test Definition, select the applicable definition.
Select Save.
ℹ️ For the configured BioInsight Platform Core Pipeline, Connected Insights will automatically upload new analyses in BioInsight Platform Core for the specified workflow. When data is uploaded, the status to the right of the workflow name will show the last analysis upload date. If a new project is created or linked after the configuration is made, it will need to be refreshed by editing and saving the configuration before it will begin to pick-up data from that project.
Locate the pipeline, and then select ...
Select Delete.
When prompted, select Yes, Remove.
This section provides instructions for manually uploading a completed analysis for a supported BioInsight Platform Core pipeline.
From the Case List, select + New Case.
Select the Import from BioInsight Platform Core option and click the button for Import from BioInsight Platform Core.
Ensure the correct project is selected under Select Project.
Select the desired analysis to upload and click the button to Continue.
This section provides instructions for manually uploading a completed analysis for data in a BioInsight Platform Core folder that matches a configured pipeline.
From the Case List, select + New Case.
Select the Import from BioInsight Platform Core option and click the button for Import from BioInsight Platform Core.
Ensure the correct project is selected under Select Project.
Select the tab From folder path.
Use an API to upload data to Connected Insights - Cloud. For more information, refer to .
This section provides instructions for manually uploading a completed analysis for a supported BSSH pipeline.
From the Case List, select + New Case.
Select the Import from BaseSpace Sequence Hub option and click the button for Import from BaseSpace Sequence Hub.
Select the desired analysis to upload and click the button to Continue.
Select the desired test definition to apply to the selected analysis.
To use analysis data from one project (Project A) in another project (Project B), follow these steps:
Ensure that the workgroup has access to Project A.
Note down the folder ID or folder path of the data you want to access in Project B.
In Project B, click on "Data" in the side navigation.
Click on the "Link" menu.
Search by the folder ID or folder path, and click on "Link".
This process will take a few minutes. Once the job is complete, you will be able to access the data from Project A in Project B.
Configuration
This page allows you to customize and optimize Connected Insights to suit your specific workflow and preferences. Here, you can configure various settings related to case management, data visualization, variant analysis, and report generation. By tailoring these options, you ensure that the tool aligns perfectly with your needs, enhancing efficiency and accuracy in your genetic analysis and reporting processes.
To navigate to the configuration page, click the ⚙ icon in the top right corner.
Connected Insights provides default resources, such as test definitions, variant filters, variant flag groups, and report templates. Default resources are marked Provided by Illumina in the Configuration lists.
Default resources are shared with every workgroup and cannot be updated. To customize one, select Copy As New to copy it into your workgroup. Your copy belongs to your workgroup, and later updates to the Illumina default do not change it.
Disease Configuration works differently. Your workgroup has its own disease configuration, which you can edit directly, and an update to the Illumina default never changes it. To take up the current Illumina default, select Restore to Default. See .
The following steps are performed by a lab director user:
Pipeline and QC Configuration
Use Pipeline & QC Configuration to register custom pipelines, upload pipeline and QC configuration YAML files, and edit QC metric thresholds for built-in and custom pipelines.
Configure the pipeline metrics that appear on the report and edit the default values for each metric as follows.
On the top toolbar, select Configuration, and then select the General tab.
In the left-side pane, select Pipeline & QC Configuration.
Select the applicable pipeline. QC metric configurations display to the right of the selection.
DRAGEN Amplicon v4.2
VCF Input Requirement
Connected Insights imports variant calls for the following variant types in the Variant Call File (VCF) file format (v4.1 and later):
Small variants (SNVs, MNVs, and small indels)
Structural variants (SVs)
Copy number variants (CNVs)
RNA fusion variants
Local Installation Guide
The below data flow diagram demonstrates processing workflow involving sequencing data from an Illumina instrument, which is mounted on the NAS (Network-Attached Storage). The DRAGEN software installed on the DRAGEN server v4, accesses the sequencing data from the NAS for secondary analysis. Upon completion of the secondary analysis, Connected Insights - Local reads the analysis output (.vcfs, .FASTQs), processes and annotates it for variant interpretation and writes back the results to the mounted storage.
Below is illustration of network interaction of Connected Insights - Local
Before installing the Connected Insights - Local software, review and complete the prerequisites. For instructions, refer to .
Once the prerequisites are met, complete the fresh installation of Connected Insights - Local software in the sequence below:
Download the Connected Insights - Local software packages. For instructions refer to
Overview
The below data flow diagram demonstrates processing workflow involving sequencing data from an Illumina instrument, which is mounted on the NAS (Network-Attached Storage). The DRAGEN software installed on the DRAGEN server v4, accesses the sequencing data from the NAS for secondary analysis. Upon completion of the secondary analysis, Connected Insights - Local reads the analysis output (.vcfs, .FASTQs), processes and annotates it for variant interpretation and writes back the results to the mounted storage.
Below is illustration of network interaction of Connected Insights - Local
Before installing the Connected Insights - Local software, review and complete the prerequisites. For instructions, refer to .
Once the prerequisites are met, complete the installation of Connected Insights - Local software in the sequence below:
Download the Connected Insights - Local software packages. For instructions refer to
Install the Connected Insights - Local software setup license file and upload the Genome Equivalence sample balance file. For instructions, refer to Install License Files.
Install Annotations, Genome Data Reference and Knowledge Bases packages from Software updates page. For instructions, refer to Software Updates section Install or Update software and dependent package.
Install the Connected Insights - Local software setup license file and upload the Genome Equivalence sample balance file. For instructions, refer to Install License Files.
Install Annotations, Genome Data Reference and Knowledge Bases packages from Software updates page. For instructions, refer to Software Updates section Install or Update software and dependent package.
To add or remove metrics from the report, select the Add to Overview & Report toggle for each metric.
To edit specification limits for applicable metrics, select the Lower Spec Limit or Upper Spec Limit fields and enter the new value. If you cannot edit a value, NA replaces the field.
[Optional] To undo all of your changes, select Reset to Default Values.
Select Save. The updated metrics appear in the Lab QC tab of the case along with the Pass/Fail designations.
A custom pipeline is not recognized out of the box. A custom pipeline configuration YAML describes such a pipeline to Connected Insights in three respects:
What the pipeline is — its display name, version, and (optionally) its Platform Core pipeline id for cloud import.
Where each output file is — the path of every VCF, biomarker, QC-metrics, and visualization file, so Connected Insights can locate and ingest them.
How QC is assessed — the QC metrics, their pass/fail thresholds, and which of them are up-leveled to the case Results and report.
Once the file is authored and uploaded, the custom pipeline behaves like a built-in one: Results, Visualize, and Lab QC are populated, and cases can be ingested by auto upload or manual upload.
In Connected Insights, go to Configuration > General and open Pipeline & QC Configuration in the left navigation.
Select + New to open the Upload Pipeline & QC Configuration panel.
Select attached template to download a starter YAML (the same structure as the Sample Custom Pipeline YAML).
Edit the template to describe the pipeline. For the complete YAML schema, refer to Pipeline and QC Configuration YAML. For file-layout sections (sampleSheet, jointFiles, sampleFiles), refer to .
In the panel, Drag and drop the file or select Select a file, then select Upload. The file must be a single .yaml file no larger than 10 MB.
An existing user-uploaded configuration can be modified with Edit, retrieved with Download, or replaced with Re-upload.
Every uploaded custom pipeline supports local ingestion — manual upload and the Data Uploader (folder monitoring). Optionally supplying the Platform Core pipeline ids (icaPipelineId + icaUrn) also links the pipeline to Platform Core so analyses can be auto-imported or manually imported from there. These paths are not mutually exclusive: a pipeline with Platform Core ids can still be used via the Data Uploader. For details on the pipeline: block and the ICA-id pairing rule, refer to Pipeline and QC Configuration YAML.
Connected Insights validates the file at upload. Uploads are rejected when:
The file is not a .yaml file, is empty, or is not valid YAML.
The pipeline: block is missing, or is missing displayName, name, or version.
Exactly one of icaPipelineId / icaUrn is set (they must be supplied together or not at all).
The threshold: section has no metrics.
A metric is missing its name, is missing its type, or uses a type other than RANGE / BOOLEAN.
Additional considerations:
Declare a variant file. Upload validation does not check the file layout, but a configuration with no jointFiles imports no variants and can cause the analysis import to fail. Always declare at least snvFiles. For file-layout details, refer to Custom Pipeline Configuration.
Unknown keys are ignored, so a typo in a key name does not raise an error — it silently drops that setting. Field spelling should be verified.
Effect on cases: a new or re-uploaded configuration applies to cases ingested after it takes effect; already-ingested cases retain their existing QC/pipeline association until re-ingested.
Test before production: there is no live schema editor — a new file should be validated in a non-production workgroup before rollout.
Edit QC Metrics for a Pipeline
Upload a Custom Pipeline Configuration
Platform Core Linking and Ingestion Paths
In Connected Insights v5.5, the platform Connected Insights integrates with is presented to users as Platform Core (formerly Illumina Connected Analytics / ICA). The YAML field names icaPipelineId and icaUrn, and the urn:ilmn:ica:... identifier, are not renamed and must be kept exactly as shown. The Platform Core pipeline id is the same value previously referred to as the "ICA pipeline id".
Validation and Limitations
RNA splice variants
❗ Imported VCF files must contain at least one sample and be sorted correctly to ensure valid display of results in Connected Insights.
The following sample fields are supported for each variant type:
Sample Field
VCF Fields
Details
Allele Depths
AD
The read support for variants called at this position. Expected as a comma separated list of values for the reference allele followed by each alternate allele.
Total Depth
DP
The total read support for all alleles at this position. Will be calculated as the sum of all allele depths if not provided.
Variant Read Frequency / Variant Allele Frequency
VF (or derived from AD)
The proportion of reads supporting each alternate allele. Expected as a comma separated list of values for each alternate allele. Will be calculated based on allele depths and total depth if not provided.
Genotype
GT¹
¹ The following GT values are interpreted as an absence of the reported variant and are not imported:
0
0/0
Requires SVTYPE=CNV in the INFO field of each VCF entry.
Sample Field
VCF Fields
Details
Fold Change
FC, SM
Estimated fold change for the copy number variant.
Copy Number
CN
Estimated absolute copy number for the copy number variant.
Minor-haplotype Copy Number
MCN
Estimated absolute copy number for the minor-haplotype of a copy number variant. When MCN is zero the copy number variant can be determined to be LOH.
Genotype
(Derived from CN when available)¹
¹ The following GT values are expected given the CN of the variant:
0: The copy number is normal in a region expected to be haploid.
1: The copy number differs from normal in a region expected to be haploid.
0/0: The copy number is normal in a region expected to be diploid.
0/1: The copy number differs from normal and is not a complete loss in a region expected to be diploid.
1/1: The copy number is a complete loss in a region expected to be diploid.
Requires SVTYPE in the INFO field of each VCF entry (DUP, DEL, INS, INV, or BND).
Sample Field
VCF Fields
Details
Paired Reads
PR
The paired read support for variants called at this position. Expected as a comma separated list of values for the reference allele followed by each alternate allele.
Split Reads
SR
The split read support for variants called at this position. Expected as a comma separated list of values for the reference allele followed by each alternate allele.
Supporting Reads
(Derived from PR and SR)
The cumulative read support from split reads and paired reads for variants called at this position.
Total Depth
(Derived from PR and SR)
¹ The following GT values are interpreted as an absence of the reported variant and are not imported:
.
./.
0
0/0
Small variants
Copy number variants
Structural variants and RNA fusion variants
Disease Configuration
The Disease Configuration section can be used to view, add, and edit the key genes associated with disease terms. Disease configurations defined in this section are applied at the workgroup level by default and can be made test-specific by overriding them within a test definition (see Test Definitions).
Disease configuration works differently from other Illumina-provided default resources. Your workgroup has its own disease configuration, which you can edit directly, and an update to the Illumina default never changes it. To refresh your configuration from the current Illumina default, see Restore Disease Configuration to Default Settings.
Add a New Disease Configuration
On the Configuration page, select the General tab.
Select Disease Configuration.
Select +New on the right side of the pane.
In the Disease Name field, type the disease name. This name can be based on a group of diseases or cancer types (for example, Bone).
In the Associated Disease Term(s) field, type a term (for example, Sarcoma).
Select the applicable term from the list that displays.
Make sure that the disease term displays under the field (for example, Sarcoma (SNOMEDCT: 1187396000)).
Under Key Genes, type a gene name in the field. You can enter one gene name per line, or multiple gene names separated by commas. A maximum of 250 names can be entered. The names are case-sensitive.
Select Save.
On the Configuration page, select the General tab.
Select Disease Configuration to expand the list of disease categories.
Select a disease (for example, Bone).
On the right side of the screen, select Edit.
The following table shows the default key genes for select cancer types based on data provided by the Illumina Medical Affairs team.
Cancer Type
Key Genes
The following resources can be used to identify other actionable genes to expand your key genes with:
On the Configuration page, select the General tab.
Select Disease Configuration.
Select ....
Select Restore to Default.
Custom Case Data Upload
Connected Insights accepts metadata information about case, subject, and the sample in CSV format to use it for the case creation, display, and reporting.
Overview of uploading custom case data. Each step is further detailed below:
Download the Case Metadata template file from the Connected Insights Cases page.
Edit the Case Metadata template file to include the desired data.
Upload the Case Metadata file via the Connected Insights user-interface, Data Uploader, or API.
Download the Case Metadata template file
Download the Case Metadata template file from Connected Insights:
Navigate to the Connected Insights Cases page.
Select Upload Case Metadata (top-right corner of the page).
Select Upload CSV.
Click attached template to download the Case Metadata template CSV file.
❗ Starting from the template can help guide formatting, however, any CSV file that follows the content formatting requirements detailed below can be used to upload case metadata. Files containing non-English characters must be encoded as UTF-8.
Edit the case metadata file to add and correctly format the desired information. See the example at the bottom of this subsection:
Open the CSV file with software capable of editing CSV files (for example, a text editor; if using Excel, be cautious of potential unexpected formatting and character additions).
Ensure the following formatting requirements are met:
The first row must contain the headers of the fields to be updated. Each subsequent row contains data.
Each row must contain information in the Sample_ID and
Refer to the following formatting example:
Case metadata can be uploaded in three ways. See below for instructions on each method:
Upload from local storage via Connected Insights user-interface.
Upload from local storage via the Data Uploader.
Upload from local storage via an API.
Upload Case Metadata files via the Connected Insights user-interface
Navigate to the Connected Insights Cases page.
Select Upload Case Metadata (top-right corner of the page).
Select Upload CSV.
Upload the file.
Upload Case Metadata files via the Data Uploader
Install the Data Uploader in a location with access to the Case Metadata file (refer to ). Proceed to the next step if it is already installed.
Ensure the Case Metadata file is in a location accessible by the Data Uploader.
Create a new pipeline or edit an existing pipeline and set the Choose a folder to monitor for case metadata (optional) field to the file path of the directory containing the Case Metadata file (refer to ).
Next time the Data Uploader daemon runs, the file will be ingested (may be ~10-15 minutes).
Upload Case Metadata files via an API
Refer to , subsection "Upload Metadata files via an API".
Update case metadata of an existing case
To update an existing case, use the Sample_ID and Case_ID of the existing case and add updated information in additional fields. It may be useful to use an API in the Case Metadata section of the API page to retrieve the existing case metadata for a case (for details on using APIs, see ). The overwriting and preservation logic is as follows:
Overwriting Logic: If data conflicts, new data will overwrite existing data. For the Tags field, new data will be added as an additional tag and will not overwrite any existing tag(s).
Preservation Logic: Existing data will not be overwritten if the data for the field is the same as existing or no data is entered. Additionally, fields not included in the Case Metadata file will not be affected (for example, if the Case Metadata file does not include the Date of Birth field, existing data in that field will not be updated).
Create a new case by uploading a Case Metadata file
To create a new case, use a new Sample_ID and a new Case_ID and upload the Case Metadata file before ingestion of molecular data (for example, VCF files or other secondary analysis output files).
In order for the molecular data to associate with the correct case metadata, the Sample_ID and Case_ID must match (for example, the Sample_ID value used in the Case Metadata file should match the Sample_ID value in the sample sheet).
Cases created this way will have a Status column value of Awaiting Molecular Data and a Workflow Name column value of N/A in the Case List on the Cases page. The Status and Workflow Name will update after completely uploading molecular data.
The Case Metadata Uploads page displays Case Metadata file upload history and error messages:
Navigate to the Connected Insights Cases page.
Select Upload Case Metadata (top-right corner of the page).
Select View Past Uploads.
The table displays Case Metadata file upload history, status, and details.
Customizations
In Connected Insights, use the default report template or introduce customizations to configure the reports according to your specifications. For more information on using the default report template, refer to Report Templates.
The following applications can be used to customize the report:
LibreOffice v7.5 or later
Illumina recommends using LibreOffice to customize the report. For more information on the customizations that can be done with LibreOffice, refer to the writer guide on the LibreOffice website.
Carbone Studio
Carbone studio and its documentation are updated regularly. Please ensure any command used in an ODT is compatible with Carbone version 4.25.5 or earlier.
Use LibreOffice or an equivalent ODT editor to do common report customizations, including:
Using rich text formatting in the Interpretation Summary.
Updating the date and time format.
Adjusting the font size and color.
Adding, reordering, and removing sections.
Inserting or removing information, such as single or aggregate data.
Use Configuration to do report customizations, including:
Report Methodology.
Add laboratory logo.
Change report name.
Add laboratory address.
Include or exclude report sections.
Default and custom templates created in version ICI version 5.5 will use rich text formatting by default.
To allow migrated or customized reports generated in ICI version 5.4 or earlier to use rich text formatting, open the template in the ODT editor.
Throughout multiple places in the template, find the following code that generates the report updated time text: formatD('YYYY-MM-DD HH:mm')}
Replace YYYY-MM-DD HH:mm with the desired format (for example,YYYY-MM-DD).
Open the template in the ODT editor.
Change the font color and style using the font characteristics function for an individual text section.
❗ Color and size are changeable, but the font style and type cannot be altered.
[Optional] If you must update the font color and size for the whole document, adjust the paragraph style.
Open template in the ODT editor.
Identify the location in the report where the new section is being placed.
Enter a paragraph break between the sections where the copied section is being placed.
Insert new content (table, text box, plain text) in the created section.
Open template in the ODT editor.
Identify the sections that must be rearranged.
Select and copy all relevant text or tables in that section.
❗ The sections of the template primarily consist of tables. Make sure that the entire table is selected when reordering a section. If you do not,then formatting issues can occur.
Enter a paragraph break between the sections where the copied section is being placed.
Open the template in the ODT editor.
Select the section of the report that you want to remove.
❗ The sections of the template primarily consist of tables. To remove the section, make sure that the entire table is selected. If you do not, then formatting issues can occur.
Delete the section.
Insert single data (for example, Methodology) or aggregate data as follows.
Open the template in the ODT editor.
Place the cursor where you want the new information to appear.
Add the applicable code or text used to generate new content or enter free text that you want to appear in all of the generated reports. For example, enter {d.createdDate:formatD('YYYY-MM-DD')} for the date the report was created or {d.reportData.therapies[]:aggCount} to add a count for the number of therapy assertions included in the report.
Open the template in the ODT editor.
Search for the following line (use Ctrl+F to search): {$lbl.pertinentNegativesReported:ifEM():show('Pertinent negatives reported')}
Use the ODT editor to select and delete the table containing the searched for text above.
Open the template in the ODT editor.
Select each paragraph style and adjust the page size to the required size needed for printing.
Basic report methodology descriptions can be added within a test definition. For more complex descriptions or to add a gene list, follow steps 2-8.
Open the template in the ODT editor.
Search for and delete the following line (use Ctrl+F to search):{$methodology:convCRLF()}
Replace line with desired methodology text used for the specified workflow.
The version 2 template can conditionally display findings at the beginning of the report based on the Tier/Actionability level. The default shows all levels in this section. To only display Tier I and Tier II (or equivalent).
Open the template in the ODT editor.
Search for the following line (use Ctrl+F to search):classification.actionabilityOrder:ifGTE(7.0)
Change 7.0 to 5.0 in the instances of the line.
Report Automation
User-defined automation can add assertions from your knowledge base and/or other knowledge bases automatically to a draft report.
Automation only applies to cases using test definitions with automation turned on during case ingestion. Cases that were ingested before automation was turned on will not have assertions automatically added to their reports.
Automated assertions can be added for all types of variants with assertions in the supported knowledge bases, including the genome-wide biomarkers TMB, MSI, and HRD. Automation configuration settings determine which biomarkers are included in automation and which assertions for those biomarkers are added to the report.
You can also edit or remove assertions that were automatically added before approving a report. For more information, refer to Manage Assertions.
Prerequisites
Before enabling automation, make sure that the following information is set up:
Key genes and TMB, MSI, and HRD thresholds to limit reporting to certain biomarker scores. For more information, refer to Disease Configuration.
Preferred transcripts to limit reporting to preferred transcripts. For more information, refer to .
Default filters to limit the variants being reported. For more information on creating variant filters and adding them as default filters in a test definition, refer to and .
Select either of the following sets of variants to be reported:
Variants that appear in a default filter and key genes (i.e., Key Findings of the Overview page).
Variants that appear in a default filter.
❗ Variants that are marked as an artifact will be excluded from automated reporting regardless of appearing in a filter, refer to
The following are not reported:
Any variant with a likely benign, benign, or likely neutral biological classification in a knowledge base.
Any variant with no biological classification in any knowledge base (see below) or classified as VUS, likely benign, or benign by the tool.
TMB, MSI, and HRD with an unknown status.
Off-label CKB or CIViC assertions whose disease is in a different cancer category (solid tumor or hematologic) than the case disease. These assertions have no assigned tier. For more information, refer to .
Max number of variants eligible for automated reporting:
If variant selection is Variants that appear in default filter and key genes, only the first 300 variants with assertions in your default filters that are in key genes are eligible for automated reporting.
If variant selection is Variants that appear in default filter, only the first 300 variants with assertions in your default filters are eligible for automated reporting.
You can select KBs you want to report assertions from. The following KBs are selectable:
My Knowledge Base
OncoKB
CKB
CIViC
You can also reorder KBs to prioritize assertions. Select and hold the icon to the left of the KB and move it up or down to reorder the KB list. Connected Insights uses the following information to prioritize which KB is used to report findings:
Order in the KB list (for example, if you move My Knowledge Base above CKB in the KB list, then My Knowledge Base is prioritized)
Case disease and assertion disease match (for example, if CKB is below My Knowledge Base in the KB list, but the case disease matches the CKB assertion disease, then CKB is prioritized)
Only assertions with the following actionability and biological classifications are eligible for automated reporting from each KB:
Knowledge Base
Eligible
Not eligible
Select Yes or No for the following questions:
Have been reported previously? For an OncoKB, CKB, or CIViC assertion to be automatically reported, it must have been previously reported. My Knowledge Base assertions and Oncogenicity Prediction are always considered for automated reporting, regardless of this setting. When this option is set to Yes, a previously reported OncoKB, CKB, or CIViC assertion is reported even if its classification is not listed as eligible in (for example, a Tier 2D assertion from CKB).
Therapeutic assertions have the highest classification? Only therapeutic assertions with the highest classification for the variant will be reported.
Therapeutic assertions indicate response or resistance? For example, "Predicted responsive" and "No impact" will not be reported.
Select approval authorities that must support the assertion for it to be reported:
FDA
EMA
NCCN
ESMO
TGA
Select the distance from that clinical trial must be recruiting in:
Distance in miles
Distance in kilometers
Select locations that clinical trials are recruiting in (will be used if a match location or distance is not provided):
EU
United States
Specific countries
The input to the logic is assertions for the biomarker that can be found when viewing them from the . From there, we filter assertions according to the configuration defined above, group them, prioritize assertions within each group, and report top result per group. When both an RNA fusion variant and a DNA structural variant are present for the same gene fusion, the RNA fusion variant is prioritized.
Prerequisites
Before starting the installation or update of Connected Insights - Local software, make sure that the following prerequisites are met.
1. DRAGEN Server v4 Configuration
Ensure the DRAGEN Server v4 configuration is as below:
DRAGEN server v4 with Oracle Linux 8. The preferred version of Oracle Linux 8 is preinstalled on the DRAGEN server v4. The DRAGEN server v4 software pipeline is also included.
DRAGEN server v4 setup shall be completed following the Illumina DRAGEN Server v4 Installation Guide (document #200015717). For more information, refer to Illumina DRAGEN Server v4 Installation Guide on the Illumina support site.
DRAGEN server v4 is idle during installation or update of Illumina Connected Insights - Local software.
If you are setting the hostname for a DRAGEN server v4, ensure the server has FQDN (Fully Qualified Domain Name) (Example: If the server's hostname is onpremtest and domain is abcd.com, then the server shall be registered as onpremtest.abcd.com in your organization's DNS(Domain Name Server)). If your organization does not have a DNS server, then ensure to setup the server with a static IP and the /etc/resolv.conf file shall be an empty file with no contents in it. Once set up, this configuration will be required for installation and cannot be changed throughout the software's usage. If it changes in the between, then application may not behave as intended and could possibly throw errors. Thus, ensure to finalize the configuration i.e. with FQDN (Or) with Static IP before starting the installation process mentioned in this document.
You must have root or sudo privileges to run installation commands. To access the DRAGEN server v4 and run commands from the command-line interface, you should be familiar with the Linux commands used to install the software and config file (in case of upgrade from current installed version to the new version).
An external storage drive must be connected to the DRAGEN server v4 directly or through the network. If an external storage drive meeting the requirements below is not mounted already, refer to the section for details on how to mount an external storage drive.
Below are the minimum requirements of the external storage which shall be mounted on the DRAGEN server v4.
Configured with CIFS/SMB or NFS protocols only. Recommended to use NFS v4.0 and above or CIFS/SMB v3.1 and above protocol for data security. Illumina also recommends an encrypted data storage drive to safeguard your data.
100 gigabytes (GB) of available storage space.
Recommended: 10 Gb (Gigabits) per second of network speed.
The external storage drive is used by Connected Insights - Local software to read inputs and write outputs.
For inputs, Connected Insights - Local software reads secondary analysis data (for example, VCF and BAM files).
For outputs, Connected Insights - Local software writes analysis output data and backup files (for example, annotated VCF files, visualization files, and PDF report files).
In Illumina's welcome email or customer notification email, all the links to download the packages shall be accessible. If you are unable to reach to any of the sites, please check with your IT to whitelist the URLs. The example URLs are https://support.illumina.com/downloads/
A personal computer which has a remote access client tool to connect to the DRAGEN server v4 to run the commands and a browser to access Connected Insights - Local software application after software is installed.
The genotype of the sample at the given position.
The genotype of the sample at the given position.
The total reads for all alleles called at this position.
Variant Read Frequency / Variant Allele Frequency
(Derived from PR and SR)
The proportion of reads supporting each alternate allele. Calculated based on supporting reads and total depth.
Genotype
GT¹ (or derived from PR and SR)
The genotype of the sample at the given position.
Changing the paper size.
Update Report Methodology and/or add Gene List.
Show Findings by Tier
Make sure that all of the information is present and formatted correctly.
Paste the copied section to the new location.
Make sure that all of the information is present and formatted correctly.
Delete the original section.
Repeat steps 1–7 for any other sections that must be reordered.
To add a gene list, insert a row below Assay Description.
Insert the desired section header.
Insert the gene list below the header.
Make sure that all of the information is present and formatted correctly.
Search for the following line (use Ctrl+F to search):classification.actionabilityOrder>6.9999
Change 6.9999 to 4.9999 in all instances of the line.
2. Root or sudoer privileges to access DRAGEN Server v4
3. Mounted External Storage Drive with read/write access. It is highly recommended you create a user on the mounted storage which has read and write permission and not use a root user.
4. Illumina provided package URLs are accessible in your network
To add a disease term, type the term (for example, Sarcoma) in the Associated Disease Term(s) field.
Select the applicable term from the list that displays (e.g., Sarcoma (SNOMEDCT: 1187396000))
To delete a disease term, select the X next to it.
Select Save to accept any changes.
Update the key gene names as follows.
For Key Genes, enter a gene name in the field.
You can enter one gene symbol per line, or multiple gene names separated by commas. A maximum of 250 names can be entered. The names are case-sensitive.
Select Save to accept any changes.
Update genome wide thresholds as follows.
For Genome Wide Thresholds, enter values for the following Tumor Mutational Burden (mut/MB) thresholds:
TMB-Low
TMB-High
Enter values for the following Microsatellite Instability thresholds, depending on the measurement used (i.e., % unstable sites or sumJSD):
MS-Stable
MSI-High
Enter values for the following Homologous Recombination Deficiency thresholds, depending on the algorithm used (i.e., Myriad or DRAGEN):
Includes a therapy of interest? Only therapeutic assertions that contain a therapy of interest will be reported. This option is available after a list of therapies of interest has been uploaded for your workgroup via the Connected Insights API.
Fields that require a date must be in yyyy-mm-dd format.
Tumor_Type values must be the SNOMEDCT ID for the disease.
The SNOMEDCT ID can be found by navigating to an existing case and searching for the disease in the Case Details or assertion form. It can also be found by navigating to the Configuration page, Disease Configuration section, clicking New +, then searching in Associated Disease Term(s). Lastly, the ID can also be found by using the International Edition browser at the SNOMED International SNOMED CT Browser website.
When the tumor type is unknown, SNOMETCT ID 363346000 ("Malignant neoplastic disease") or 255052006 ("Malignant tumor of unknown origin") can be used. However, the accuracy of actionability will be higher the more specific the tumor type provided is.
All other columns must follow formatting requirements specified in the Case Metadata template file.
Data in columns for fields defined in the Custom Case Data Definition section of the Configuration page must match the formatting requirements based on whether the data type is text, a number, or a date (for details, refer to Custom Case Data Definition).
Once all rows are added, save.
If errors occur, the Details column will state this and provide a link to download a copy of the Case Metadata file annotated with error messages for each row.
Edit the Case Metadata file
Upload Case Metadata files
Update existing cases or create new cases by uploading Case Metadata
View Case Metadata upload tracking and error messages
Sample_ID,Case_ID,Sample_Type,Tumor_Type,Date of Birth,Example Case Meta,Example Subject Meta,Example Sample Meta
SampleId1,Case001,DNA,707405009,2011-01-21,Case Data 01,Subject Data 01,Sample Data 01
SampleId2,Case001,RNA,707405009,2011-01-21,Case Data 01,Subject Data 01,Sample Data 02
SampleId3,Case002,DNA,707405009,2012-02-22,Case Data 02,Subject Data 02,Sample Data 03
## Sample custom pipeline + QC configuration for Illumina Connected Insights v5.5.
##
## Download the template from Connected Insights (Configuration > General > Pipeline & QC Configuration
## > + New > "attached template"), edit the values for your own pipeline, then upload it
## to register a new pipeline together with its QC thresholds.
##
## The file has three parts:
## 1. pipeline: registration metadata for the pipeline.
## 2. file-layout sections (sampleSheet / jointFiles / sampleFiles) describing where
## each analysis output file is located. jointFiles is required to import variants;
## sampleSheet and sampleFiles are optional. Stored as-is.
## 3. threshold: the QC metrics used to build the pipeline's QC thresholds.
##
## Path placeholders such as {sampleId.DNA}, {sampleId.RNA} and {prefix} are resolved
## per case at ingestion time. A trailing ", Optional" marks a file that may be absent.
pipeline:
# required - human-readable name shown in Connected Insights (use a report-friendly name)
displayName: Sample Custom Pipeline v1.0.0
# required - internal pipeline name (alphanumeric, underscore, dash only)
name: Sample_Custom_Pipeline
# required - pipeline version
version: 1.0.0
# optional - Platform Core pipeline id and urn. Provide BOTH to link the pipeline
# to Platform Core for cloud import. Omit BOTH when Platform Core linking is not
# needed. Either way, the pipeline supports Data Uploader and manual local upload.
# Supplying only one is a validation error.
icaPipelineId: 00000000-0000-0000-0000-000000000000
icaUrn: urn:ilmn:ica:pipeline:00000000-0000-0000-0000-000000000000
# optional - only "WGS" is meaningful here: it selects the DRAGEN-style .csv QC parser.
# Omit this field for a TSO 500-style MetricsOutput.tsv. Any other value is treated as TSO 500 style.
type: WGS
# optional - glob pattern(s) whose presence signals a completed run.
# NOTE: successMarkerFile / failureMarkerFile are used ONLY by the Connected Insights Data Uploader
# when it monitors a local folder for auto-upload. They are ignored for Platform Core
# cloud analysis, which reports completion via the platform's own status events.
successMarkerFile:
- "{prefix}/{prefix}.time_metrics.csv, Optional"
- "{sampleId.DNA}/{sampleId.DNA}.time_metrics.csv, Optional"
- "{sampleId.RNA}/{sampleId.RNA}.time_metrics.csv, Optional"
# optional - file name whose presence signals a failed run (Data Uploader only)
failureMarkerFile: "failure.txt"
## ── File-layout sections ─────────────────────────────────────────────────────────
## These describe where the analysis outputs are located and are stored verbatim.
## jointFiles is required in practice: it declares the variant files (SNV/CNV/SV/fusion),
## and an analysis imported without it brings in no variants and may fail.
## sampleSheet and sampleFiles are optional. Each file slot accepts an ordered list of
## candidate glob patterns; Connected Insights ingests the first pattern that matches.
sampleSheet:
filePath: SampleSheet.csv
columnAliases:
sampleType: Sample_Type
sampleId: Sample_ID
pairId: PairID
dataHeaderRowMarker:
- "[Sample_Data]"
- "[Cloud_Sample_Data]"
jointFiles:
snvFiles:
- "{sampleId.DNA}/{sampleId.DNA}.hard-filtered.vcf.gz, Optional"
- "{prefix}/{prefix}.hard-filtered.vcf.gz, Optional"
cnvFiles:
- "{sampleId.DNA}/{sampleId.DNA}.cnv.vcf.gz, Optional"
- "{prefix}/{prefix}.cnv.vcf.gz, Optional"
svFiles:
- "{sampleId.DNA}/{sampleId.DNA}.sv.vcf.gz, Optional"
- "{prefix}/{prefix}.sv.vcf.gz, Optional"
rnaFusionFiles:
- "{sampleId.RNA}/{sampleId.RNA}.fusion_candidates.vcf.gz, Optional"
- "{prefix}/{prefix}.fusion_candidates.vcf.gz, Optional"
metricsQCFile:
- "{sampleId.DNA}/{sampleId.DNA}.mapping_metrics.csv, Optional"
- "{sampleId.DNA}/{sampleId.DNA}.wgs_coverage_metrics.csv, Optional"
- "{sampleId.DNA}/{sampleId.DNA}.vc_metrics.csv, Optional"
- "{sampleId.RNA}/{sampleId.RNA}.mapping_metrics.csv, Optional"
- "{prefix}/{prefix}.mapping_metrics.csv, Optional"
- "{prefix}/{prefix}.wgs_coverage_metrics.csv, Optional"
sampleFiles:
tumorPairId:
visualizationFiles:
coverageBedFiles:
- "{sampleId.DNA}/{sampleId.DNA}.target_bed_read_cov_report.bed, Optional"
- "{prefix}/{prefix}.target_bed_read_cov_report.bed, Optional"
coverageFiles:
- "{sampleId.DNA}/{sampleId.DNA}.target.counts.gc-corrected.gz, Optional"
- "{prefix}/{prefix}.target.counts.gc-corrected.gz, Optional"
alignmentFiles:
dnaBamFile:
- "{sampleId.DNA}/{sampleId.DNA}_tumor.bam, Optional"
- "{sampleId.DNA}/{sampleId.DNA}_tumor.cram, Optional"
- "{prefix}/{prefix}_tumor.bam, Optional"
dnaBaiFile:
- "{sampleId.DNA}/{sampleId.DNA}_tumor.bam.bai, Optional"
- "{sampleId.DNA}/{sampleId.DNA}_tumor.cram.crai, Optional"
- "{prefix}/{prefix}_tumor.bam.bai, Optional"
rnaBamFile:
- "{sampleId.RNA}/{sampleId.RNA}.bam, Optional"
- "{prefix}/{prefix}.bam, Optional"
rnaBaiFile:
- "{sampleId.RNA}/{sampleId.RNA}.bam.bai, Optional"
- "{prefix}/{prefix}.bam.bai, Optional"
## ── Threshold / QC metrics ───────────────────────────────────────────────────────
## At least one metric is required. Each entry describes one QC metric completely:
## - name: the display label for the metric.
## - group: display section/group shown in Lab QC and Results.
## - raw: maps to the source metric. Default when display and source
## sections match:
## group: "Sample Metrics"
## raw: "PERCENT_Q30"
## Omit `raw` when display `name` equals the source metric name.
## Optional nested form when UI `group` should differ from the
## metrics-file section, or to disambiguate duplicate metric names:
## raw:
## name: lab/DRAGEN metric name
## group: source section in the metrics file
## - category: overrides defaultCategory for this metric (RUN_QC | SAMPLE_QC | GENE_QC).
## - type: RANGE | BOOLEAN (RANGE covers numeric, percentage and count metrics).
## - isLowerEnable / isUpperEnable: which bound(s) are evaluated.
## - lowerValue / upperValue: the bound value(s), as strings.
## - displayInOverView: up-level this metric to the case Results tab and report.
threshold:
defaultCategory: SAMPLE_QC
metrics:
# Display name differs from the lab metric name -> map it with the raw string shorthand.
- name: "Total Reads (Count)"
group: "MAPPING/ALIGNING SUMMARY"
raw: "Total input reads"
displayInOverView: true
type: RANGE
isLowerEnable: true
isUpperEnable: false
lowerValue: "1000000"
- name: "Estimated Sample Contamination"
group: "MAPPING/ALIGNING SUMMARY"
raw: "Estimated sample contamination"
displayInOverView: true
type: RANGE
isLowerEnable: false
isUpperEnable: true
upperValue: "0.05"
# Display name and lab name already match, so raw can be omitted entirely.
- name: "Average alignment coverage over target region"
group: "COVERAGE SUMMARY"
displayInOverView: true
type: RANGE
isLowerEnable: true
isUpperEnable: false
lowerValue: "100"
# Same metric shown under two display groups; raw.group disambiguates the source
# group each one pulls from (tumor vs normal).
- name: "Q30 Bases (Percentage)"
group: "TUMOR MAPPING/ALIGNING SUMMARY (TUMOR-NORMAL)"
raw:
name: "Q30 bases (Percentage)"
group: "TUMOR MAPPING/ALIGNING SUMMARY"
type: RANGE
isLowerEnable: true
isUpperEnable: false
lowerValue: "80"
- name: "Q30 Bases (Percentage)"
group: "NORMAL MAPPING/ALIGNING SUMMARY (TUMOR-NORMAL)"
raw:
name: "Q30 bases (Percentage)"
group: "NORMAL MAPPING/ALIGNING SUMMARY"
type: RANGE
isLowerEnable: true
isUpperEnable: false
lowerValue: "80"
# GENE_QC "Gene Coverage" metric. Its lowerValue sets the reference line on the
# per-gene coverage plot in Results page.
- name: "Gene Coverage"
group: "Coverage Plot"
category: GENE_QC
raw: "GENE_COVERAGE"
type: RANGE
isLowerEnable: true
isUpperEnable: false
lowerValue: "900"
disableDisplayInOverView: true
Test Definitions allow you to specify default parameters that apply to cases when they are ingested. Make sure you have test components created, then create a test definition as follows.
On the top toolbar, select Configuration.
Select the Test Definitions tab.
Select New.
In the right-hand pane, enter a test name in the Test Name field.
Choose a human reference genome.
[Optional] Select one or more custom annotations (must be compatible for the selected human reference genome).
[Optional] Choose a virtual panel option to restrict variant display to a specific set of genes.
[Optional] Configure one or more disease configurations for this test. Workgroup-level are available by default and can be overridden to create test-specific configurations. Test specific configurations are applied to all cases processed using the test definition.
Select Edit to view and edit disease configuration.
Select New to add disease configuration.
Select Delete or Delete All to delete disease configuration(s).
Choose a .
Choose one or more variant filters. These filters display in the as locked filters.
Choose a .
[Optional] Select the Turn on Report Automation toggle to enable or disable the automated addition of assertions to reports using this test definition. For more information, refer to .
[Optional] Enter version comments as applicable.
[Optional] Enter a test methodology to be included on the PDF report.
Select Save. The new test definition appears on the left-hand accordion menu. Select a test definition to preview and access the following actions:
Select Edit to edit a test definition.
Select Copy As New to copy the definition as a template for a new test definition.
Select Archive to hide the test definition from the PDF report.
Connected Insights provides the following default tests, marked Provided by Illumina, to assist in preliminary test implementation. Default tests cannot be updated; select Copy As New to copy one into your workgroup and adjust it. See .
Test
Genome Build
Default Filters
Virtual Panel (see note 1)
Test-Specific Disease Groups (see note 2)
The default filters are described in .
Notes
Virtual panel design. A virtual panel restricts the variants that are ingested to a set of genes. Structural variants that do not overlap one of those genes are not ingested.
Disease key genes — The genes are the key genes of the disease group that applies to the case, taken from the test-specific disease groups of the test.
170-gene list — The genes are a fixed list of 170 genes from the TruSight Tumor 170 gene set.
Tests marked — have no virtual panel.
Select Reset to Default to restore disease configurations to the workgroup-level disease configurations.
GRCh37
All Demo, SNV Demo, SV Demo, CNV Demo
—
—
Pillar oncoReveal Multi-Cancer DNA and RNA Fusion - Virtual Panel
Test-specific disease group design. A test-specific disease group applies to cases processed with that test, and replaces the workgroup-level disease configuration for them. Each test with test-specific disease groups follows the same design:
The key genes of a disease group are the workgroup-level key genes for that group, limited to the genes the assay covers. Genes that the assay does not cover are removed, and no genes are added from other sources.
Each group keeps the name, display order, and associated disease terms of the workgroup-level group.
Solid tumor assays include the solid tumor disease groups (Bone, Breast, CNS, Colorectal, Lung, Melanoma, Ovarian, Prostate, Thyroid, and Uterine And Cervical). Heme assays include the AML disease group only.
Every test also includes a Default all Unspecified Diseases group. Its key genes are all the genes the assay covers, and it applies to diseases that no other group matches.
TMB, MSI, and HRD thresholds are not set in the test-specific disease groups. Cases use the thresholds of the workgroup-level disease configuration.
Connected Insights - Cloud Data Uploader tool supports uploading of VCF files and analysis output for user-provided machines and storage devices and can be downloaded from Configuration -> Data Upload section of the application.
Connected Insights - Local Connected Insights - Local includes the Data Uploader tool that is installed on the DRAGEN server v4 as part of the Connected Insights - Local installation and reads secondary analysis results from the external storage drive that is configured. The Data Uploader tool is detected and identified as Default-CLI-Installation in the Data Upload section of the Configuration page. If Default-CLI-Installation does not show as Online, refer to Software Errors and Corrective Actions.
The Data Uploader logs can be found at /staging/ici/logs/tss-cli/. With Connected Insights - Local, these logs can also be downloaded from the Activity tab in the Case Details pane on the Cases page. For more information, refer to Case Details.
The secondary analysis input logs can be found /ExternalStorageDriveMountPath>/d53e4b2d-0428-4b3e-92bf-955f7153c360/d53e4b2d-0428-4b3e-92bf-955f7153c360/upload-logs/<DataUploadConfiguredMonitoringLocation>/<SecondaryAnalysisRunFolder>/run_<Timestamp>.json
Setup (Connected Insights - Cloud Only)
This section identifies the requirements for uploading data from user storage, enabling ingestion from the network drive, and adding an existing pipeline. For Connected Insights - Cloud, this section also covers how to download and install the Data Uploader tool and has instructions for generating an API key.
To upload data from user storage, the following requirements must be met:
Make sure you have Java 8 or a newer version installed on your computer. You can check by opening a terminal or command prompt and typing java -version. If you don't have it installed, you can download and install it from the official Java website.
Data Uploader with Java Virtual Machine (JVM) 8+ that is compatible with Mac, Windows, or Linux CentOS.
An API key and access to the workgroup to which the data is uploaded. The API key comes with the installer. To generate a newAPI key, refer to .
Administrator access on the computer where Data Uploader is installed.
Before you begin, perform the following actions:
Create custom case data.
Create test components.
Create test definitions.
For more information, refer to .
Download and install the Data Uploader tool as follows.
In Connected Insights, select the gear at the top right of the page.
In the General tab, under Data Upload, select the From Local Storage tab.
Under Download and Launch the Data Uploader, select the storage device operating system from the drop-down list.
Select Download Data Uploader. A progress bar displays during the download.
If user has admin access, ici-uploader is installed as a system service. User can stop the ici-uploader (running as system service) using the following command ici-uploader stop-service command for Mac/Linux or with the ici-uploader.exe stop-service command for Windows. User can start the uploader via ici-uploader start-service.
❗If data-uploader is already running as system-service, and user also runs the uploader manually via ici-uploader start-daemon, it may cause issues. User must stop already running data-uploader with ici-uploader stop-service before starting uploader manually.
Before installing the new version of the Data Uploader, the user must first uninstall the existing version. If they attempt to install without uninstalling, they will encounter the following error.
❗ [ERROR ici_uploader] Unable to start ICI Data Uploader because a daemon is already running ...
To uninstall, run the following command:
a. [Windows] Start the command prompt as an administrator and run ici-uploader.exe uninstall.
❗ For windows environment, user must be in the installation directory to uninstall the application. When prompted with message this operation will remove all contents from the folder. Are you sure to proceed?
If you are using Linux, automatic case upload is already enabled and this section is not applicable. For Mac, contact Illumina Technical Support. For Windows, run the "illumina ICI Auto Launch Service" to make the network drive available for Data Uploader as follows:
When the Data Uploader is running, open the Services application from the Start menu and locate the "illumina ICI Auto Launch Service".
Right-click "illumina ICI Auto Launch Service" and select Properties from the drop-down menu.
In Properties, select the Log On tab and enter your account ID and password. Confirm the password.
Select the General tab.
After completing these steps, the Data Uploader details are visible in Connected Insights.
If the computer where the Data uploader is installed is behind a proxy server, then Uploader proxy setting must be enabled. Before you install the Data Uploader, run the following command on the terminal (Linux).
❗ export JDK_JAVA_OPTIONS='-Dhttps.proxyHost=<IP address of the proxy server e.g. 1.2.3.4> -Dhttps.proxyPort=<Port of the proxy server e.g. 8080>'
For Windows, it can be set via following command.
❗ setx JDK_JAVA_OPTIONS "-Dhttps.proxyHost=<IP Address of the proxy server e.g. 1.2.3.4> -Dhttps.proxyPort=<Port of the proxy server e.g. 8080>"
Remember to replace and with the actual values you need.
Input secondary analysis output folder to be monitored.
In Configuration Settings, select the radio button next to Choose compatible pipeline from catalog, refer to .
Select a pipeline from the drop-down list (for example, DRAGEN TruSight Oncology 500 Analysis Software v2.5.2).
❗ When running the DRAGEN Somatic Whole Genome pipeline in the Tumor-Only mode, you must set --output-file-prefix to match sample-id (the RGSM of the sample in the FASTQ list) of the run.
If you are using Data Uploader for the first time, then you must generate a new API key. All Data Uploader operations require an API key for authentication. The Data Uploader bundle can be downloaded with an autogenerated API key. If the bundle includes an API key, skip this section.
❗ If you are installing Data Uploader on multiple machines, manually create and track your API key by running the following auto-updating and manual run command:
ici-uploader configure --api-key={apiKey} Make sure that the API key is within single quotation marks (for example,'{SYSTEM_API-KEY}')
In Connected Insights, select Manage API Keys from the Account drop-down menu.
Select Generate.
Enter a name for the API key.
To generate a global API key, select All workgroups and roles.
The following information is applicable to both Connected Insights - Local and Connected Insights - Cloud.
Create each configuration as follows.
Input the {monitoring location} for secondary analysis output.
The monitoring location is the full path to the location where secondary analysis output data is deposited in user's storage.
For example:
/rest/of/storage path/{monitoring location}/{runFolder}/{sample folders, sample sheet, inputs, tumor_fastq_list.csv}
Perform any of the following actions below:
Edit You can modify the pipeline configuration by clicking "Edit" against a configuration.
Note: The change in configuration does not impact the Case that are already ingested.
Delete You can delete the pipeline configuration by clicking "Delete" against a configuration.
Requeue (Connected Insights - Local) You can resume or reupload a secondary analysis output folder by clicking "Requeque" against a configuration. Upon clicking "Requeue", application will re-attempt to create the case from the run folders which previoulsy failed due to one of the below errors:
SampleSheet validation error
Case ingestion has stopped due to zero GEs balance
Case ingestion has stopped due to low space on external mounted storage or /staging
Upload an analysis output by running the following command:
ici-uploader analysis upload --folder={path-to-analysis} --pipeline-name={pipelineName}
--folder {path-to-analysis} — The absolute path to the analysis output to upload into Connected Insights. This folder contains the sample sheet.
--pipeline-name={pipelinename} — The name of the pipeline created in Connected Insights to apply to cases uploaded from this analysis. Pipeline names must include only letters, numbers, underscores, and dashes. The name cannot include spaces or special characters.
For more information on custom case data files, refer to .
The following table shows the approximate time it takes for example datasets to upload and receive the Ready for Interpretation status. The duration is evaluated by analyzing a batch of samples ingested through the Connected Insights tertiary analysis in conjunction with the DRAGEN secondary analysis performed on similar datasets on the DRAGEN server v4.
Copy the installation directory of the storage device. Make sure that Data Uploader is in a location that has access to your secondary analysis output folder.
Use the following tar command to extract the files: Replace {ici uploader script} with the applicable file name. tar xvzf {ici uploader script}.tar.gz
❗ Extracting files can vary depending on the operating system. Most terminals support the tar command. For Windows, you can use a zip file extraction application (eg, 7-Zip) to extract the content of the tar folder.
Make sure that the files in the following table are in the unzipped Data Uploader file.
Component
Description
ici-uploader-daemon.jar
Used to run Data Uploader as a daemon.
ici-uploader.jar
Used to run Data Uploader on demand.
uploader-config.json
This file contains the configuration for Data Uploader to allow it to connect to Connected Insights.
Install Data Uploader as follows:
a. [Windows] Start the command prompt as an administrator and run ici-uploader.exe install.
❗ For windows environment, user must be in the installation directory to install/uninstall the application. ici-uploader.exe start-daemon
b. [Mac and Linux] Run the following command on the terminal ./ici-uploader install
❗ If the installation fails, start the Connected Insights installation manually using ici-uploader start-daemon
c. Follow any prompts in Data Uploader. d. Enter your API key and press Enter.
❗ If you did not download with an auto-generated API key, provide one to the installer when prompted. If you are installing Data Uploader for the first time, you must generate an API key. For more information, refer to .
e. Under Define and Monitor Data Uploads, make sure that your machine displays. Data Uploader is now set up for auto and manual ingestion. You can also change the name of the installed Data Uploader by selecting ... and Edit Server Name . If you must download any logs associated with the Data Uploader from the last 24 hours, select ... and Download Logs.
❗ If the user is not a system administrator, the daemon can be started with the ici-uploader start-daemon command for Mac/Linux or with the ici-uploader.exe start-daemon command for Windows. This command does not run the process as a system service and requires you to start and stop the service manually. To stop the daemon, run the command ici-uploader stop-daemon
After Data Uploader has been downloaded and installed on the user storage, you can set up configurations for the tool to upload data into Connected Insights automatically. Each configured pipeline monitors user storage for new molecular data.
, press Y.
b. [Mac and Linux] Run the following command on the terminal ./ici-uploader uninstall
❗ When prompted with message this operation will remove all contents from the folder. Are you sure to proceed?, press Y.
Select Stop, then select Start to start the service. The network drive is available for ingestion for Data Uploader.
For Test Definition, select the applicable definition.
[Optional] Input the location of the folder where case metadata data is available.
Select Save.
Select Generate.
In the API Key Generated window, select one of the following options:
Show — Reveals the API key.
Download API Key — Downloads the API key in .TXT file format.
Select Close after you have stored the API key.
❗ The API key cannot be viewed again after closing this window. Download the API key or save it in a secure location.
The API key is added to the Manage API keys list.
Perform any of the following actions in the Manage API Keys list:
Select Regenerate to generate a new API with the existing API key name.
Select Edit to edit the API key name or change the workgroups and roles selection.
Select Delete to delete the API key.
For DRAGEN server v4 standalone results:
/rest/of/storage path/{monitoring location}/{sample name folders}/{sample sheet, inputs, VCFs, BAMs/CRAMs}
Associate the workflow schema for this pipeline.
a. Under Choose compatible pipeline from the catalog, select the applicable pipeline from the drop-down list (for example, DRAGEN WGS Somatic v4.2).
b. If you are running a custom workflow, then upload a workflow schema that corresponds to the data that the configuration uploads. For more information, refer to Custom Pipeline Configuration.
Select the test definition to be associated with cases created by this configuration. For more information, refer to Test Definition Setup.
[Optional] Input the location where custom case data is stored. This location is the full path to where custom case data is deposited in user storage. For more information, refer to Custom Case Data Upload.
Select Save to complete the configuration. If Data Uploader is running, it monitors this configuration for data upload.
[Optional] --runId={path-to-config} — The id of the run to be created in place of the run ID determined by the run folder name.
[Optional] --pair-id={pair-id} — The pair-id of the analysis to upload from the Sample Sheet when limiting upload to a single analysis.
[Optional] --case-display-id={case-display-id} — The id of the case to be created in place of the pair-id when uploading a single analysis with --pair-id={pair-id}.
[Optional] ici-uploader logs show — Run to display the logs in ICI_Data_Upload_log.json.
[Optional] Downloads > ici-uploader logs download — Download Data Uploader logs in a zipped folder by running the command prompt.
Upload the custom case data file associated to one or more cases by running the following command:
ici-uploader case-data --filePath={absolute-path-of-csv-file}
The custom pipeline option is designed to make Connected Insights understand the structure of the secondary analysis output files produced by a pipeline that is not yet compatible with the software. This option also requires the creation of a workflow schema file that describes the content and location of the secondary analysis output files. For an example of a how to configure a custom pipeline for TSO 500 Analysis Module v2.2, refer to Custom Pipeline Configuration Example.
In Connected Insights v5.5 and later, the same YAML file also registers the pipeline and its QC thresholds when uploaded from Configuration > General > Pipeline & QC Configuration. For pipeline registration (pipeline:), QC thresholds (threshold:), and uploading the file, refer to Pipeline and QC Configuration and Pipeline and QC Configuration YAML. The sections below document the file-layout portions of the YAML (sampleSheet, jointFiles, sampleFiles).
Select Custom Pipeline
Input secondary analysis output folder to be monitored.
In Configuration Settings, select the radio button next to Configure custom pipeline.
If necessary, create a workflow schema file by selecting Download the template file. For more information on setting up the template file, refer to
Select Choose File to upload your template file.
For Custom Pipeline Name, enter a name for the pipeline.
For Test Definition, select the applicable definition.
[Optional] Input the location of the folder where is available.
Select Save.
To set up data upload for secondary analysis output data that is not yet compatible with Connected Insights, create a workflow schema file (.yaml format). This file specifies the files in the secondary analysis output data that Connected Insights analyzes. This file is only used when configuring a custom pipeline through Data Upload > From Local Storage.
Download a workflow schema file template from Connected Insights as follows.
On the top toolbar, select Configuration.
Select the General tab.
Select Data Upload.
Select From Local Storage.
The pipeline: block (display name, version, Platform Core ids, marker files) and the threshold: QC section are documented in . For uploading a complete configuration file from Configuration > General > Pipeline & QC Configuration, refer to .
These three sections tell Connected Insights where each analysis output file lives; they are what populate the Results, Visualize, and Lab QC tabs. The jointFiles section is required in practice — it declares the variant files, and an analysis imported without it brings in no variants and may fail; sampleSheet and sampleFiles are optional. Every file slot takes an ordered list of candidate glob patterns (first match wins), and each pattern may carry the trailing , Optional marker.
Locates the sample sheet and maps its columns to Connected Insights expected fields.
Key
Purpose
columnAliases — each is set to the corresponding column name in the lab's sample sheet:
Alias
Maps to
For disease aliases, if the disease name or ID is not provided, then the Status column on the Cases page displays Missing Required Data. This message displays until the disease name or ID is added. You can add a disease by uploading disease information as custom case data. You can also open the case in Connected Insights and enter the disease for an individual case. For more information on key genes, refer to .
Per-case variant, biomarker, and QC files. Each key is one logical file type; all are lists of candidate patterns. At least one variant file type (typically snvFiles) must be declared for an analysis to import any variants.
File names can include symbolic references to the files that depend on the Sample ID or Pair ID:
{sampleId.DNA}: Corresponds to the Sample_ID column when Sample_Type is DNA.
{sampleId.RNA}: Corresponds to the Sample_ID column when Sample_Type is RNA.
{sampleId.tumor.DNA}: Corresponds to the Sample_ID column when Sample_Type is DNA and Sample_Classification is TUMOR.
When using the workflow schema file template downloaded from the Configuration page, lines for files that are not uploaded can be deleted. The , Optional designation can be removed unless the file is an optional file for the pipeline.
Key
Typical format
Contents
Per-sample alignment and visualization files that drive IGV and the coverage plots. They are grouped by tumorPairId and normalPairId. For more information on IGV, refer to .
alignmentFiles — dnaBamFile, dnaBaiFile, rnaBamFile, rnaBaiFile (and generic bamFile / baiFile) drive the Visualize tab's IGV / BAM track. CRAM/CRAI may be supplied in place of BAM/BAI — listing both as candidates, as above, lets Connected Insights ingest whichever is present.
The following example shows the custom pipeline configuration process using Local Run Manager TruSight Oncology 500 Analysis Module v2.2. For details on this process, refer to .
Uploaded data is organized as cases that provide details about the sample. A case is a secondary analysis result that has been imported and annotated.These files include VCF files for genetic variants (or CSV files for TruSight Oncology 500 RNA Fusion variants). The cases page lists all cases for your account or workgroup. The following files can be uploaded, but are not required:
BAM files
JSON, TSV, and CSV files for TMB, MSI, and HRD biomarkers or for QC metrics
Make sure that the sample sheet is included in the secondary analysis results folder. The following example shows the structure of the [Data] section of the sample sheet:
Using this example, Connected Insights creates the following cases:
Open the secondary analysis results folder and find the files that must be identified in the workflow schema file. The following example shows the secondary analysis results folder structure:
For more information, refer to Create a Workflow Schema File in .
The following example shows the workflow schema file structure:
ici-uploader.exe
The installer for ici-uploader on Mac or Windows.
Mac: com.illumina.isi.daemon.plist
The installer for ici-uploader on Mac or Windows.
Linux: com.illumina.isi.daemon.service
The installer for ici-uploader on Mac or Windows.
ici-uploader or ici-uploader.exe
The installer for ici-uploader on Mac or Windows.
wrapper.exe
This file is used to set up Data Uploader as a system service on Windows.
README.txt
Third-party licensing information for Data Uploader.
For Define and Monitor Data Uploads, select Add Path.
For Configuration Settings, select the radio button next to Configure custom pipeline.
Select Download a template file to download the workflow schema template file.
If you do not want to create a pipeline, select Cancel. When prompted, select Yes, clear.
Edit the file as needed to reflect the files for upload. Refer to the following topics that pertain to the workflow schema template file sections:
Pipeline and QC Configuration YAML — pipeline: registration and threshold: QC metrics (required when uploading from Pipeline & QC Configuration)
After the workflow schema file is edited, create a pipeline. Then, select Configure manually under Configuration Settings.
Select Choose File and upload the edited workflow schema file.
Complete the remaining fields and save the pipeline.
If this pipeline is used for manual uploading, make sure that the pipeline name only consists of numbers, letters, underscores, and dashes. The name cannot include spaces or special characters. This name is used in the --pipeline-name= command listed in
specimenType
Specimen type
subjectId
Subject id
disease.id
Disease SNOMED CT code
disease.name
Disease SNOMED CT name
{sampleId.tumor.RNA}: Corresponds to the Sample_ID column when Sample_Type is RNA and Sample_Classification is TUMOR.
{sampleId.normal.DNA}: Corresponds to the Sample_ID column when Sample_Type is DNA and Sample_Classification is NORMAL.
{pairId}: Corresponds to the Pair_ID column, used for Case Display ID; the samples with same Pair_ID will group together as one case.
{prefix}: Corresponds to the Output_file_prefix column.
rnaSpliceFiles
VCF
RNA splice variants
rnaFusionFiles
VCF
RNA fusions
rnaGeneExpressionFiles
.quant.genes.sf
RNA gene expression (TPM)
tmbFile
JSON / CSV
Tumor mutational burden
msiFile
JSON
Microsatellite instability
gisFile
JSON
Genomic instability score
hrdFile
CSV
Homologous-recombination deficiency
mrdFile
JSON
Minimal residual disease
hlaFile
TSV
HLA genotyping
purityPloidyFiles
VCF / TSV
Purity and ploidy
metricsQCFile
CSV / TSV
QC metrics — the source for the threshold: section
visualizationFiles — coverageFiles, coverageBedFiles, and ballelesFiles feed the gene-coverage and B-allele-frequency plots.
Because every slot is an ordered list, several naming variants (tumor vs. prefix-based, BAM vs. CRAM, etc.) can be listed for one logical file, and Connected Insights picks the first that exists — useful when a pipeline's output names vary by run.
Custom Pipeline Configuration Example
Uploaded Data
Example Sample Sheet
Example Analysis Results
Example Workflow Schema File
If , Optional is after the file name, then Connected Insights uploads the file if it is available or moves on to the next available file.
This topic documents the complete YAML schema for a custom pipeline configuration file — the pipeline: registration block, the threshold: QC section, and how they relate to the file-layout sections. For file-layout sections (sampleSheet, jointFiles, sampleFiles), refer to Custom Pipeline Configuration. For uploading the file in Connected Insights, refer to Pipeline and QC Configuration.
Schema Overview
The file has three logical parts. The pipeline: and threshold: blocks are required. The file-layout sections are what make the Results, Visualize, and Lab QC tabs populate, and in practice at least the jointFiles section is needed: it declares the variant files (SNV/CNV/SV/fusion), and an analysis imported without it brings in no variants and may fail. The sampleSheet and sampleFiles sections are genuinely optional.
Part
YAML key(s)
Purpose
Required?
Path placeholders are substituted per case at ingestion time:
Placeholder
Resolves to
, Optional suffix — a file path ending in , Optional may be absent without failing ingestion. Omit the suffix to require the file.
Ordered candidate lists — every file slot accepts a list of candidate patterns. Connected Insights uses the first pattern that matches, so several naming variants can be listed for the same logical file.
Unknown keys are ignored — Connected Insights silently skips keys it does not recognize, so extra annotation in the file is harmless.
The pipeline: block registers the pipeline in Connected Insights. displayName, name, and version are required; everything else is optional.
Field
Required
Value / notes
icaPipelineId and icaUrn must be supplied together or not at all:
Both present → the pipeline is also linked to Platform Core and can be auto- or manually imported from there. It still supports Data Uploader and manual local upload.
Both absent → no Platform Core link; the pipeline is used via the Data Uploader and manual local upload only.
Only one present → the upload is rejected with a validation error.
successMarkerFile and failureMarkerFile are used only by the Connected Insights Data Uploader when it monitors a local folder for auto-upload — they signal to the uploader when a run in that folder has finished (success) or failed. They are ignored for Platform Core cloud analysis, which reports completion through the platform's own analysis-status events. Omit both when relying solely on Platform Core analysis status for completion.
The threshold: section defines the QC metrics Connected Insights evaluates for cases from this pipeline. At least one metric is required. Each metric is described once, with both its display properties and its pass/fail bounds:
Field
Meaning
QC evaluation follows the Connected Insights rule shown in the UI:
If a lower, an upper, or both limits are defined, the corresponding QC metric is marked Pass or Fail. If no limits are specified, it is marked Unspecified.
So a metric with isLowerEnable: true and a lowerValue receives a Pass/Fail verdict; a metric with neither bound enabled is carried as Unspecified (informational only).
The metrics Connected Insights evaluates come from the file(s) declared in jointFiles.metricsQCFile. A custom pipeline must emit that file in one of the two formats Connected Insights parses. Each threshold metric then points at a row in that file through its raw mapping. For the metricsQCFile key and other file-layout keys, refer to .
Every metric is either a run-level metric (applies to the whole sequencing run — e.g. percent reads passing filter, percent Q30) or a sample-level metric (applies to one sample — e.g. coverage, contamination). The two formats mark that distinction differently, as described below. Run-level metrics are configured with category: RUN_QC; sample metrics with SAMPLE_QC (or GENE_QC).
TSO 500 style — MetricsOutput.tsv (recommended). A single tab-separated file divided into bracketed sections. The run-level metrics live in the [Run QC Metrics] section; each remaining bracketed section (e.g. [DNA Library QC Metrics]) holds sample metrics. Every section starts with a header row, then one row per metric: the metric name, optional LSL Guideline / USL Guideline columns, then the value — a single Value column for the run section, or one value column per sample for a sample section.
The [Header] and [Analysis Details] sections are read as run metadata (a set of Field <tab> Value rows) and shown as run/analysis information. The field name is a fixed, closed set — Connected Insights recognizes only the labels below (case-insensitive), and any other row in these sections is ignored:
Field label (either section)
Shown as
These are metadata only, not QC thresholds; only the [Run QC Metrics] and sample sections carry the metrics matched by the threshold: config.
DRAGEN style — individual *.csv files. One or more comma-separated files (mapping_metrics.csv, wgs_coverage_metrics.csv, …), each row being Section,Group,Metric,Value[,Percentage]. The first column is the section name. Here the run-level metrics are the rows whose section is Run QC Metrics; all other rows are sample metrics. An optional 5th column is exposed as a separate <Metric> (Percentage) metric. To use this format, set pipeline.type: WGS so Connected Insights selects the DRAGEN parser.
(Typical DRAGEN *_metrics.csv files contain only sample sections; run-level rows are uncommon in that format.)
Choose one style. The TSO 500 MetricsOutput.tsv is preferred — its sections and per-sample columns are self-describing, which makes the raw mapping easier to author.
raw connects a threshold metric to a specific row in the QC file. Connected Insights matches (case-insensitively):
Scalar raw or raw.name → the metric name in the file (the first column of the metric row).
Source section for binding → raw.group when set in a nested object; otherwise the top-level group on the metric.
The metric's category selects which part of the file is searched: RUN_QC metrics are matched against the run-level section by metric name only; SAMPLE_QC (and GENE_QC) metrics are matched within a section by metric name and source section.
raw can be written three ways:
Omit raw when the display name already equals the metric name in the file.
Scalar string (default) — the file's metric name when the display name differs and the UI group matches the metrics-file section:
Nested object (optional) — when the UI display group should differ from the source section, or when the same metric name appears in more than one source section:
Worked example (TSO 500 style). Given the [DNA Library QC Metrics] section above, this metric surfaces CONTAMINATION_SCORE on the Results tab as "Estimated Sample Contamination" with an upper limit of 3106. The display group is shortened for the UI; raw.group points at the source section:
Category
Use
To set the reference line on the per-gene coverage plot (shown on the Results / Visualize tabs), add a GENE_QC metric named Gene Coverage. Its lowerValue is the coverage line:
This is the supported way to configure the coverage-plot line; it is set here rather than through a separate plot-config field.
A pipeline ingested via the Data Uploader or manual local upload, with a single small-variant file and one Pass/Fail metric. Platform Core ids omitted — no Platform Core import path.
For jointFiles details, refer to .
Adding the paired Platform Core ids links the pipeline to Platform Core for auto- or manual import. The same configuration remains usable via the Data Uploader. Both ids must be present, or neither.
A GENE_QC metric named Gene Coverage whose lowerValue sets the coverage reference line:
Use raw.group to point two same-named metrics at their respective source groups:
A complete, copy-pasteable file combining all of the above — pipeline block, file layout, and the full QC metric set — is provided in .
Upload Assertions
Previous interpretations can be uploaded to My Knowledge Base at any time (e.g., during onboarding) so you can immediately start prioritizing variants and reporting based on information from your lab.
Upload Assertions
Navigate to the app selector (grid icon) in the top-right of the screen.
Select My Knowledge Base.
In My Knowledge Base screen, select Add Assertions in the top-right.
Select attached template to download the upload CSV template.
Edit the template by adding values to the columns. After editing, save the file.
Upload the template file. The maximum upload file size is 10 MB.
The file will begin processing. To view the processing status, select View Past Uploads icon in the top-right of the My Knowledge Base screen.
If a file has issues, download the file and view the issues in a new error column provided in the download. Issues can be resolved in the file and re-uploaded.
Depending on the type of assertion you want to upload, only certain columns are required.
Assertion Type
Required Columns
Depending on the type of biomarker the assertion is about, only certain columns are required.
Biomarker Type
Level
Example
Required Columns
Column
Acceptable Values
If , Optional is after the file name, then Connected Insights uploads the file if it is available or moves on to the next available file.
QC metrics, pass/fail bounds, Results up-leveling, gene-coverage line
Yes (≥ 1 metric)
{prefix}
the output file prefix column
No
Platform Core pipeline UUID. Provide together withicaUrn to link the pipeline to Platform Core for cloud import. The pipeline remains usable via the Data Uploader with or without these ids. See the pairing rule below.
icaUrn
No
Platform Core pipeline URN (urn:ilmn:ica:pipeline:<uuid>). Provide together with icaPipelineId.
type
No
Connected Insights only acts on the value WGS, which selects the DRAGEN-style .csv QC-metrics parser. Set type: WGS when the QC metrics are DRAGEN .csv files; omit type entirely when they are a TSO 500-style MetricsOutput.tsv. Any other string has no effect (treated as TSO 500 style). See .
successMarkerFile
No
One or more glob patterns whose presence marks a completed run. Data Uploader only — see note below.
failureMarkerFile
No
File name whose presence marks a failed run. Data Uploader only — see note below.
metrics[].type
Rule type: RANGE (numeric / percentage / count) or BOOLEAN.
metrics[].isLowerEnable
true to evaluate the lower bound (lower spec limit).
metrics[].isUpperEnable
true to evaluate the upper bound (upper spec limit).
metrics[].lowerValue
Lower bound value, as a string. Omit if the lower bound is disabled.
metrics[].upperValue
Upper bound value, as a string. Omit if the upper bound is disabled.
metrics[].displayInOverView
true to up-level this metric to the case Results tab and report.
metrics[].disableDisplayInOverView
true to force-hide the metric from the Results tab.
Output Date or Report Date
Output Date
Output Time or Report Time
Output Time
1. Pipeline registration
pipeline:
Names the pipeline, sets version and type, and (optionally) the Platform Core pipeline id
Yes
2. File layout
sampleSheet:jointFiles:sampleFiles:
Tells Connected Insights where each analysis output file lives
jointFiles needed to import variants; sampleSheet / sampleFiles optional
3. QC thresholds
{sampleId.DNA}
the DNA sample's Sample_ID
{sampleId.RNA}
the RNA sample's Sample_ID
{sampleId.tumor.DNA}
the tumor DNA Sample_ID
{sampleId.normal.DNA}
the normal DNA Sample_ID
{pairId}
the Pair_ID that groups the case
pipeline:
displayName: Sample Custom Pipeline v1.0.0
name: Sample_Custom_Pipeline
version: 1.0.0
icaPipelineId: 00000000-0000-0000-0000-000000000000
icaUrn: urn:ilmn:ica:pipeline:00000000-0000-0000-0000-000000000000
type: WGS # only "WGS" is meaningful (selects the DRAGEN .csv QC parser); omit for TSO 500-style MetricsOutput.tsv
successMarkerFile:
- "{prefix}/{prefix}.time_metrics.csv, Optional"
failureMarkerFile: "failure.txt"
displayName
Yes
Human-readable name shown in Connected Insights. Use a report-friendly name.
name
Yes
Internal pipeline name. For pipelines used in manual upload, use only letters, numbers, underscores, and dashes (no spaces or special characters).
Overrides defaultCategory for this metric: RUN_QC, SAMPLE_QC, or GENE_QC.
Pipeline Version
Codon
BRAF p.(Val600)
genomeBuild
transcriptId
codon
geneSymbol
Exon
EGFR exon 19 deletion
genomeBuild
transcriptId
exon
geneSymbol
consequences (recommended)
Gene
ASXL1 oncogenic variant
genomeBuild
geneSymbol
consequences (recommended)
Copy Number Variant
Annotation Overlap
17:37844990-37884569 gain
variantType
genomeBuild
chromosome
position
end
Exon
BRCA1 2-11 copy number gain
genomeBuild
transcriptId
exon
geneSymbol
consequences (recommended)
Gene
TP53 copy number loss
variantType
genomeBuild
geneSymbol
consequences (recommended)
Structural Variant
Annotation Overlap
17:37844990-37884569 deletion
variantType
genomeBuild
chromosome
position
end
Partial fusion
ALK fusion
genomeBuild
geneSymbol
Exact fusion
EML4::ALK fusion
genomeBuild
geneSymbol
fusionGeneSymbol
fusionPosition
RNA Splice Variant
Exon
MET exon 14 skipping
genomeBuild
transcriptId
exon
geneSymbol
consequences (recommended)
Splice
AR-V7
genomeBuild
transcriptId
chromosome
start
stop
RNA Fusion Variant
Partial Fusion
ALK fusion
genomeBuild
geneSymbol
Specific Fusion
EML4, ALK fusion
genomeBuild
geneSymbol
fusionGeneSymbol
Exact Fusion
EML4::ALK fusion
genomeBuild
geneSymbol
fusionGeneSymbol
fusionPosition
Genomic Analysis
TMB
TMB high
status
MSI
MSI high
status
HRD
HRD high
status
classificationBackgroundColor
We recommend using the same classification specified in the screen.
Examples are as follows:
For Biological:
• For Pathogenic, red
• For Likely Pathogenic, pink
• For Uncertain Significance, yellow
• For Likely Benign, light green
• For Benign, green
For Therapeutic, Prognostic, Diagnostic:
• For Tier 1A, red
• For Tier 1B, red
• For Tier 2C, violet
• For Tier 2D, violet
• For Tier 3, blue
• For Tier 4, green
direction
For Therapeutic:
• Responsive
• Non-responsive
• Contraindicated
For Prognostic:
• Favorable
• Unfavorable
status
For TMB:
• High
• Low
For MSI:
• High
• Stable
For HRD:
• High
• Low
For Copy Number Variant:
• Copy number variation
• Copy number loss
• Copy number gain
• Copy number neutral
For Structural Variant:
• Tandem duplication
• Insertion
• Deletion
• Inversion
• Translocation
level
For Small variants:
• Nucleotide
• Amino Acid
• Codon
• Exon
• Gene
For Copy number variants:
• Annotation overlap
• Gene
For Structural variants:
• Annotation overlap, if variant type is not Translocation
• Partial fusion
• Exact fusion
For RNA splice variants:
• Annotation overlap
• Exon
For RNA fusion variants:
• Partial fusion
• Exact fusion
For Genomic analysis:
• TMB
• MSI
• HRD
genomeBuild
• 37
• 38
chromosome
• 1–22
• X
• Y
• MT
position
The VCF position. Acceptable values are within the chromosome range.
end
The VCF position. Acceptable values are within the chromosome range.
ref
The VCF reference allele. Acceptable values are combinations of A, T, C, G.
alt
The VCF alternate allele. Acceptable values are combinations of A, T, C, G.
transcriptId
Acceptable values are RefSeq and Ensembl transcript IDs, with or without the version.
codon
Acceptable values are a three-letter hgvsp abbreviation, including prefix “p.(“ and suffix “)” and without the amino acid change, such as p.(Val600).
hgvs
Acceptable values are a three-letter hgvsp abbreviation, including prefix “p.(“ and suffix “)”, such as p.(Val600Glu).
exon
Exon #
geneSymbol
NCBI gene symbol (e.g., BRAF)
ncbiGeneId
NCBI gene ID (e.g., 673)
fusionGeneSymbol
Fusion NCBI gene symbol (e.g., ALK)
fusionNcbiGeneID
Fusion NCBI gene ID (e.g., 238)
fusionPosition
Acceptable values are 0 and 1.
• Without a value, the fusion directionality is unspecified, such as EML4/ALK.
• If 0 is provided, the fusion is at position 0, such as [fusion gene]-[gene].
• If 1 is provided, the fusion is at position 1, such as [gene]-[fusion gene].
consequence
When a consequence is specified, the assertion only matches to a case variant that has the consequence specified, such as EGFR exon 19 inframe deletion.
If using Excel, make sure that no auto-formatting has occurred.
Editing the CSV Template
Required Assertion Columns
Required Biomarker Columns
Acceptable Column Values
genomeBuild
transcriptId
hgvsp
geneSymbol
We recommend using the same classifications specified in the screen.
Examples are as follows:
For Biological:
• For Oncogenic, 1.0
• For Likely Oncogenic, 2.0
• For Uncertain Significance, 3.0
• For Likely Benign, 4.0
• For Benign, 5.0
For Therapeutic, Prognostic, Diagnostic:
• For Tier 1A, 1.0
• For Tier 1B, 2.0
• For Tier 2C, 3.0
• For Tier 2D, 4.0
• For Tier 3, 5.0
• For Tier 4, 6.0
When multiple classifications map to the same AMP/ASCO/CAP tier, increment the tenths value. For example, if Level 1 and Level 2 both map to Tier 1A, the classificationOrder is as follows:
• Level 1, 1.0
• Level 2, 1.1
Note: When a new BioInsight Platform Core pipeline subversion is released your existing configurations will need to be refreshed to support them. Under the Data Upload section of the configuration, edit the relevant configuration (no additional actions are necessary) and save the configuration.
Connected Insights directly supports the following pipelines:
Pipeline Name and YAML
Support (Local Storage / BioInsight Platform Core)
