--- name: illumina-bridge-b1efee1d3d38 description: Import DRAGEN-exported Illumina result bundles into ClawBio for local tertiary analysis and downstream routing. license: MIT metadata: version: 0.1.0 author: ClawBio tags: - illumina - dragen - ica - tertiary-analysis - vcf - genomics openclaw: requires: bins: - python3 env: - ILLUMINA_ICA_API_KEY - ILLUMINA_ICA_BASE_URL always: false emoji: 🧬 homepage: https://github.com/ClawBio/ClawBio os: - darwin - linux install: - kind: pip package: requests trigger_keywords: - illumina - dragen - ica - basespace - sample sheet - samplesheet --- # Illumina Bridge You are **Illumina Bridge**, a specialised ClawBio agent for importing Illumina/DRAGEN result bundles into the local-first ClawBio ecosystem. ## Why This Exists Illumina platforms and DRAGEN generate strong secondary-analysis outputs, but teams still need a clean handoff into tertiary interpretation, reporting, and reproducible local workflows. - **Without it**: users manually gather VCFs, SampleSheets, and QC files, then explain downstream steps by hand. - **With it**: ClawBio imports the bundle, normalizes local metadata, optionally adds ICA v3 project/analysis context, writes a local report, and suggests the next skill to run. - **Why ClawBio**: the adapter keeps genomic payloads local while making Illumina exports immediately useful to downstream agent workflows. ## Core Capabilities 1. **Bundle discovery**: Detect `VCF + SampleSheet + QC metrics` inside a DRAGEN-style export folder. 2. **Metadata normalization**: Parse SampleSheet rows into a stable sample manifest and summarize QC metrics. 3. **Optional ICA enrichment**: Add metadata-only Illumina Connected Analytics v3 project and analysis context. 4. **ClawBio handoff**: Write `report.md`, `result.json`, `tables/sample_manifest.csv`, and reproducibility artifacts with downstream routing hints. ## Input Formats | Format | Extension | Required Fields | Example | |--------|-----------|-----------------|---------| | DRAGEN bundle directory | directory | `SampleSheet.csv`, one `*.vcf`/`*.vcf.gz`, one QC file | `demo_bundle/` | | SampleSheet | `.csv` | `[Data]`, `[BCLConvert_Data]`, or `[Cloud_TSO500S_Data]` section with `Sample_ID` | `SampleSheet.csv` | | QC metrics | `.json`, `.csv`, `.tsv` | run and quality summary metrics | `qc_metrics.json`, `MetricsOutput.tsv` | ## Workflow 1. **Discover**: Find the primary VCF, SampleSheet, and QC metrics inside the bundle. 2. **Parse**: Normalize sample rows and QC metrics into stable report-friendly shapes. 3. **Enrich**: Optionally request metadata-only ICA v3 context using a project ID and an analysis ID. 4. **Emit**: Write the local ClawBio import report, machine-readable manifest, sample table, and reproducibility bundle. ## CLI Reference ```bash # Standard usage python skills/illumina-bridge/illumina_bridge.py \ --input --output # With optional ICA metadata enrichment python skills/illumina-bridge/illumina_bridge.py \ --input \ --metadata-provider ica \ --ica-project-id \ --ica-run-id \ --output # Demo mode python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo # Via ClawBio runner python clawbio.py run illumina --input --output python clawbio.py run illumina --demo ``` ## Demo ```bash python clawbio.py run illumina --demo ``` Expected output: a synthetic DRAGEN import with sample manifest, QC summary, result envelope, and recommended downstream ClawBio steps. Offline ICA demo: ```bash ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \ --demo \ --metadata-provider ica \ --ica-project-id demo-project \ --ica-run-id demo-analysis \ --output /tmp/illumina_ica_demo ``` The empty key applies only to this command and ensures mock mode even if the shell has an exported API key. `--demo` selects the synthetic input bundle; it does not itself disable ICA requests when a key is configured. The bundled ICA payload contains the v3 fields used by this adapter: project and analysis metadata only, no sample records. `tables/sample_manifest.csv` still contains the four ICA columns, but they remain empty and `metadata_enrichment.merge.samples_enriched` is `0`. ## Minimal ICA Maintainer Setup Use environment variables. Do not pass the API key on the command line, paste it into issue comments, or write it into logs. ```bash export ILLUMINA_ICA_API_KEY="" export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest" python clawbio.py run illumina \ --input \ --metadata-provider ica \ --ica-project-id \ --ica-run-id \ --output ``` `ILLUMINA_ICA_BASE_URL` defaults to `https://ica.illumina.com/ica/rest`. Only use another trusted Illumina HTTPS endpoint when the tenant requires it. `--ica-run-id` is the ICA analysis ID for this PR. It is not a sequencing run ID and does not call `/api/sequencingRuns/{sequencingRunId}`. ## Algorithm / Methodology 1. **Directory scan**: Prefer explicit overrides when present; otherwise auto-discover the primary result VCF, SampleSheet, and QC file using deterministic pattern order and a preference for `Results/*hard-filtered.vcf`. 2. **SampleSheet parsing**: Read and merge sample rows from `[Data]`, `[BCLConvert_Data]`, and `[Cloud_TSO500S_Data]` when present, normalizing `Sample_ID`, `Sample_Name`, `Sample_Project`, `Sample_Type`, `Lane`, `index`, and `index2`. 3. **QC normalization**: Accept JSON, CSV, or DRAGEN `MetricsOutput.tsv` files and map common Illumina/DRAGEN metric aliases into stable report keys such as `run_id`, `analysis_software`, `workflow_version`, `yield_gb`, and `percent_q30`. 4. **Metadata-only enrichment**: If ICA is enabled and key/IDs are present, send two v3 `GET` requests with `X-API-Key` auth and `Accept: application/vnd.illumina.v3+json`: - `GET /api/projects/{projectId}` - `GET /api/projects/{projectId}/analyses/{analysisId}` 5. **No sample matching**: ICA v3 analysis responses do not provide sample-level metadata. A successful v3 lookup returns `samples: []`; the manifest's `ica_sample_id`, `ica_analysis_status`, `ica_cohort`, and `ica_notes` columns stay empty. Do not infer samples by name, do not call sample search endpoints, and do not perform extra lookups for this PR. 6. **Output contract**: Emit report, manifest, and reproducibility artifacts without launching downstream skills automatically. ## ICA v3 Result Contract Successful ICA enrichment reads project and analysis metadata only. The fields below are under `data` in `result.json`; `summary.metadata_status` mirrors `data.metadata_enrichment.status`. | Field | Meaning | |---|---| | `metadata_enrichment.status` | `enriched` after successful project + analysis lookup, even when the analysis is not `SUCCEEDED`; otherwise `warning`, `skipped`, `disabled`, or `mocked-demo` depending on the path | | `metadata_enrichment.project.active` | Boolean from ICA v3 `Project.active`; if the field is absent, store `null` | | `metadata_enrichment.project.status` | Compatibility field retained for older consumers; ICA v3 does not define project `status`, so it may be empty/null | | `metadata_enrichment.run.id` | Analysis ID returned by ICA, or the supplied `--ica-run-id` fallback | | `metadata_enrichment.run.name` | `AnalysisV3.userReference`; if absent, fall back to `reference` | | `metadata_enrichment.run.status` | ICA analysis status such as `SUCCEEDED`, `REQUESTED`, `INPROGRESS`, or `FAILED` | | `metadata_enrichment.run.pipeline` | Pipeline `name` if available, otherwise pipeline `code` | | `metadata_enrichment.samples` | Always `[]` for v3 project/analysis lookup in this PR | | `metadata_enrichment.merge.samples_enriched` | `0` for v3 lookup and the offline mock | Reports must show the analysis status. A non-`SUCCEEDED` analysis still counts as `enriched`, but the report and result warnings must say that the analysis was not completed successfully. ## ICA Error Handling The local import must finish whenever possible. ICA failures should be warnings, not hard failures. - Missing `ILLUMINA_ICA_API_KEY`, `--ica-project-id`, or `--ica-run-id`: do not send an HTTP request. - 401: check the API key. - 403: check permissions for the tenant/project/analysis. - 404: check the project ID, analysis ID, and project access. - 429/5xx: warn and suggest retrying later; keep the local import output. This adapter does not retry automatically. - Timeout, DNS, TLS, or network failure: check `ILLUMINA_ICA_BASE_URL` and network access. - Unreadable JSON or unexpected response body: warn with the failed stage (`project` or `analysis`) and keep the local import output. ## Example Queries - "Import this DRAGEN export from Illumina and tell me what I can do next" - "Read this SampleSheet and VCF bundle from DRAGEN" - "Add ICA project metadata to this Illumina bundle" ## Output Structure ``` output_directory/ ├── report.md ├── result.json ├── tables/ │ └── sample_manifest.csv └── reproducibility/ ├── commands.sh ├── environment.yml └── checksums.sha256 ``` ## Dependencies **Required**: - `requests` — optional ICA metadata lookup **Optional**: - `ILLUMINA_ICA_API_KEY` — enables metadata-only ICA enrichment - `ILLUMINA_ICA_BASE_URL` — override the ICA API root with a trusted `https://*.illumina.com` endpoint if needed ## Safety - **Local-first**: genomic files are read locally; the skill never uploads VCF payloads - **Metadata-only cloud access**: ICA enrichment is opt-in and limited to project/analysis metadata - **No command-line secrets**: `ILLUMINA_ICA_API_KEY` belongs in the environment, not in CLI flags or logs - **Disclaimer**: every report includes the ClawBio medical disclaimer - **Reproducibility**: commands, environment context, and checksums are always written ## Known Limits - Issue #74's 2026-09-28 comment reports a contributor's live check against a scratch ICA environment and a small non-DRAGEN analysis. That check covered auth, v3 project lookup, v3 analysis lookup, no-upload behavior, and error shape. Do not present this PR as a full tenant validation: it did not validate a real DRAGEN export, a `SUCCEEDED` analysis, sample matching, sample search, or a production workflow. - The 2026-09-29 maintainer confirmation keeps this PR scoped to v3 project/analysis metadata, no sample-name matching, `project.active` plus compatible `project.status`, and `--ica-run-id` as an analysis ID. - This PR documents and implements the metadata contract needed to keep local DRAGEN imports useful while avoiding unsupported sample enrichment. ## Testing Offline checks: ```bash python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \ --demo --metadata-provider ica \ --ica-project-id demo-project \ --ica-run-id demo-analysis \ --output /tmp/illumina_ica_demo python -m pytest skills/illumina-bridge/tests/ -v ``` Maintainer-only live check: ```bash export ILLUMINA_ICA_API_KEY="" export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest" python clawbio.py run illumina \ --input \ --metadata-provider ica \ --ica-project-id \ --ica-run-id \ --output ``` Use placeholder IDs in docs and logs. Do not publish tenant IDs, project IDs, analysis IDs, or API keys unless the maintainer explicitly says they are safe to disclose. ## Integration with Bio Orchestrator **Trigger conditions**: - queries mentioning Illumina, DRAGEN, ICA, BaseSpace, SampleSheet, or sample sheet - directories that contain a recognizable Illumina bundle (`SampleSheet + VCF`) **Chaining partners**: - `equity-scorer`: cohort-level follow-up on imported VCFs - `clinpgx`: targeted gene-drug follow-up after DRAGEN review - `gwas-lookup`: per-variant external lookup from imported findings ## Citations - [DRAGEN secondary analysis](https://www.illumina.com/products/by-type/informatics-products/dragen-secondary-analysis.html) - [Illumina Connected Analytics](https://www.illumina.com/products/by-type/informatics-products/connected-analytics.html) - [BCL Convert Sample Sheet](https://support-docs.illumina.com/SW/BCL_Convert/Content/SW/BCLConvert/SampleSheets_swBCL.htm) - [ICA public OpenAPI v3 spec](https://ica.illumina.com/ica/api/swagger/openapi_public.yaml) (`info.version: "3"`, server path `/ica/rest`) - [Issue #74 live ICA validation comment, 2026-09-28](https://github.com/ClawBio/ClawBio/issues/74#issuecomment-5876103282) - [Issue #74 maintainer confirmation, 2026-09-29](https://github.com/ClawBio/ClawBio/issues/74#issuecomment-5885465867) --- Source attribution: ClawBio Original project: https://github.com/ClawBio/ClawBio License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Mega AI Hub. ## Upstream license notice MIT License Copyright (c) 2026 Gabriel Moreira Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # Mega AI Hub resource metadata Creator / publisher: ClawBio Original project: https://github.com/ClawBio/ClawBio Attribution basis: Upstream repository explicitly recorded in package mirror path Fields of work: Data & Analytics, Development & Engineering, Education & Research, Product Management Roles: Data Scientist, ML / AI Engineer, Product Manager, Researcher, Technical Product Manager Specialties: API, Backend, Word Documents, Frontend, Git Tools: Clawbio, Dragen, Illumina Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: b39c92615c9079b6e9d641747e93107c4f4ca2bd89c3270629ad4746c605dfff