Internal - Version 2.2
Request changes: high-risk issues require remediation before approval.
Verdict: Request changes: high-risk issues require remediation before approval.
Risk posture: Elevated based on 6 normalized finding(s).
local repository and generated documentation/report artifacts
source code, docs, examples, local script execution, generated report artifacts
True
No live MCP calls were used. The MCP necessity test did not pass because local source, docs, examples, generated artifacts, and bundled scripts were sufficient.
This detailed review is personalized for the Hades Blackwatch skill repository. The specialist notes separate the agents that materially contributed from agents whose domain was checked but not deeply exercised because the repo is a local skill/tooling package rather than a deployed AI application.
Scope, authorization, MCP necessity, final severity, and report structure
Kept the review local and source-driven, avoided unnecessary MCP calls, and prioritized six findings by report-leakage, evidence integrity, MCP audit depth, schema migration compatibility, documentation accuracy, and tests.
Keep this role as the default entry point for full-stack review so every generated report explains what was actually tested and what was not.
Current OWASP, MITRE, NIST, and MCP guidance relevant to skills, agent tools, scanner evidence, and MCP boundaries
Added dated threat-intelligence signals and official source links so the report explains why redaction, MCP least privilege, evidence provenance, and governance controls matter for this repo.
Run this agent when reports claim current framework coverage, CVEs, vendor tool behavior, or external threat activity.
Trust boundaries between prompts, local scripts, generated reports, scanner evidence, and future MCP tools
Identified the main local attack paths: untrusted scanner evidence flowing into reports, generated reports polluting future analysis, and MCP schema under-classification.
MCP escalation policy and manifest risk-tier analysis
Validated that no live MCP call was needed and found that nested tool schemas need recursive inspection before real Burp, Qualys, Sonar, or custom MCP adapters are trusted.
Scanner backend claims, evidence ingestion, and auto-detection expectations
Confirmed report ingestion and local scripts exist, while scanner auto-detection remains roadmap-only and should be described that way until implemented.
Prompt and report-content poisoning paths
No live model target was present, but generated artifacts and scanner evidence were treated as untrusted context that can influence later agent runs.
Tool permissions, evidence-to-report flow, and state-changing MCP policy
Mapped the most important tool risks to evidence disclosure, MCP authority classification, and approval boundaries for future scanner adapters.
Retrieved content and data-store exposure
The repository does not implement a RAG runtime, but the same context-poisoning concern appears in generated reports being re-ingested by later local scans.
Python scripts, JSON normalization, HTML/PDF rendering, and local workflow integrity
Found redaction, generated-artifact scanning, schema migration, and missing automated test risks.
Skill, script, schema, and scanner-adapter supply-chain integrity
Highlighted schema migration compatibility and future scanner-adapter expectations; live package vulnerability lookup was excluded from this local review.
Deduplication, severity calibration, and root-cause grouping
Grouped the findings into three practical themes: sensitive evidence handling, evidence quality/provenance, and implementation-to-documentation alignment.
Regression tests, fixtures, and release gates
Converted each finding into a concrete test or checklist item, with emphasis on redaction, surface exclusion, recursive MCP schema checks, schema identity, and CI coverage.
HTML/PDF deliverables, personalized recommendations, threat intel, and audit-ready detail
Generated the upgraded report structure with first-class skill coverage, source-linked threat intelligence, and target-specific recommendations.
As of 2026-05-28
The current threat context for Hades Blackwatch centers on untrusted evidence flowing through agentic review pipelines, MCP/tool authority boundaries, skill and adapter supply-chain integrity, and governance-grade traceability. These signals are mapped to the local findings instead of being generic framework text.
OWASP LLM guidance continues to emphasize sensitive information disclosure and improper output handling. For this repo, the practical risk is raw Burp, Qualys, Sonar, Semgrep, CodeQL, Trivy, or custom JSON evidence being copied into polished reports.
Add recursive redaction before normalization and rendering, then make report generation tests assert that secrets never survive into HTML or PDF bytes.
MCP security guidance calls out token passthrough, SSRF, local server compromise, session handling, and scope minimization. For Hades Blackwatch, this makes recursive manifest inspection and escalation-only MCP use important before connecting Burp, Qualys, Sonar, or internal scanners.
Upgrade nested schema analysis, require risk-tier evidence for every MCP tool, and keep live MCP calls behind the necessity test.
Agentic systems are sensitive to stale or attacker-shaped context. The local surface mapper currently treats reports as review evidence, which can anchor later analysis on old findings and inflated keyword counts.
Exclude generated report paths by default and require explicit opt-in when report artifacts themselves are the audit target.
OWASP Agentic Skills and NIST AI RMF style governance both push toward provenance, traceability, and repeatable controls. Undocumented schema migration and manual-only smoke tests weaken downstream trust in this skill package.
Document schema migration behavior and add automated tests plus CI before new adapters are added.
These actions are tailored to the target, authorization model, actual evidence sources, and the current finding set.
Hades Blackwatch is designed to ingest Burp, Qualys, Sonar, Semgrep, CodeQL, Trivy, and custom evidence. Without deterministic minimization, the report generator can turn private scanner data into a broadly shared HTML/PDF artifact.
This repository generates rich security reports under reports/. Letting future scans read those files as source evidence can inflate surface counts and reintroduce stale findings.
The skill's strategy is MCP only when absolutely needed. That policy is strongest when the auditor can see nested command, URL, credential, body, header, and action fields before a tool is trusted.
The skill now emits the Hades Blackwatch schema after the rename, but downstream consumers may still have older Cerberus-era expectations. A migration note keeps reports, dashboards, and future agents aligned.
The skill should feel powerful, but security users need exact evidence. Capability wording should distinguish implemented scripts, report ingestion, role guidance, and roadmap work.
Hades Blackwatch is becoming reusable security infrastructure. Redaction, schema naming, report rendering, generated-output exclusions, and recursive MCP checks should fail fast when changed.
HB-DET-001 - manual source review, normalize_findings.py, generate_report.py
scanner evidence normalization and report generation
scripts/normalize_findings.py:103; scripts/generate_report.py:211; scripts/generate_report.py:406
Not provided.
HTML/PDF security reports generated from scanner exports
{
"summary": "The generic normalizer assigns the entire scanner item to the normalized evidence field. The report generator then renders the evidence field into HTML and PDF. HTML escaping is present, but no recursive redaction or evidence minimization is applied before report generation.",
"observed_code_paths": [
"normalize_generic stores evidence: item",
"render_html renders finding evidence through html_block",
"flatten_for_pdf serializes finding evidence into PDF text"
],
"positive_control": "HTML content is escaped, reducing script injection risk. This does not prevent disclosure of sensitive scanner evidence."
}A Burp, Qualys, Sonar, or custom scanner export includes request headers, cookies, bearer tokens, API keys, session identifiers, request bodies, or response snippets. normalize_findings.py copies the raw item into the evidence field. generate_report.py embeds that evidence in a polished report. The report is then shared more broadly than the raw scanner export, leaking sensitive operational details.
The skill explicitly treats scanner output as untrusted input, and the report agent turns that input into downstream deliverables. In an agentic workflow, this expands the blast radius of sensitive scanner evidence unless redaction is deterministic.
As of 2026-05-28
Current AI security guidance treats tool output, scanner output, and report artifacts as sensitive context that can leak credentials or operational details when copied without minimization.
{
"mitre_atlas": "Exfiltration and credential access through tool-output handling",
"cwe": "CWE-200; CWE-532"
}Use fixtures with nested secrets and assert normalized JSON, HTML, and PDF bytes do not contain secret values.
For Hades Blackwatch Skill Repository, Raw scanner evidence can be copied into shareable HTML/PDF reports without secret redaction can be understood as a control failure where untrusted influence reaches scanner evidence normalization and report generation and can affect HTML/PDF security reports generated from scanner exports. A Burp, Qualys, Sonar, or custom scanner export includes request headers, cookies, bearer...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into scanner evidence normalization and report generation.
A Burp, Qualys, Sonar, or custom scanner export includes request headers, cookies, bearer tokens, API keys, session identifiers, request bodies, or response snippets. normalize_findings.py copies the raw item...
The unsafe path can affect HTML/PDF security reports generated from scanner exports.
Create a fixture with nested request and response objects containing Authorization, Cookie, Set-Cookie, api_key, password, bearer token, private key markers, and session IDs. Run normalize_findings.py and gene...
Influence scanner evidence normalization and report generation so the workflow reaches HTML/PDF security reports generated from scanner exports in an unintended way.
The skill explicitly treats scanner output as untrusted input, and the report agent turns that input into downstream deliverables. In an agentic workflow, this expands the blast radius of sensitive scanner evidence unless redaction is deterministic.
Missing or incomplete deterministic enforcement at the trust boundary.
HTML/PDF security reports generated from scanner exports
Create a fixture with nested request and response objects containing Authorization, Cookie, Set-Cookie, api_key, password, bearer token, private key markers, and session IDs. Run normalize_findings.py and generate_report.py, then assert none of the secret val...
Add a recursive redaction layer before storing normalized evidence and before rendering reports. Redact key names and values matching Authorization, Cookie, Set-Cookie, token, secret, password, api_key, bearer, private_key, session, credential, client_secret, and configurable organization patterns. Prefer a safe evide...
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Sensitive Information Disclosure",
"owasp_agentic": "Information Disclosure through Tool and Context Handling",
"mitre_atlas": "Credential Access / Exfiltration through tool-output handling",
"nist_ai_rmf": "Map, Measure, Manage",
"cwe": "CWE-200; CWE-532",
"cve": null
}{
"system": "AIVSS",
"value": "High",
"rationale": "The issue can expose high-value security evidence and credentials, is easy to reproduce with realistic scanner exports, and is amplified by automated report sharing."
}Add a recursive redaction layer before storing normalized evidence and before rendering reports. Redact key names and values matching Authorization, Cookie, Set-Cookie, token, secret, password, api_key, bearer, private_key, session, credential, client_secret, and configurable organization patterns. Prefer a safe evidence summary allowlist over storing full raw scanner items by default.
Create a fixture with nested request and response objects containing Authorization, Cookie, Set-Cookie, api_key, password, bearer token, private key markers, and session IDs. Run normalize_findings.py and generate_report.py, then assert none of the secret values appear in normalized JSON, HTML, or PDF bytes.
Some scanner proof data can remain sensitive even after key-based redaction. High-risk reports should carry a classification label and support an evidence-minimized mode.
HB-DET-003 - mcp_manifest_audit.py, manual source review
MCP tool contract audit
scripts/mcp_manifest_audit.py:68-98
Not provided.
MCP risk tier assignment and tool safety classification
{
"summary": "The auditor reads only top-level schema properties for explicit issue generation. It may see nested words through JSON stringification, but it does not recursively report nested free-form parameters, unconstrained object schemas, additionalProperties, arrays of action objects, or weakly constrained nested URL/path/command/body fields.",
"positive_control": "The example manifest correctly flags a top-level command parameter as T4.",
"gap": "A tool can hide broad authority under a neutral top-level object such as config, request, options, action, or payload."
}An MCP server exposes a tool with a benign top-level schema but nested command, URL, path, headers, body, credential, or action fields. The current auditor may not clearly emit high-risk nested JSON-path issues, causing the orchestrator to underestimate the server's risk tier.
MCP tools are privileged agent boundaries. Under-classifying nested tool authority can make prompt injection or malicious tool metadata more dangerous in a connected agent environment.
As of 2026-05-28
MCP security guidance emphasizes authorization boundaries, SSRF, token handling, local server risk, and scope minimization. Nested schemas can hide exactly the kind of authority that needs strict review.
{
"mitre_atlas": "Tool abuse via agent capability",
"cwe": "CWE-20; CWE-749"
}Add nested MCP manifest fixtures with command, URL, headers, body, credential, and additionalProperties paths; assert T4 classification.
For Hades Blackwatch Skill Repository, MCP manifest auditor can under-classify nested permissive tool schemas can be understood as a control failure where untrusted influence reaches MCP tool contract audit and can affect MCP risk tier assignment and tool safety classification. An MCP server exposes a tool with a benign top-level schema but nested command, URL, path, headers, body, credential, or action fields. The...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into MCP tool contract audit.
An MCP server exposes a tool with a benign top-level schema but nested command, URL, path, headers, body, credential, or action fields. The current auditor may not clearly emit high-risk nested JSON-path issue...
The unsafe path can affect MCP risk tier assignment and tool safety classification.
Add a manifest fixture with inputSchema.properties.config.properties.command, inputSchema.properties.request.properties.url, nested headers/body fields, and additionalProperties true. Assert the auditor return...
Influence MCP tool contract audit so the workflow reaches MCP risk tier assignment and tool safety classification in an unintended way.
MCP tools are privileged agent boundaries. Under-classifying nested tool authority can make prompt injection or malicious tool metadata more dangerous in a connected agent environment.
Missing or incomplete deterministic enforcement at the trust boundary.
MCP risk tier assignment and tool safety classification
Add a manifest fixture with inputSchema.properties.config.properties.command, inputSchema.properties.request.properties.url, nested headers/body fields, and additionalProperties true. Assert the auditor returns T4 and reports each nested JSON path.
Walk input schemas recursively and report exact JSON paths. Flag additionalProperties true or absent on object-like inputs, nested sensitive parameter names, arrays of action objects, missing required fields, broad string parameters, and URL/path/command-like fields without enums, allowlists, regex constraints, or app...
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Excessive Agency; Insecure Plugin Design",
"owasp_agentic": "Tool Misuse; Agentic Supply Chain",
"mitre_atlas": "Tool abuse and execution via agent capability",
"nist_ai_rmf": "Map, Measure, Manage",
"cwe": "CWE-20; CWE-749",
"cve": null
}{
"system": "AIVSS",
"value": "Medium",
"rationale": "The weakness affects detection quality for potentially high-risk tools. Impact depends on the reviewed MCP manifests and whether live high-privilege MCP tools are connected."
}Walk input schemas recursively and report exact JSON paths. Flag additionalProperties true or absent on object-like inputs, nested sensitive parameter names, arrays of action objects, missing required fields, broad string parameters, and URL/path/command-like fields without enums, allowlists, regex constraints, or approval language.
Add a manifest fixture with inputSchema.properties.config.properties.command, inputSchema.properties.request.properties.url, nested headers/body fields, and additionalProperties true. Assert the auditor returns T4 and reports each nested JSON path.
Static manifest checks cannot prove runtime authorization. High-risk MCP tools still need runtime review, allowlists, approval gates, and audit logs.
HB-DET-002 - surface_mapper.py, manual source review
local attack surface discovery
scripts/surface_mapper.py:76-92; .gitignore:9
Not provided.
review evidence quality and finding accuracy
{
"summary": "DEFAULT_SKIP_DIRS excludes common build and dependency folders but not reports/ or out/. The rerun included generated report HTML, PDF, and report input JSON as if they were source surfaces. The surface counts increased after report generation: mcp rose to 32, authz to 57, and secrets to 55 because prior reports contained those words.",
"observed_noise": [
"reports/hades-blackwatch-fullstack-review-input.json",
"reports/hades-blackwatch-skill-repository-hades-blackwatch-full-stack-security-review.html",
"reports/hades-blackwatch-skill-repository-hades-blackwatch-full-stack-security-review.pdf"
],
"control_gap": "reports/ is ignored in .gitignore but not skipped by surface_mapper.py."
}Generated reports contain security terminology and possibly sensitive evidence. A later full-stack analysis scans those reports and treats them as current code/docs evidence, producing duplicated findings, stale evidence, or false positives.
Generated reports can become context for subsequent agent runs. If the tool cannot separate primary source from generated evidence, specialist agents may anchor on stale report content rather than current system behavior.
As of 2026-05-28
Agentic review workflows are vulnerable to stale or attacker-shaped context when generated artifacts are re-ingested as if they were primary source evidence.
{
"mitre_atlas": "Data poisoning / evidence manipulation analogue",
"cwe": "CWE-703"
}Add generated report fixtures and assert surface_mapper.py excludes them unless an explicit include-generated option is used.
For Hades Blackwatch Skill Repository, Surface mapper scans generated reports and previous report inputs, polluting later analysis can be understood as a control failure where untrusted influence reaches local attack surface discovery and can affect review evidence quality and finding accuracy. Generated reports contain security terminology and possibly sensitive evidence. A later full-stack analysis scans those rep...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into local attack surface discovery.
Generated reports contain security terminology and possibly sensitive evidence. A later full-stack analysis scans those reports and treats them as current code/docs evidence, producing duplicated findings, sta...
The unsafe path can affect review evidence quality and finding accuracy.
Create reports/generated.html containing token, MCP, credential, and authorization terms. Run surface_mapper.py and assert reports/generated.html is excluded by default. Then run with an explicit include-gener...
Influence local attack surface discovery so the workflow reaches review evidence quality and finding accuracy in an unintended way.
Generated reports can become context for subsequent agent runs. If the tool cannot separate primary source from generated evidence, specialist agents may anchor on stale report content rather than current system behavior.
Missing or incomplete deterministic enforcement at the trust boundary.
review evidence quality and finding accuracy
Create reports/generated.html containing token, MCP, credential, and authorization terms. Run surface_mapper.py and assert reports/generated.html is excluded by default. Then run with an explicit include-generated option and assert it appears.
Add reports/, out/, coverage/, .pytest_cache/, .mypy_cache/, and other generated-output paths to DEFAULT_SKIP_DIRS. Add --include-generated and --exclude options for controlled overrides. Consider reading .gitignore patterns for default exclusions.
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Context/Data Poisoning analogue for review evidence",
"owasp_agentic": "Evidence Integrity / Context Poisoning",
"mitre_atlas": "Data poisoning and evasion-adjacent evidence manipulation",
"nist_ai_rmf": "Measure, Manage",
"cwe": "CWE-703",
"cve": null
}{
"system": "AIVSS",
"value": "Medium",
"rationale": "The issue does not directly execute code or expose new data, but it materially degrades the accuracy and repeatability of agentic security reviews."
}Add reports/, out/, coverage/, .pytest_cache/, .mypy_cache/, and other generated-output paths to DEFAULT_SKIP_DIRS. Add --include-generated and --exclude options for controlled overrides. Consider reading .gitignore patterns for default exclusions.
Create reports/generated.html containing token, MCP, credential, and authorization terms. Run surface_mapper.py and assert reports/generated.html is excluded by default. Then run with an explicit include-generated option and assert it appears.
Some users may intentionally want to audit report artifacts. That should be explicit and separate from source surface mapping.
HB-DET-005 - manual docs review
README and documentation accuracy
README.md:159; README.md:228; README.md:440; docs/index.html:753; docs/index.html:895
Not provided.
user expectations for scanner integrations
{
"summary": "README and docs state that scanner backends are detected and that the skill chooses the safest integration path. The roadmap also says scanner_registry.py still needs to be added. The current implementation has a scanner-registry agent role but no executable scanner registry script.",
"current_implementation": "Role guidance exists in agents/roles/scanner-registry.md, but scanner_registry.py is roadmap-only.",
"impact": "Users may expect automatic detection of Sonar, Burp, Qualys, Semgrep, CodeQL, or Trivy when the current repo mainly supports manual/report-driven workflows."
}A user relies on documentation that implies automated scanner detection, then assumes coverage exists when no scanner backend was actually queried or imported. This can create false assurance in a security review.
Agentic tools should be explicit about what evidence was actually collected. Overstated capability descriptions can cause both humans and agents to over-trust missing scanner coverage.
As of 2026-05-28
Threat-informed security reporting depends on evidence provenance. Overstating scanner or MCP coverage can create false assurance and hide missing controls.
{
"mitre_atlas": "Defense evasion via false assurance",
"cwe": "CWE-1059"
}Add a docs consistency check that compares capability claims to implemented scripts and report evidence.
For Hades Blackwatch Skill Repository, Documentation describes scanner auto-detection before the registry implementation exists can be understood as a control failure where untrusted influence reaches README and documentation accuracy and can affect user expectations for scanner integrations. A user relies on documentation that implies automated scanner detection, then assumes coverage exists when no scanner backend...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into README and documentation accuracy.
A user relies on documentation that implies automated scanner detection, then assumes coverage exists when no scanner backend was actually queried or imported. This can create false assurance in a security rev...
The unsafe path can affect user expectations for scanner integrations.
Add a docs consistency checklist or test that verifies README capability claims match implemented scripts and roadmap status.
Influence README and documentation accuracy so the workflow reaches user expectations for scanner integrations in an unintended way.
Agentic tools should be explicit about what evidence was actually collected. Overstated capability descriptions can cause both humans and agents to over-trust missing scanner coverage.
Missing or incomplete deterministic enforcement at the trust boundary.
user expectations for scanner integrations
Add a docs consistency checklist or test that verifies README capability claims match implemented scripts and roadmap status.
Adjust wording to say Hades Blackwatch is designed to support scanner detection and currently supports local scripts, report ingestion, examples, and role guidance. Keep auto-detection language only after scanner_registry.py exists and is tested.
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Overreliance / Supply Chain Tooling Gap",
"owasp_agentic": "Tool Reliability / Coverage Gap",
"mitre_atlas": "Defense evasion via false assurance",
"nist_ai_rmf": "Govern, Map, Measure",
"cwe": "CWE-1059",
"cve": null
}{
"system": "AIVSS",
"value": "Low",
"rationale": "This is a documentation and expectation risk, not a direct vulnerability. It matters because the skill is intended to drive security decisions."
}Adjust wording to say Hades Blackwatch is designed to support scanner detection and currently supports local scripts, report ingestion, examples, and role guidance. Keep auto-detection language only after scanner_registry.py exists and is tested.
Add a docs consistency checklist or test that verifies README capability claims match implemented scripts and roadmap status.
Even after scanner registry implementation, reports should always list which backends actually ran, which were skipped, and why.
HB-DET-004 - normalize_findings.py, manual source review
finding normalization and downstream interoperability
scripts/normalize_findings.py:127
Not provided.
normalized findings consumed by reports, CI, or downstream tooling
{
"summary": "normalize_findings.py now emits schema: hades-blackwatch.findings.v1 after the rename. Existing historical reports, dashboards, or downstream consumers may still expect a previous skill schema, but no compatibility note or alias is documented.",
"observed_output": "schema: hades-blackwatch.findings.v1",
"legacy_output": "previous skill schema",
"expected_output": "schema: hades-blackwatch.findings.v1 with documented migration behavior for older consumers"
}Downstream systems or report consumers route, validate, or store findings based on the schema value. An undocumented schema migration can cause failed validation, incorrect ownership, broken dashboards, or mixed historical data after the skill rename.
Agentic review workflows depend on machine-readable handoffs. Undocumented schema changes weaken provenance and can cause downstream agents or integrations to misinterpret the evidence source.
As of 2026-05-28
Agentic security workflows rely on provenance, source identity, and repeatable machine-readable handoffs. Undocumented schema migrations reduce trust in downstream automation and audit evidence.
{
"mitre_atlas": "Defense evasion through tooling inconsistency",
"cwe": "CWE-1104"
}Assert normalize_findings.py emits hades-blackwatch.findings.v1 or a documented compatibility alias.
For Hades Blackwatch Skill Repository, Hades rename changed the normalized finding schema without compatibility notes can be understood as a control failure where untrusted influence reaches finding normalization and downstream interoperability and can affect normalized findings consumed by reports, CI, or downstream tooling. Downstream systems or report consumers route, validate, or store findings based on the sche...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into finding normalization and downstream interoperability.
Downstream systems or report consumers route, validate, or store findings based on the schema value. An undocumented schema migration can cause failed validation, incorrect ownership, broken dashboards, or mix...
The unsafe path can affect normalized findings consumed by reports, CI, or downstream tooling.
Run normalize_findings.py against examples/scanner-findings.example.json and assert the schema field equals hades-blackwatch.findings.v1.
Influence finding normalization and downstream interoperability so the workflow reaches normalized findings consumed by reports, CI, or downstream tooling in an unintended way.
Agentic review workflows depend on machine-readable handoffs. Undocumented schema changes weaken provenance and can cause downstream agents or integrations to misinterpret the evidence source.
Missing or incomplete deterministic enforcement at the trust boundary.
normalized findings consumed by reports, CI, or downstream tooling
Run normalize_findings.py against examples/scanner-findings.example.json and assert the schema field equals hades-blackwatch.findings.v1.
Document the Hades Blackwatch schema in references/report-format.md. If backward compatibility is needed, add schema_aliases or schema_version fields for previous skill schemas and explain how old reports should be handled.
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Supply Chain / Model and Tooling Integrity",
"owasp_agentic": "Agentic Supply Chain / Tool Contract Drift",
"mitre_atlas": "Defense evasion through tooling inconsistency",
"nist_ai_rmf": "Map, Measure, Manage",
"cwe": "CWE-1104",
"cve": null
}{
"system": "AIVSS",
"value": "Medium",
"rationale": "The issue affects integration integrity rather than direct exploitability. It can still break automated review pipelines and evidence provenance during migration."
}Document the Hades Blackwatch schema in references/report-format.md. If backward compatibility is needed, add schema_aliases or schema_version fields for previous skill schemas and explain how old reports should be handled.
Run normalize_findings.py against examples/scanner-findings.example.json and assert the schema field equals hades-blackwatch.findings.v1.
Previously generated reports may still contain old schema identifiers. Migration notes should clarify that old reports are historical artifacts.
HB-DET-006 - manual source review, local script smoke tests
test and release assurance
scripts/generate_report.py; scripts/normalize_findings.py; scripts/mcp_manifest_audit.py; scripts/surface_mapper.py
Not provided.
future reliability of security review outputs
{
"summary": "Local smoke checks passed for skill validation, surface mapping, MCP manifest audit, prompt pack generation, normalization, and report generation. However, these checks are not codified as a test suite or CI gate.",
"successful_checks": [
"quick_validate.py returned Skill is valid",
"mcp_manifest_audit.py classified run_shell_command as T4",
"normalize_findings.py produced one normalized finding",
"generate_report.py produced HTML and PDF outputs"
],
"gap": "No committed tests assert redaction, report generation, schema name, surface exclusion, nested MCP schemas, or PDF/HTML output integrity."
}A future change regresses redaction, report escaping, schema names, surface exclusions, or MCP risk classification. Without automated tests, the regression may be discovered only after a report is shared or a review misses a risky tool.
Security skills become reusable agent infrastructure. Regression in the skill can mislead future security reviews and downstream agent decisions.
As of 2026-05-28
Reusable AI security skills become supply-chain components. Automated regression gates are needed so redaction, MCP risk classification, schema identity, and report rendering do not silently weaken over time.
{
"mitre_atlas": "Defense evasion through tooling weakness",
"cwe": "CWE-1059"
}Add unittest or pytest coverage plus a CI workflow for the security-critical script paths.
For Hades Blackwatch Skill Repository, Smoke checks are manual and not enforced as automated regression tests can be understood as a control failure where untrusted influence reaches test and release assurance and can affect future reliability of security review outputs. A future change regresses redaction, report escaping, schema names, surface exclusions, or MCP risk classification. Without automated tests, the re...
Untrusted input or context reaches the reviewed workflow.
The workflow passes data into test and release assurance.
A future change regresses redaction, report escaping, schema names, surface exclusions, or MCP risk classification. Without automated tests, the regression may be discovered only after a report is shared or a...
The unsafe path can affect future reliability of security review outputs.
Create tests/test_normalize_findings.py, tests/test_generate_report.py, tests/test_mcp_manifest_audit.py, and tests/test_surface_mapper.py with the current examples plus security fixtures.
Influence test and release assurance so the workflow reaches future reliability of security review outputs in an unintended way.
Security skills become reusable agent infrastructure. Regression in the skill can mislead future security reviews and downstream agent decisions.
Missing or incomplete deterministic enforcement at the trust boundary.
future reliability of security review outputs
Create tests/test_normalize_findings.py, tests/test_generate_report.py, tests/test_mcp_manifest_audit.py, and tests/test_surface_mapper.py with the current examples plus security fixtures.
Add a standard-library unittest or pytest suite for redaction, normalization schema, HTML escaping, PDF generation, surface exclusions, and recursive MCP schema risk classification. Add a CI workflow to run skill validation and tests on every change.
Generated from the finding evidence, attack path, affected component, affected asset, fix, and regression test.
Conceptual defensive flow only. No exploit payloads or unauthorized testing steps are provided.
{
"owasp_llm": "Supply Chain Vulnerabilities",
"owasp_agentic": "Agentic Supply Chain / Tool Reliability",
"mitre_atlas": "Defense evasion through tooling weakness",
"nist_ai_rmf": "Measure, Manage",
"cwe": "CWE-1059",
"cve": null
}{
"system": "AIVSS",
"value": "Low",
"rationale": "This is an assurance gap, but it affects security-critical review tooling."
}Add a standard-library unittest or pytest suite for redaction, normalization schema, HTML escaping, PDF generation, surface exclusions, and recursive MCP schema risk classification. Add a CI workflow to run skill validation and tests on every change.
Create tests/test_normalize_findings.py, tests/test_generate_report.py, tests/test_mcp_manifest_audit.py, and tests/test_surface_mapper.py with the current examples plus security fixtures.
Automated tests should be supplemented by manual review for new scanner adapters, new MCP integrations, and report template changes.
| ID | Severity | Finding | Recommended Fix |
|---|---|---|---|
| HB-DET-001 | High | Raw scanner evidence can be copied into shareable HTML/PDF reports without secret redaction | Add a recursive redaction layer before storing normalized evidence and before rendering reports. Redact key names and values matching Authorization, Cookie, Set-Cookie, token, secret, password, api_key, bearer, private_key, session, credential, client_secret, and configurable organization patterns. Prefer a safe evidence summary allowlist over storing full raw scanner items by default. |
| HB-DET-003 | Medium | MCP manifest auditor can under-classify nested permissive tool schemas | Walk input schemas recursively and report exact JSON paths. Flag additionalProperties true or absent on object-like inputs, nested sensitive parameter names, arrays of action objects, missing required fields, broad string parameters, and URL/path/command-like fields without enums, allowlists, regex constraints, or approval language. |
| HB-DET-002 | Medium | Surface mapper scans generated reports and previous report inputs, polluting later analysis | Add reports/, out/, coverage/, .pytest_cache/, .mypy_cache/, and other generated-output paths to DEFAULT_SKIP_DIRS. Add --include-generated and --exclude options for controlled overrides. Consider reading .gitignore patterns for default exclusions. |
| HB-DET-005 | Low | Documentation describes scanner auto-detection before the registry implementation exists | Adjust wording to say Hades Blackwatch is designed to support scanner detection and currently supports local scripts, report ingestion, examples, and role guidance. Keep auto-detection language only after scanner_registry.py exists and is tested. |
| HB-DET-004 | Low | Hades rename changed the normalized finding schema without compatibility notes | Document the Hades Blackwatch schema in references/report-format.md. If backward compatibility is needed, add schema_aliases or schema_version fields for previous skill schemas and explain how old reports should be handled. |
| HB-DET-006 | Low | Smoke checks are manual and not enforced as automated regression tests | Add a standard-library unittest or pytest suite for redaction, normalization schema, HTML escaping, PDF generation, surface exclusions, and recursive MCP schema risk classification. Add a CI workflow to run skill validation and tests on every change. |
| ID | Test / Control |
|---|---|
| HB-DET-001 | Create a fixture with nested request and response objects containing Authorization, Cookie, Set-Cookie, api_key, password, bearer token, private key markers, and session IDs. Run normalize_findings.py and generate_report.py, then assert none of the secret values appear in normalized JSON, HTML, or PDF bytes. |
| HB-DET-003 | Add a manifest fixture with inputSchema.properties.config.properties.command, inputSchema.properties.request.properties.url, nested headers/body fields, and additionalProperties true. Assert the auditor returns T4 and reports each nested JSON path. |
| HB-DET-002 | Create reports/generated.html containing token, MCP, credential, and authorization terms. Run surface_mapper.py and assert reports/generated.html is excluded by default. Then run with an explicit include-generated option and assert it appears. |
| HB-DET-005 | Add a docs consistency checklist or test that verifies README capability claims match implemented scripts and roadmap status. |
| HB-DET-004 | Run normalize_findings.py against examples/scanner-findings.example.json and assert the schema field equals hades-blackwatch.findings.v1. |
| HB-DET-006 | Create tests/test_normalize_findings.py, tests/test_generate_report.py, tests/test_mcp_manifest_audit.py, and tests/test_surface_mapper.py with the current examples plus security fixtures. |
| ID | Framework | Mapping |
|---|---|---|
| HB-DET-001 | owasp_llm | Sensitive Information Disclosure |
| HB-DET-001 | owasp_agentic | Information Disclosure through Tool and Context Handling |
| HB-DET-001 | mitre_atlas | Credential Access / Exfiltration through tool-output handling |
| HB-DET-001 | nist_ai_rmf | Map, Measure, Manage |
| HB-DET-001 | cwe | CWE-200; CWE-532 |
| HB-DET-003 | owasp_llm | Excessive Agency; Insecure Plugin Design |
| HB-DET-003 | owasp_agentic | Tool Misuse; Agentic Supply Chain |
| HB-DET-003 | mitre_atlas | Tool abuse and execution via agent capability |
| HB-DET-003 | nist_ai_rmf | Map, Measure, Manage |
| HB-DET-003 | cwe | CWE-20; CWE-749 |
| HB-DET-002 | owasp_llm | Context/Data Poisoning analogue for review evidence |
| HB-DET-002 | owasp_agentic | Evidence Integrity / Context Poisoning |
| HB-DET-002 | mitre_atlas | Data poisoning and evasion-adjacent evidence manipulation |
| HB-DET-002 | nist_ai_rmf | Measure, Manage |
| HB-DET-002 | cwe | CWE-703 |
| HB-DET-005 | owasp_llm | Overreliance / Supply Chain Tooling Gap |
| HB-DET-005 | owasp_agentic | Tool Reliability / Coverage Gap |
| HB-DET-005 | mitre_atlas | Defense evasion via false assurance |
| HB-DET-005 | nist_ai_rmf | Govern, Map, Measure |
| HB-DET-005 | cwe | CWE-1059 |
| HB-DET-004 | owasp_llm | Supply Chain / Model and Tooling Integrity |
| HB-DET-004 | owasp_agentic | Agentic Supply Chain / Tool Contract Drift |
| HB-DET-004 | mitre_atlas | Defense evasion through tooling inconsistency |
| HB-DET-004 | nist_ai_rmf | Map, Measure, Manage |
| HB-DET-004 | cwe | CWE-1104 |
| HB-DET-006 | owasp_llm | Supply Chain Vulnerabilities |
| HB-DET-006 | owasp_agentic | Agentic Supply Chain / Tool Reliability |
| HB-DET-006 | mitre_atlas | Defense evasion through tooling weakness |
| HB-DET-006 | nist_ai_rmf | Measure, Manage |
| HB-DET-006 | cwe | CWE-1059 |
Residual risk depends on complete remediation, regression test adoption, scanner coverage, evidence redaction, and continued enforcement of deterministic authorization around model-driven actions.