| Section | Detail |
|---|---|
| 1. Tool Overview | Purpose: Automatically track, verify, and score the delivery of ministerial promises against public spending and audit data. Deployment Context: Used by National Transparency Operations Centre (NTOC) analysts as part of the TallySticks transparency platform. |
| 2. Rationale for AI Use | Problem: Manual review of thousands of Hansard statements, FOI disclosures, and contracts is impractical and error-prone. AI Benefit: Enables near real-time accountability and surfacing of divergence between promises and spending. Alternatives Considered: Manual audits, off-the-shelf BI dashboards. Rejected due to latency and lack of constitutional rule integration. |
| 3. Technical Specification | Models: Claude 3.5 Sonnet, Gemini 1.5 Pro deployed through private Google Cloud Vertex AI instances; no public API usage. Inputs: Hansard transcripts, Contracts Finder data, FOI responses, Treasury datasets (CSV/JSON). Outputs: Accountability Score (0-100), Opacity Score (0-100), Divergence Alerts with linked evidence. |
| 4. Human in the Loop | The system provides decision support only. All findings are reviewed by TallySticks analysts or designated public officers before publication. No automated legal or financial actions occur without human sign-off. |
| 5. Data Sources & Processing | Source Integrity: All external data is sourced from authenticated government endpoints (Hansard, Contracts Finder, WhatDoTheyKnow) and cached in an encrypted Golden Dataset. Processing: Data passes through the Promise Tracker pipeline (ingestion → promise normalization → constitution scoring). |
| 6. Fairness & Bias Controls | Utility Belt Check (Victory #146): Ensures each agent has correct tools, certificates, and read-only constitutional knowledge before execution. Rainy Day Protocol (Victory #147): Detects recurrent data-access issues and prevents runaway retries that could bias results. |
| 7. Risks & Mitigations | Hallucinated Contracts: Mitigated by cross-checking every AI assertion against the Golden Dataset with cryptographic hashes. Model Drift: Monthly regression suite comparing LLM outputs to historical "gold" annotations. Unauthorized Data Transfer: Prevented through the Constitutional Air-Gap—no sensitive data leaves the council's cloud tenancy. |
| 8. Appeals & Challenge Process | Public authorities can challenge any score by submitting additional evidence via the Transparency Portal. Each challenge is logged, triaged by NTOC analysts, and, if upheld, results in a published audit addendum. |
| 9. Accountability & Contact | Senior Responsible Owner: Kenneth J. Pringle, Director, TallySticks UK CIC. Transparency Portal: https://tallysticks.uk/transparency (ATRS record publicly accessible). |