open-source · since 2026-10-30
Analytics Agent
TonyLee-99
Open-source text-to-SQL analytics agent with a governed metric layer, confidence-based abstention, read-only SQL guardrails, PII pseudonymization, FastAPI/MCP serving, and reproducible BIRD Mini-Dev plus Olist evaluations.
Initial audit 2026-10-30: repository reports 62.6%→73.2% execution accuracy on 500 BIRD questions; 14/30→27/30 governed business-metric questions with prompt definitions and 29/30 with compiled metrics; 86.3% accuracy at 69.5% coverage under unanimous five-sample abstention. It explicitly reports negative results: confidence AUROC 0.69 missed its 0.70 target, self-correction was not significant, and confidence passed 5/8 wrong-definition answers. Safety suite reports 64/64 attacks blocked and 0/550 false positives after five fixes. Author-run evidence; no independent adoption yet.