Hi,
Field Note #3. Today's case: DraftCat, the GitHub issue-triage agent I run on my own repos - and what I learned the hard way about auto-closing.
A triage agent is useful if it gets three things right: labels, duplicates, and a first-draft reply - and leaves the close-button click to a human. I let it auto-close high-confidence duplicates in v1. It was not worth it.
What's inside:
• Why auto-close at confidence > 0.95 still goes wrong (and the public-thread argument that killed it)
• Hybrid duplicate detection: exact-title pre-pass + vector embeddings (cosine ≥ 0.85 over the last 1000 closed issues)
• Three confidence bands (auto / review / defer) - not one threshold
• Idempotency keys on every webhook (because GitHub redelivers, and you double-label without them)
• Full architecture + an 8-step checklist for your own repos
Read the full case →
René