AI & workflow
After the AI pilot: when the desk still belongs to humans
Generative tools compress drafting latency; they do not by themselves compress verification. The durable desk pattern treats the model as a fast research assistant whose every sentence must point at ground truth—filings, feed ids, calculator snapshots—or be cut. When prose is cheap, provenance becomes the scarce asset; committees should reward people who slow down to attach receipts.
What actually changed
Drafting speed skyrocketed; verification workload often follows. Teams that skip verification inherit elegant lies.
Capacity planning mistake: staffing for old memo throughput without adding reviewer hours—watch queue age, not word count shipped.
Grounding patterns
Require each paragraph of machine summary to link to object ids: filing accession, data pull id, or calculator snapshot. If a link cannot be produced, the paragraph does not ship.
Use structured summaries: claim → evidence handle → confidence class (high/medium/speculative). Speculative without flag is a defect.
Human review economics
Batch machine output by risk: client-facing, IC-grade, internal scratch—different review depth per class. Do not pretend everything needs the same eyes.
Pair junior drafting with senior spot-check on highest-risk pages—invert the old pyramid if juniors lean on models hardest.
Prompt and model inventory
Version prompts like APIs; log model family and temperature for outputs that leave the building. ‘We used ChatGPT’ is not an audit trail.
Maintain a kill list of prompt patterns that historically hallucinate in your domain—ban them from external paths.
When models suggest numbers
Treat suggested figures as untrusted until replayed through deterministic desk tools. AI can propose parameters; the calculator owns the result.
Log parameter suggestions separately from accepted runs—future disputes distinguish ‘model idea’ from ‘desk decision.’
Training and culture
Reward catching errors in drafts; punish shipping unverified claims even if fast. Speed without trace is liability.
Celebrate engineers who tighten grounding UI—better checkboxes beat another policy PDF.
Vendor and data rights
Understand whether prompts and outputs may be retained by third parties; client NDAs may prohibit certain routes.
If vendors train on customer uploads by default, route sensitive drafts through enterprise contracts with explicit retention.
Red-team sampling
Weekly random sample of AI-assisted paragraphs; label grounded, over-generalized, or wrong; trend by desk and model version.
Spikes correlate with model upgrades—treat upgrades like feed migrations with replay tests.
Metrics
Track correction rate on AI-assisted memos, time saved vs verification added, and incident count tied to missing grounding.
Add ‘rescued before external’ count—organizations undervalue invisible quality work until it vanishes.
Public desk routes referenced above
Same URL discipline applies after you authenticate—bookmarkable paths, no tenant data on these pages.
- Public calculator hub GET-shareable patterns you can mirror for every material desk run; use as the pattern for replay links.
- Operating rhythm Template language for stages, owners, and completion rules in a mandatory morning loop.
- Product atlas Route-level map so reviewers can open the exact module that produced a figure.
- Guides Pair public guides with internal SOPs so training points at one narrative spine.
- Platform framing How tenancy and public chrome split when SEO and auth walls must coexist.
- Terms of use Stable legal URL when client-facing calculator links depart the marketing host.
- Privacy policy Maps subprocessors and flows; keep language consistent with how you split marketing and tenancy.
Operator toolkits
Checklists and scripts you can lift into runbooks, vendor RFPs, or board appendices.
AI governance patterns for desk teams; not a statement on model safety in general.