Measure coverage of the data-flow inventory
A completed architecture diagram is not evidence that every payload path is understood. Track which paths have been observed, configured and tested against their requirements.
Read articleCobnex blogs / Global
Cobnex blog
AI implementation, software architecture and cloud operations for teams worldwide.
146 articles in AI engineering
Page 4 of 9
Report the expensive tail of AI task costs
Typical cost is useful for planning, but rare long runs can dominate expenditure. Show the distribution, unsuccessful work and the task types behind the tail.
Read articleCount unnecessary refusals as well as unsafe answers
A stricter assistant can look more accurate by answering fewer questions. Measure the quality of answered cases and the useful work lost through abstention.
Read articleInvestigating a suspected retrieval leak
When an assistant appears to reveal a restricted document, contain the affected path and preserve enough evidence to find where access was lost.
Read articleDiagnose a relevance regression from both candidate lists
When search gets worse, identify the stage that changed. Preserve the question, source versions and branch results before tuning the final answer.
Read articleTrace a misleading answer back to its chunk
Before changing the prompt, inspect the evidence the model received. A missing heading or broken table can turn accurate source text into a misleading answer.
Read articleHandle a citation that staff can no longer open
A failed source link may indicate a moved document, changed access or missing historical evidence. Establish which one happened before changing the answer.
Read articleFind the layer still serving the old policy
Diagnose a stale answer by following its source revision through the serving path. Rebuilding the whole index should not be the first response.
Read articlePreserve the document that changed agent behaviour
When an assistant attempts an unexpected action, retain the relevant evidence and contain the capability. Editing the prompt immediately can make the incident harder to understand.
Read articleExplain why an agent run stopped
A stopped run may be waiting for approval, out of budget or uncertain about a completed write. Give operators a state they can act on.
Read articleRecover an approved proposal with an uncertain submission
Approval confirms permission to act. It does not tell you whether a timed-out execution already happened. Resolve that uncertainty before submitting again.
Read articleResume the run whose approval expired overnight
A saved checkpoint preserves progress, but it does not freeze authority or business state. Inspect completed work before deciding what can continue.
Read articleInvestigate disagreement between human evaluators
Reviewer disagreement may reveal an unclear question, missing evidence or an ambiguous scoring rule. Resolve the cause before treating one label as ground truth.
Read articleRestore the previous behaviour after a model regression
Confirm the affected configuration and contain new failures before changing prompts in place. A known prior release is easier to assess than an unrecorded emergency variation.
Read articleGive the reviewer the contradiction, not the whole queue
A useful exception record says which fields disagree and shows their evidence. That lets reviewers resolve the problem without repeating the extraction process manually.
Read articleInvestigate a payload found in unexpected diagnostics
Establish which data was copied, who could access it and which path created it. Contain further logging while preserving the evidence needed to assess the incident.
Read articleStop an over-budget run with a recoverable status
Prevent new work, account for operations already accepted and preserve the useful result. Cancelling the conversation alone may leave costs and effects running in the background.
Read articleRoute an unresolved answer to someone who can help
A handoff should carry the unanswered question and evidence already checked. Make the next owner clear without promising work the application has not actually submitted.
Read article