Track three clocks in the ingestion pipeline
Source change time, processing time and query visibility describe different stages. Keeping all three makes stale information easier to diagnose.
Read articleCobnex blogs / Global
Cobnex blog
AI implementation, software architecture and cloud operations for teams worldwide.
146 articles in AI engineering
Page 2 of 9
Validate the proposed action outside the model
A well-formed tool call can still be unauthorised. Build the executor around the user's permitted task, not the model's explanation for its request.
Read articleDefine the smallest order change an agent may submit
Build a delivery-note tool by separating generated input, business validation and the durable write. Each layer answers a different question.
Read articleCreate the proposal before requesting approval
Save a complete proposed change, render the review from that record and execute from the same record. This removes a common gap between what people see and what systems submit.
Read articlePersist the workflow step before continuing the dialogue
Model the next business action as durable state. A restarted worker should know whether to dispatch, reconcile or report a result without asking the model to guess.
Read articleWrite the expected behaviour before tuning the prompt
Start each evaluation case with a clear decision about success. This prevents the current model's answer from quietly becoming the standard it is judged against.
Read articleVersion the prompt and tools with the model
A model identifier alone cannot reproduce an application response. Capture the configuration that shapes its inputs, available actions and accepted outputs.
Read articleDraw the request path before choosing the region
Start with every service that receives task data. A regional model endpoint does not describe the location or access rules of the rest of the pipeline.
Read articleAttach one budget to the whole task
Give every model call and tool attempt the same task identity, then reserve capacity before dispatch. This makes cost controls work across retries and concurrent workers.
Read articleDesign three outcomes for missing evidence
Give the application explicit answer, clarification and escalation states. Keep access denials separate so uncertainty handling cannot bypass permissions.
Read articleRevoke access while an answer is still cached
A permission test should include the period after access changes. This walkthrough checks fresh requests, cached answers and conversations that were already open.
Read articleAlmost the same product code is the wrong product
Similar manuals are a useful stress test for semantic retrieval. Build examples where a one-character difference changes the correct answer.
Read articleTest the table that continues on the next page
Multi-page tables expose mistakes that ordinary paragraph tests miss. Check the relationship between a value, its column heading and the note that qualifies it.
Read articlePlant a real citation beside an unsupported exception
A useful citation test uses a relevant source and a subtly wrong answer. This exposes failures that broken-link checks cannot find.
Read articleDelete a source that has already produced an answer
Test withdrawal after the system has created passages, caches and conversation history. An empty search result alone does not prove removal is complete.
Read articleHide an export instruction inside a supplier file
Exercise the document-to-tool boundary with a synthetic file and a harmless test destination. Measure what the application does, not only what the assistant says.
Read articleTimeout after the write, before the tool response
The hardest retry case is a successful mutation whose response never reaches the agent. Inject that failure deliberately and inspect the business record.
Read articleChange one field after the reviewer approves
A small payload mutation is enough to test whether approval is bound to the action. Use a material field and verify that execution cannot reuse the earlier decision.
Read article