https://ugolattanzi.com/en/writing/ Essays on putting generative AI into production: when not to automate, how to test an LLM system, and what actually changes when you measure it. en Sat, 29 Aug 2026 00:00:00 +0000 https://ugolattanzi.com/en/writing/human-in-the-loop/ https://ugolattanzi.com/en/writing/human-in-the-loop/ Sat, 29 Aug 2026 00:00:00 +0000 "Let us put a human in the middle" can mean five different things, with opposite costs and guarantees. The patterns, and how to tell whether review works. https://ugolattanzi.com/en/writing/llm-as-judge/ https://ugolattanzi.com/en/writing/llm-as-judge/ Thu, 20 Aug 2026 00:00:00 +0000 Using one model to evaluate another works, but only once you have calibrated it against human judgement. The variants, the documented biases, the real cost. https://ugolattanzi.com/en/writing/golden-dataset-and-test-set/ https://ugolattanzi.com/en/writing/golden-dataset-and-test-set/ Tue, 11 Aug 2026 00:00:00 +0000 Without a set of examples carrying the expected answer, "it works better" stays an opinion. How to build the yardstick that says whether you are improving. https://ugolattanzi.com/en/writing/when-not-to-automate/ https://ugolattanzi.com/en/writing/when-not-to-automate/ Fri, 31 Jul 2026 00:00:00 +0000 The most autonomous system is rarely the right one. Notes on designing AI around the constraint instead of arguing against it. https://ugolattanzi.com/en/writing/how-to-test-an-llm-system/ https://ugolattanzi.com/en/writing/how-to-test-an-llm-system/ Fri, 17 Jul 2026 00:00:00 +0000 Test the arguments a model produces, not its prose. A cheap, deterministic regression suite for retrieval and tool calls — and why it also cuts the bill. https://ugolattanzi.com/en/writing/six-months-to-hours/ https://ugolattanzi.com/en/writing/six-months-to-hours/ Fri, 03 Jul 2026 00:00:00 +0000 How a research body went from a six-month manual process for updating its guidelines to an agentic pipeline that runs in hours.