How the catalog works: a catalog of prepared talks across different audience levels; a company selects a session and it’s delivered remotely or on-site. Each session draws on production work — architecture decisions with their rationale, failure modes met in practice, and evaluation data from systems actually built and measured. The material takes positions; it is not a neutral survey.
Executive / Leadership Sessions (60–90 minutes)
The Four Ways Companies Use AI
…and why the one that pays the most is the one nobody’s selling you
There are four fundamentally different ways a company puts AI to work, and they do not pay off the same way. Most organizations are heavily invested in the two that produce the least return and have barely touched the one that produces the most — not by choice, but because the highest-value use is the one that can’t be demonstrated in a meeting, so no vendor ever shows it to them. This session maps the four modes in plain terms, shows with a live example where the money actually is, and explains why it’s there, what it costs to capture, and the one question to ask of any process before spending a dollar on AI to improve it. No hype, and a straight answer on where AI is the wrong tool.
Key takeaway: one question to sort high-return AI work from low-return AI work — can this process be boxed? Format: 60 min (standard) or 90 min (adds a live worked example on the room’s own numbers). One live demo, one-page leave-behind.
Engineering / Technical Sessions (90 minutes → full-day)
“AI Is a Component, Not a Product“
Architecture patterns for building systems where the model is one fallible part
Most AI systems fail in the same place: a team treats the language model as the product, ships a demo that worked on their laptop, and watches it break the first time the model is wrong at scale. This is the opposite discipline. It treats the LLM as one fallible component inside a deterministic system and walks — with live examples on a real pipeline — through everything that surrounds it: designing around how the model actually behaves, splitting the model’s job from the code’s, decomposing a task so it runs cheaper and more accurately, measuring confidence when there’s no answer key, routing for cost, building an evaluation harness whose green result means something, and knowing when an agent is the wrong tool. Built from production systems, not slides about possibility.
Key takeaway: the deterministic-first, decomposed, measured pipeline pattern — applicable Monday — plus a checklist of the ways these systems fool the people building them. Formats (a ladder): 90-min engineering overview (the lead; the session leadership books for their engineers after the exec talk) · half-day core · four-part series (4×90 min) · full-day intensive.
“Evaluating AI: How to Know If Your System Actually Works“
Most teams evaluate an AI system against their own test set, watch it go green, and ship — then it fails in production on exactly the cases the harness never really tested. This deep-dive is about building an evaluation harness whose green result means something: walling the answer key off from the system, planting your own errors and scoring blind, treating variance as a first-class metric, and using LLM-as-judge without letting it grade its own work. Built from production systems where a trusted-but-wrong eval was worse than no eval at all.
Key takeaway: an evaluation harness you can trust, and a catalog of the specific ways harnesses lie. Format: 90-min or half-day.
“You Can’t Ask a Model How Sure It Is“
Measuring confidence without ground truth
The single output your system just produced, in production, with no answer key — how much do you trust it? You cannot ask the model; its self-reported confidence reflects fluency, not accuracy. This deep-dive is about where trustworthy confidence actually comes from: agreement across independent readers, computed from behavior rather than elicited from the model. It covers the traps that manufacture confident, meaningless answers, and makes the case that a confident wrong answer is the failure the whole system exists to prevent. Built from systems that had to grade their own output with no key to check against.
Key takeaway: a method for computing per-output confidence from behavior, and a catalog of the ways teams fool themselves into trusting output they never verified.
“Agentic AI Architecture: When to Hand the Model the Control Flow”
Everyone is being sold agents, and a large share of agentic projects are being canceled — not because the models fail, but because teams hand control to the model on work that never needed it. This deep-dive draws the line precisely: what an agent actually is (the model owns the control flow), when that’s the right trade, what it costs that no one mentions, and how to build one with the discipline that keeps it from becoming a cancellation statistic. The honest version of the most hyped topic in AI — a right-tool talk, not an anti-agent talk.
Key takeaway: a decision rule for agent-vs-workflow, the real cost model, and the boundary-layer discipline an agent needs to survive production.