Jim Booth — AI Infrastructure Engineering

Live · In production

01 · Production system

A multi-agent system that runs a business unattended

Autonomous agents sourcing, writing, verifying, and packaging real work 24/7 — with a watchdog, human approval gates, and daily production testing

Not a demo. A production system where a reasoning agent walks each job through a state machine every ten minutes: it sources live material, evaluates each option against the goal, verifies every external link is genuinely live, generates the tailored work products, and assembles the package. A watchdog restarts it if it dies. A separate agent browser-tests the whole pipeline in production every day, forced-failure cases included.

The interesting part isn't wiring a model to tools — that takes an afternoon. It's the reliability work demos never show: money-touching correctness, self-diagnosing failure telemetry, and guardrails that forbid the agent from the two irreversible actions. Knowing where to put the human is as much of the engineering as the automation.

Why this matters

Every company is standing up an agent initiative and stalling at the wall between a working demo and a system that survives production. I've already paid that tuition, with real transactions on the line. This is the far side of that wall.

Autonomous agents State machine launchd / cron Human-in-the-loop Production testing Reliability engineering

01 · Architecture — the loop, the watchdog, and the gate the agent can't cross

Read the full case study ↗

In progress · Ongoing

02 · Personal research

From Watts to Weights

Closing the loop between sizing infrastructure-up and model-down

I've been sizing AI factories from the bottom up — power envelope, cooling capacity, rack density. A modern AI-cluster customer needs the other end of that equation: model parameters, parallelism strategy, GPU count, interconnect topology.

This is the synthesis I've been building — seven diagrams walking the layers of an AI cluster from compute up through fabric and back down through storage and OS. The full chain a customer needs to size, and the seams where infrastructure-up thinking meets model-down thinking.

Why this matters

Most can do the parallelism math. Very few can connect it back to choices of RDHX, PG-25 liquid cool, or two-phase DTC cooling. The conversation a customer building an AI factory actually needs runs the full chain — from model arithmetic through cluster topology to power and cooling. I want to be able to hold both ends of that conversation.

NVLink / NVSwitch InfiniBand / RDMA 3D Parallelism NCCL GPUDirect Storage DGX SuperPOD GB200 NVL72 Magnum IO

01 · Inside the GPU — the compute layer

02 · The network fabric — scale-up meets scale-out

03 · Mapping parallelism to the fabric

04 · Feeding the cluster — storage

05 · Storage in depth

06 · The bypass — getting bytes into HBM

07 · Under the hood — the Linux systems layer

Drag → to see all 7 diagrams

Full writeup with all seven layers ↗

Complete · Writeup published

03 · Personal research

Fine-Tuning Llama-3.1 for AI Infrastructure

LoRA · 0.52% of parameters touched · 22 seconds on one H100

How much can a small, fast LoRA fine-tune shift a general model toward a specialized domain? Synthetic AI-infrastructure Q&A dataset, Llama-3.1-8B base, rank-16 LoRA on a single H100 via Brev. Eighteen optimizer steps, twenty-two seconds of wall clock. Loss came down cleanly. Eval numbers looked fine.

The interesting part wasn't the numbers. I asked the tuned model to explain CDU sizing for a 150 kW GPU rack — a domain I know well from technical sales work on two-phase cooling at Accelsius. Despite five clean CDU examples in my training data, the model confidently answered with inrush current and electrical-overload concepts. Pure category error: that's PDU reasoning, not CDU reasoning. Llama's pretraining corpus had millions of co-occurrences of sizing + kW + rack with electrical concepts. My LoRA saw five. The prior won by orders of magnitude.

What it taught

Fine-tune for form, RAG for facts. LoRA shifts behavior — style, format, domain fluency, task specialization. It's a weak tool for installing facts that contradict strong pretraining priors. NVIDIA's NeMo Framework + NeMo Retriever stack is designed for exactly this combination — and it's the conversation NCP customers asking "should we fine-tune Nemotron?" actually need to have.

Llama-3.1-8B LoRA rank-16 PEFT Unsloth H100 / Brev vLLM NVIDIA NeMo RAG

01 · Training loss — 18 steps, 22 sec wall clock

02 · PDU vs CDU — the diagnostic finding

03 · What fine-tuning changed — and didn't

04 · Pipeline architecture

05 · Fine-tune vs RAG — when to use which

06 · Evaluation methodology

07 · Terminal output — training run

08 · Base vs fine-tuned — response comparison

09 · Where the tools sit — research vs NCP production

Drag → to see all 9 slides

Full writeup with code & eval methodology ↗

In progress

04 · Personal research

AI Agent From Scratch

Working through Manning's Build an AI Agent (From Scratch)

The natural next layer up from the LLM build. Same philosophy: no LangChain, no AutoGen, no framework abstractions hiding the mechanics. Tool-use loops, planning, memory, and evaluation — built from primitives so I understand what the popular agent frameworks are actually doing under the hood.

The agent so far handles multi-step tool use, simple planning over an action space, and short-term memory across turns. Each component goes in by hand before any abstraction layer goes on top.

Why it matters

AI infrastructure conversations are shifting from "how big a model can we run?" to "how does this agentic workload behave under load?" The two are very different problems. Building agents from scratch is the only way to answer the second one honestly.

Python Anthropic API Tool use ReAct loop Agent memory Eval harness

Live · Running 24/7

05 · Personal research · Dry-run mode

Prediction Market Trading Engine

Multi-strategy Python engine · Kalshi & Polymarket

Prediction markets are one of the few public order books where retail can compete with institutions on price discovery — narrow products, clean data, no insider information. I wanted a sandbox where my code's decisions had real consequences.

The engine runs six concurrent strategies under launchd: an arbitrage scanner crossing Kalshi and Polymarket on the same underlying events, a passive market-maker quoting around microprice fair-value with quarter-Kelly sizing, and weather-contract trading driven by calibrated NWS forecast models with regional bias corrections. Settlement, take-profit, and reinvestment all happen automatically.

Why this matters

It's been my best classroom for understanding what AI infrastructure actually has to support. Latency budgets matter when prices move in milliseconds. Memory access patterns matter when you hold state across thousands of markets. Observability matters when something's bleeding and you don't know which strategy. Most of what I sell becomes concrete when you have to operate it yourself.

Python asyncio Kalshi API Polymarket NWS forecast models launchd Quarter-Kelly sizing Microprice FV

Live operator dashboard — positions, P&L, signal feed — 01 · Operator dashboard — positions, P&L, signal feed

Strategy signals — logging output across strategies — 02 · Strategy signals — live logging

Crypto research module — new strategy in development — 03 · Crypto funding research — new strategy

Drag → to see more

Complete

06 · Personal research

LLM From Scratch

A small transformer · PyTorch · No framework shortcuts

Sitting across the table from CTOs while I sold GPU clusters and high-density cooling, I kept feeling the gap between knowing what ran on the hardware and knowing how it ran. Reading papers wasn't enough. I had to build one.

A small transformer in PyTorch, layer by layer — BPE tokenizer, multi-head self-attention written from the math, residual connections, layer normalization, and the full training loop. The model that came out is small and useful mostly for proving the underlying mechanics work. The understanding that came out has shown up in every cooling-and-compute conversation since.

What it taught

Attention is a memory-bandwidth problem before it's a compute problem. Context length is a quadratic cost. Pretraining is mostly babysitting. These are the things that turn into "what's your inference budget?" questions in a sales call — and I can now answer them from the inside.

PyTorch BPE tokenizer Self-attention GPT-2 architecture Mixed precision CUDA

01 · Activation Functions — GELU vs. RELU

02 · Causal Attention Code - No previous reasoning

Drag → to see more

Thirty years in infrastructure.
Still building from scratch.

Build to understand.