AI Engineer
Building products on top of models you did not train. Thirty-six lessons from what a token is through retrieval, agents, evaluation and safety, to the work of keeping an LLM feature alive in production.
What is an AI engineer?A discipline that builds products on models it did not train, sitting between the ML engineer who trains them and the developer who consumes an API.How LLMs workNext-token prediction, a learned function with billions of parameters, and why the output is a probability distribution rather than a lookup.Tokens and tokenizationSubword tokenization, byte-pair encoding, and why token counts rather than word counts are the unit of both cost and context.EmbeddingsDense vectors whose geometry encodes meaning, where they come from, and what they are and are not good for.The context windowThe window as a budget you spend rather than a setting you configure, what happens at the limit, and why a bigger one is not a strategy.Calling a model APIMessages and roles, the request and response envelope, tokens in and out, and the error handling every production call needs.Sampling parametersTemperature, top-k, top-p and repetition penalties — what each one does to the distribution, and when touching it is justified.Structured outputGetting schema-conforming JSON out of a model reliably, using constrained decoding rather than asking politely and hoping.Choosing a modelOpen weights against closed APIs, size against latency and cost, and why a benchmark leaderboard is the weakest evidence available to you.Prompt anatomyThe parts of a working prompt — role, task, context, constraints, output format — and why structure beats the search for magic words.Few-shot promptingZero-shot against few-shot, how to choose and order examples, and the cases where examples make the output worse.Chain of thoughtGetting a model to work before it answers, from step-by-step prompting to reasoning models that do it internally.Context engineeringDeciding what goes in the window and why — write, select, compress, isolate — and the failure modes of a window packed to the limit.Prompt cachingHow a cached prefix removes repeated prefill work, how to arrange a prompt so the cache actually hits, and what it saves.ChunkingSplitting documents for retrieval, and why chunk boundaries decide retrieval quality more than the embedding model does.Vector databasesWhat a vector index actually does, how HNSW and IVF trade accuracy for speed, and when Postgres with pgvector is enough.Similarity searchScoring, top-k, hybrid search with BM25, and re-ranking — why vector search alone underperforms on real queries.The RAG pipelineIngest, chunk, embed, index, retrieve, re-rank, generate — the whole path, with the failure mode waiting at each stage.RAG or fine-tuningWhich problem each one actually solves, why fine-tune it on our docs is usually the wrong instinct, and when the answer is neither.What is an agent?A model using tools in a loop to reach a goal — where that genuinely helps, and where a fixed pipeline is the better engineering.Tool callingDeclaring tools, executing what the model asks for, feeding results back — and designing tools a model can actually use correctly.The agent loop (advanced)Plan, act, observe, repeat — owning the control loop, deciding when to stop, and why agents stall or run away.Model Context ProtocolThe problem MCP solves, how hosts, clients and servers fit together, and what a standard interface changes about wiring an agent to the world.Multi-agent systemsOrchestrators and sub-agents, handoffs, isolated context — and the honest case that one good agent usually beats five.Why evals matterWhy iterating on vibes stops working, what an eval actually is, and the error-analysis loop that turns failures into measurements.Building an eval setSourcing real failures, labelling them, writing assertions that do not need a model, and keeping the set from going stale.LLM as judgeGrading output with a model, aligning that judge against human labels, and the biases that make an unvalidated judge worse than none.Tracing and observabilityWhy LLM traces are a different problem from uptime monitoring, what a span should carry, and finding the failing step in an agent run.Prompt injectionWhy instructions and data share one channel, direct and indirect injection, the lethal trifecta, and why no complete fix exists.Guardrails and moderationInput and output checks, classifiers and moderation APIs, human review — and an honest account of what guardrails can promise.Bias and fairnessWhere bias enters a model and a product built on one, how it is measured, and what an application engineer can actually do about it.Privacy and data handlingHow user data moves through prompts, logs and retrieval, what provider terms actually say, and designing a system that does not leak by default.Fine-tuningFull fine-tuning against LoRA, what adaptation teaches and what it does not, and the real cost of owning a tuned model.Multimodal AIHow images, audio and video reach a language model, what changes in an application when the input is not text, and where the costs move.Cost and latencyToken economics, time to first token, caching, routing and batching — the levers that decide whether an LLM feature is affordable.Shipping and monitoringRollout and fallbacks, provider outages and rate limits, versioning prompts and models, and watching quality after launch.
What you are actually building on
- What is an AI engineer?A discipline that builds products on models it did not train, sitting between the ML engineer who trains them and the developer who consumes an API.
- How LLMs workNext-token prediction, a learned function with billions of parameters, and why the output is a probability distribution rather than a lookup.
- Tokens and tokenizationSubword tokenization, byte-pair encoding, and why token counts rather than word counts are the unit of both cost and context.
- EmbeddingsDense vectors whose geometry encodes meaning, where they come from, and what they are and are not good for.
Talking to a model
- The context windowThe window as a budget you spend rather than a setting you configure, what happens at the limit, and why a bigger one is not a strategy.
- Calling a model APIMessages and roles, the request and response envelope, tokens in and out, and the error handling every production call needs.
- Sampling parametersTemperature, top-k, top-p and repetition penalties — what each one does to the distribution, and when touching it is justified.
- Structured outputGetting schema-conforming JSON out of a model reliably, using constrained decoding rather than asking politely and hoping.
- Choosing a modelOpen weights against closed APIs, size against latency and cost, and why a benchmark leaderboard is the weakest evidence available to you.
Prompt and context engineering
- Prompt anatomyThe parts of a working prompt — role, task, context, constraints, output format — and why structure beats the search for magic words.
- Few-shot promptingZero-shot against few-shot, how to choose and order examples, and the cases where examples make the output worse.
- Chain of thoughtGetting a model to work before it answers, from step-by-step prompting to reasoning models that do it internally.
- Context engineeringDeciding what goes in the window and why — write, select, compress, isolate — and the failure modes of a window packed to the limit.
- Prompt cachingHow a cached prefix removes repeated prefill work, how to arrange a prompt so the cache actually hits, and what it saves.
Giving the model your data
- ChunkingSplitting documents for retrieval, and why chunk boundaries decide retrieval quality more than the embedding model does.
- Vector databasesWhat a vector index actually does, how HNSW and IVF trade accuracy for speed, and when Postgres with pgvector is enough.
- Similarity searchScoring, top-k, hybrid search with BM25, and re-ranking — why vector search alone underperforms on real queries.
- The RAG pipelineIngest, chunk, embed, index, retrieve, re-rank, generate — the whole path, with the failure mode waiting at each stage.
- RAG or fine-tuningWhich problem each one actually solves, why fine-tune it on our docs is usually the wrong instinct, and when the answer is neither.
Models that take actions
- What is an agent?A model using tools in a loop to reach a goal — where that genuinely helps, and where a fixed pipeline is the better engineering.
- Tool callingDeclaring tools, executing what the model asks for, feeding results back — and designing tools a model can actually use correctly.
- The agent loop (advanced)Plan, act, observe, repeat — owning the control loop, deciding when to stop, and why agents stall or run away.
- Model Context ProtocolThe problem MCP solves, how hosts, clients and servers fit together, and what a standard interface changes about wiring an agent to the world.
- Multi-agent systemsOrchestrators and sub-agents, handoffs, isolated context — and the honest case that one good agent usually beats five.
Knowing whether it works
- Why evals matterWhy iterating on vibes stops working, what an eval actually is, and the error-analysis loop that turns failures into measurements.
- Building an eval setSourcing real failures, labelling them, writing assertions that do not need a model, and keeping the set from going stale.
- LLM as judgeGrading output with a model, aligning that judge against human labels, and the biases that make an unvalidated judge worse than none.
- Tracing and observabilityWhy LLM traces are a different problem from uptime monitoring, what a span should carry, and finding the failing step in an agent run.
Where it goes wrong
- Prompt injectionWhy instructions and data share one channel, direct and indirect injection, the lethal trifecta, and why no complete fix exists.
- Guardrails and moderationInput and output checks, classifiers and moderation APIs, human review — and an honest account of what guardrails can promise.
- Bias and fairnessWhere bias enters a model and a product built on one, how it is measured, and what an application engineer can actually do about it.
- Privacy and data handlingHow user data moves through prompts, logs and retrieval, what provider terms actually say, and designing a system that does not leak by default.
Shipping it
- Fine-tuningFull fine-tuning against LoRA, what adaptation teaches and what it does not, and the real cost of owning a tuned model.
- Multimodal AIHow images, audio and video reach a language model, what changes in an application when the input is not text, and where the costs move.
- Cost and latencyToken economics, time to first token, caching, routing and batching — the levers that decide whether an LLM feature is affordable.
- Shipping and monitoringRollout and fallbacks, provider outages and rate limits, versioning prompts and models, and watching quality after launch.