Transformers & Generative AI

LLMs, RAG and agents. Attention and transformers, large language models, prompting, retrieval-augmented generation, tools and agents.

• Transformers & Large Language Models — The architecture behind modern AI. Learn tokenisation (BPE), embeddings and positional encodings, scaled dot-product attention (queries, keys, values), multi-head attention, causal masking, the transformer block (attention + MLP + residual + layer norm), encoder vs decoder models, pre-training by next-token prediction, scaling, instruction tuning and RLHF, decoding (greedy, temperature, top-p), context windows and KV caching, and the capabilities and limitations of LLMs (hallucination) — with NumPy code for causal self-attention and softmax temperature.
• Generative AI Applications: Prompting, RAG & Agents — Building useful products on foundation models. Learn prompt engineering (instructions, examples, structure, chain-of-thought, output formats), embeddings and semantic similarity, retrieval-augmented generation (chunking, embedding, vector databases, retrieval, re-ranking, grounding and citations), TF-IDF/BM25 vs dense retrieval, tool use and function calling, agents and planning, fine-tuning vs RAG, evaluation of GenAI apps, guardrails, cost and latency — with a runnable TF-IDF retriever that selects context for a prompt.