← Sane Labs Sections
← The lab

Learn to train the thing yourself.

Everything below was written or taught by someone else — universities, research labs, and the people who wrote the libraries you will import. None of it costs money, and none of it is summarised here: each row is a link to the original, with one line saying what it is and how long it takes. Sorted so you can start anywhere and know what you are skipping.

Links
254
Sections
20
Behind a paywall
0
Last checked
2026-09-12
00

Start here

Three honest routes in. Pick the one that describes you now, not the one that describes who you want to be.

You have never trained anything

Start at the bottom and do not skip the maths. Three to six months, honestly.

You can code and want to fine-tune this week

Skip the theory for now. Run one fine-tune end to end, then come back for the parts that broke.

You want to understand a model, not just use one

Write one. Everything after that reads differently.

254 shown
01

Foundations

12

The linear algebra, calculus and probability every later section quietly assumes. Skip it if you already read a Jacobian without flinching.

02

Python & the tooling

11

You will spend more hours in NumPy, pandas and a terminal than in any model architecture.

03

Classical machine learning

14

Gradient boosting still wins most tabular problems. Learn this before reaching for a transformer.

04

Deep learning

15

Backpropagation, optimisers, regularisation, convolutions — the layer of knowledge that does not go stale.

CoursePractical Deep Learning for Codersfast.ai · Jeremy Howardfree · top-down, code first · the best-loved course in the field CourseMIT 6.S191: Introduction to Deep LearningMITfree · one week of lectures and labs, re-recorded every year BookDive into Deep LearningZhang, Lipton, Li & Smolafree · maths, code and discussion on the same page BookDeep LearningGoodfellow, Bengio & Courvillefree HTML · the reference text BookUnderstanding Deep LearningSimon J.D. Princefree PDF, slides and notebooks · modern and beautifully drawn BookNeural Networks and Deep LearningMichael Nielsenfree · derives backprop by hand, gently CourseNYU Deep Learning (DS-GA 1008)Yann LeCun & Alfredo Canzianifree · lectures, notes and notebooks CourseStanford CS231n: Deep Learning for Computer VisionStanfordnotes and assignments open · the CNN classic DocsPyTorch TutorialsPyTorchofficial · start with 'Learn the Basics' CourseUvA Deep Learning TutorialsUniversity of Amsterdamnotebooks in both PyTorch and JAX CourseDeep Learning SpecializationAndrew Ng · DeepLearning.AIfree to audit · five courses CodemicrogradAndrej KarpathyMIT · ~150 lines · autograd small enough to read in one sitting BookDeep Learning Tuning PlaybookGoogle Research & Harvardhow to actually choose hyperparameters, from people who do it daily CodeAnnotated Paper Implementationslabml.aiMIT · 60+ architectures implemented in PyTorch with the paper alongside ArticleA Recipe for Training Neural NetworksAndrej Karpathythe debugging discipline nobody teaches you
05

NLP & transformers

14

Tokenisation, embeddings, attention — where language models actually begin.

06

Build an LLM from scratch

16

The fastest way to stop treating a model as a black box is to write one. Everything here ends in code you typed yourself.

CourseNeural Networks: Zero to HeroAndrej Karpathyfree · ~10 video lectures · micrograd, then makemore, then GPT VideoLet's build GPT: from scratch, in code, spelled outAndrej Karpathy2 h · a working GPT in one sitting VideoLet's build the GPT TokenizerAndrej Karpathy2 h 13 · BPE, and why tokenisation causes half of all LLM weirdness VideoDeep Dive into LLMs like ChatGPTAndrej Karpathy3 h 31 · pretraining, SFT and RLHF end to end, little maths required CodenanoGPTAndrej KarpathyMIT · ~300 lines that reproduce GPT-2 CodeminGPTAndrej KarpathyMIT · the readable, minimal ancestor of nanoGPT Codellm.cAndrej KarpathyMIT · GPT-2 training in plain C and CUDA, no framework CodenanochatAndrej Karpathythe full ChatGPT pipeline — pretrain, SFT, RL, web UI — in one clean repo CourseStanford CS336: Language Modeling from ScratchStanfordthe whole pipeline as coursework: tokenizer, model, training, evaluation VideoCS336 lecture videosStanford Onlineall lectures, free CodeBuild a Large Language Model (From Scratch) — codeSebastian RaschkaApache-2.0 · every chapter's notebooks are free; the book is optional CourseLLM CourseMaxime LabonneApache-2.0 · roadmaps plus dozens of free Colab notebooks CourseHugging Face LLM CourseHugging Facethe NLP course's successor: pretraining, fine-tuning, inference Codemodded-nanogptKeller Jordan et al.the community speedrun — read the diffs to see what actually helps CodelitgptLightning AIApache-2.0 · 20+ models as readable single-file implementations ArticleThe Annotated GPT-2Aman AroraGPT-2 explained alongside its own code
07

Pretraining & scale

14

What changes when the run stops fitting on one GPU: parallelism, schedules, scaling laws, and the failure modes nobody warns you about.

08

Fine-tuning

28

Taking a trained model and bending it to your task. The largest section here, because it is the part most people actually need and the part with the most folklore around it.

DocsFine-tune a pretrained modelHugging Face Transformersofficial · the honest starting point, Trainer and a plain PyTorch loop DocsPEFT documentationHugging FaceLoRA, prefix tuning, IA3, adapters · conceptual guides plus API DocsTRL — Transformer Reinforcement LearningHugging Facethe library behind most open SFT, DPO and GRPO recipes DocsSFTTrainer guideHugging Face TRLsupervised fine-tuning, including packing and completion-only loss DocsUnsloth documentationUnslothfree · fine-tuning that fits in a free Colab GPU, with the tricks explained NotebookUnsloth notebooksUnsloth100+ ready Colab notebooks — Llama, Qwen, Gemma, Mistral, vision, GRPO DocsAxolotl documentationAxolotl AIApache-2.0 · config-file fine-tuning; the YAML examples are the real course CodeLLaMA-Factoryhiyouga et al.Apache-2.0 · 100+ models, a web UI, and every method in one place DocstorchtunePyTorchnative PyTorch recipes you can read top to bottom PaperLoRA: Low-Rank Adaptation of Large Language ModelsHu et al.the paper that made fine-tuning affordable PaperQLoRA: Efficient Finetuning of Quantized LLMsDettmers et al.65B on a single 48 GB card · read alongside the bitsandbytes docs ArticleMaking LLMs even more accessible with bitsandbytes and QLoRAHugging Facethe practical write-up of 4-bit fine-tuning ArticleParameter-Efficient Fine-Tuning using PEFTHugging Facethe short version, with working code CodeAlignment HandbookHugging FaceApache-2.0 · full, reproducible recipes: SFT then DPO, configs included Coursesmol-courseHugging Facea hands-on course on aligning small models on modest hardware CourseFinetuning Large Language ModelsDeepLearning.AI & Laminifree short course · ~1 h · when to fine-tune at all ArticlePractical Tips for Finetuning LLMs Using LoRASebastian Raschkahundreds of ablations distilled into rules of thumb ArticleFinetuning LLMs Efficiently with AdaptersSebastian Raschkawhat each PEFT method actually changes in the weights DocsChat templatesHugging Face Transformersthe single most common cause of a fine-tune that trains but never answers ArticleFine-tune Llama 3 with ORPOMaxime LabonneSFT and preference alignment collapsed into one step NotebookLlama CookbookMetaofficial fine-tuning and deployment recipes NotebookGemma CookbookGooglethe same idea for Gemma, including Keras and JAX paths PaperFinetuned Language Models Are Zero-Shot Learners (FLAN)Wei et al.the paper that introduced instruction tuning PaperSelf-InstructWang et al.how to generate the instruction data you do not have PaperLIMA: Less Is More for AlignmentZhou et al.1000 carefully chosen examples beating far larger sets CodemergekitArcee AImerging fine-tuned checkpoints instead of retraining NotebookHugging Face Open-Source AI CookbookHugging Facedozens of task-shaped recipes, all runnable NotebookPyTorch Lightning / Fabric fine-tuning studiosLightning AIfree-tier GPU studios with published fine-tuning templates
09

Preference tuning & RLHF

14

Turning a model that completes text into one that answers. RLHF, DPO, GRPO, and the reasoning-model recipes that followed.

10

Reinforcement learning

8

The background the section above assumes. Worth a detour if policy gradients are still a rumour to you.

11

Evaluation

10

The part people skip, and the reason so many fine-tunes look great in a demo and fail in use.

12

Data & datasets

11

Model quality is mostly data quality. This is the least glamorous section and the highest-leverage one.

13

Quantization, inference & speed

15

Getting a model to fit, and then to be fast. GPU arithmetic, kernels, quantization formats and serving.

14

RAG, embeddings & retrieval

10

Putting the knowledge in the context instead of in the weights. Usually cheaper and more correctable than fine-tuning.

15

Prompting & agents

12

The cheapest lever, and the one to exhaust before you fine-tune anything.

16

Vision, audio & diffusion

12

Everything that is not text: image generation, vision-language models, speech.

17

Interpretability & safety

10

What is actually happening inside the weights, and what to do about the parts you would rather were not.

18

Shipping it: MLOps & systems

10

The distance between a notebook that works and a service that keeps working.

19

Staying current

10

The field moves fast enough that a course from last year has gaps. These are the feeds worth keeping.

20

Free compute & practice

8

Nothing on this page sticks until you run it. All of these give you a GPU without a credit card.

Nothing on this page is hosted here and nothing has been copied from it: every row links to its source, and the one-line notes are ours. Licences belong to the original authors — several items are Apache-2.0, MIT or CC BY-NC-SA, and that is stated where it matters.

Found something dead, or something missing that should be here? Tell us.
Sane Labs · 2026 · the lab · Synth-2
↑ Back to top