to learn
My to-do list of learning.
These are the things that I am interested in learning.
- Nested Learning — ~30–45 min -[x] Jeff Dean on performance — ~2–3 h
- Non zero sum games — ~10–15 h (whole site, ~40 short posts)
courses and competitions
ml (~85–145 h)
- Karpathy Zero to Hero — ~40–60 h coding along (~16 h watch-only)
- illustrated transformers
- RL on wordle — ~1 h read; ~6–10 h to reproduce
- RL on 2048 — ~3–5 h (mostly training runs)
- nanochat — ~15–30 h (read code + run speedrun + tinker)
- RL in Huggingface — ~25–40 h (deep RL course, ~11 units)
cuda (~110–200 h)
- leetgpu — ~30–60 h (50+ problems)
- tensara — ~20–40 h (open-ended perf tuning)
- [programming massively parallel processors] — ~60–100 h (580 pp + exercises)
cuda misc (~35–55 h)
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance — ~2–3 h
- Outperforming cuBLAS on H100: a Worklog — ~1.5–2.5 h
- Defeating Nondeterminism in LLM Inference: — ~1–1.5 h
- Making Deep Learning go Brrrr From First Principles: — ~1–1.5 h
- Transformer Inference Arithmetic: — ~2–3 h (with the end exercises)
- Domain specific architectures for AI inference — ~1.5–2.5 h
- A postmortem of three recent issues: — ~45 min
- [How To Scale Your Model:] (http://jax-ml.github.io/scaling-book/ ) — ~7–10 h read; ~25–40 h with exercises
- [The Ultra-Scale Playbook:] (http://huggingface.co/spaces/nanotron/ultrascale-playbook ) — ~15–25 h
- The Case for Co-Designing Model Architectures with Hardware: — ~1.5–2.5 h
shaders (~16–32 h)
- The book of Shaders — ~10–20 h (with exercises)
- making software — ~6–12 h (early access, still growing)
others (~800–1,200 h)
- Forking chrome to render in the terminal — ~30–60 min (read)
- Wordle-RK Training Language Model to play wordle — see ml above
- Build your own c compiler — ~120–200 h (792 pp, full implementation)
- Beej’s guide to networking — ~8–15 h
- Mamba the easy way — ~30–60 min
- Build your own redis — ~40–60 h
- Distributed system from maelstorm — ~30–50 h
- Gossip glomers, fly.io — ~20–40 h
- Advanced compilers: self-guided online course (cornell) — ~60–100 h
- pintos: a simple operating system — ~150–250 h (full-semester course, built for teams)
- let’s write a browser — ~3–6 h
- write your own git — ~10–15 h
- Explained from first principles — ~25–40 h (3 articles, ~250k words)
- Programmer’s introduction to Mathematics — ~60–100 h (with exercises)
- How does a database work, database internals — ~10–20 h (unfinished tutorial)
- makefiles — ~2–4 h
- Embedded system — ~60–100 h (labs assume TM4C board)
- Ray Tracing — ~50–80 h (all 3 books)
- Nand2Tetris — ~100–150 h (both parts)
- Stanford crypto — ~50–80 h
- associated book — ~70–100 h on its own (818 pp)
- GDPR ✅
books to read (~140–200 h)
- How to Scale Your Model — ~7–10 h
- Harry Potter and the Methods of Rationality — ~35–45 h (~660k words)
- Algorithms for decision making — ~40–60 h (~700 pp, dense)
- Linux insides — ~22–30 h (~69 chapters)
- Google SRE — ~25–35 h (SRE book + workbook)
- compiler books — ~8–12 h (~250 pp)
- Git from the ground up — ~1.5–2.5 h
- post quantum cryptography — ~3–4 h (72 pp)
blogs to read (~25–30 h)
- Larry Sanger on baby reading — ~2.5–3 h (links a 140-page essay)
- How to win friends and influence people unabridged — ~4.5–6 h (it’s the full book)
- Understanding Apache Fluss — ~45 min
- Peter Thiel’s CS183 — ~6–8 h (19 essays)
- interview with shang Yi Chiang, R&D head at TSMC — ~1 h (49 pp transcript)
- how to do Distributed locking — ~20 min
- There’s no getting around you’re building a distributed system — ~25 min
- Getting real about distributed system reliability — ~15 min
- Coding for SSDs — ~2–3 h (6 parts)
- http crash course nobody asked for — ~1.5–2.5 h
- lessons from running codepen — ~1 h (58 min podcast)
- Everything you ever wanted to know about terminals — ~35 min
- Rqlite design — ~15 min (link 404s now; docs moved to rqlite.io)
- Taxonomy of Moats — ~30 min
- Leverage Points — ~40 min
- All Markets Are Not Created Equal: 10 Factors To Consider When Evaluating Digital Marketplaces — ~15 min
- What is amazon — ~1 h (~12k words)
- Bill gates interview 1992 PDC 1992 — ~1 h (57 min video)
- https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript — ~30 min talk
- Adverserial collaboration - Daniel Kahneman — ~25–30 min
to build (~100–200 h)
- RL environment for agents — ~40–80 h
- LMDB rust — ~60–120 h
privacy (~3–5 h)
- Science, Privacy, and Freedom: Issues and Proposals for the 1970’s. Part I–The Current Impact of Surveillance on Privacy, Alan F. Westin Columbia Law Review, Vol. 66, No. 6 (Jun., 1966), pp. 1003-1050 (48 pages) https://www.jstor.org/stable/1120997 — ~3–5 h
games: (~30–60 min)
https://flexboxfroggy.com/ — ~30–60 min (24 levels)
projects to go through (~110–200 h)
- etcd — ~30–50 h (raft/mvcc/lease/watch, architecture-level)
- cockroachDB — ~80–150 h (architecture-level only; multi-million LOC)
courses and others
dolby webcast
browsers (~80–120 h)
- https://browser.engineering/ — ~80–120 h (16 chapters + exercises)
a quote i like
The things that matter most can’t be copied: you can copy goals, you can’t copy discipline. You can copy ideas, you can’t copy execution. You can copy answers, you can’t copy understanding. You can copy thinking, you can’t copy thinking for yourself. ― Shane Parrish