Portfolio project ladder
Fifteen projects, each with a 'done when', from tabular ML to a paper reproduction
Hiring managers and collaborators do not read certificates; they open your GitHub and your demo and decide in five minutes. A project is proof only when it has a clear question, an honest evaluation and a write-up that shows judgment. This lesson gives you fifteen projects mapped to the phases, each with an explicit finish line, and the packaging rules that make them count.
You'll be able to
- Choose projects at the right difficulty and map them to the phases you have completed
- Define a 'done when' criterion with a measurable evaluation before starting
- Package a project with a README, evaluation evidence, a live demo and a write-up that survives a five-minute review
- Publish on GitHub and Hugging Face Spaces and present projects on a CV
Key ideas
- A portfolio project needs a question that can fail, an evaluation with a baseline, a runnable demo and a write-up with decisions and failures.
- Decide the 'done when' metric before writing model code; it keeps scope finite and makes the result defensible.
- Alternate from-scratch projects (understanding) with real-tool projects (shipping); interviewers probe both.
Show 4 more key ideas
- The README must pass a five-minute test: summary + screenshot, results table with baseline first, three-command run, decisions, limitations.
- Publish demos on Hugging Face Spaces with Gradio; cache API calls; write a model card for anything you upload.
- On a CV, one line per project: what, how, numbers, link.
- Finish small projects in order rather than abandoning one large one; the ladder is sized to be completed.
Suggested path through this lesson
What makes a project portfolio-worthy
A portfolio project is an argument that you can do real work. It needs four things. A question sharp enough to fail: "does a fine-tuned 3B model beat the base model on my support tickets?" rather than "an LLM chatbot". An evaluation that would convince a skeptic: a held-out set, a baseline, a metric, and error analysis. A demo anyone can run in a browser without installing anything. And a write-up that explains the decisions, the failures and what you would do next. Tutorial clones, notebooks without READMEs and projects with no baseline fail the five-minute test regardless of how fancy the model is.
The ladder
Pick in order of the phase you have finished. Each project lists the phase it belongs to and the finish line. Estimated effort is a weekend for the early projects and two to four weeks for the late ones.
| # | Project | Phase | Done when |
|---|---|---|---|
| 1 | Tabular ML pipeline on a Kaggle dataset (house prices, credit default, churn) with scikit-learn pipelines and a GBDT | 3 | Cross-validated metric beats a tuned linear baseline; a one-page error analysis names the three worst segments and a fix for each |
| 2 | kNN, k-means and PCA from scratch in NumPy, vectorized, matching scikit-learn | 3 | Predictions and components agree with scikit-learn to numerical precision on three datasets; a timing table shows your vectorized vs loop versions |
| 3 | Autograd engine + MLP (micrograd-style) that learns XOR and two-moons | 4 | Gradients match PyTorch autograd within on random graphs; decision-boundary plots included |
| 4 | MNIST and Fashion-MNIST MLP and CNN in PyTorch with a reusable training template (LR finder, checkpoints, AMP) | 4–5 | At least 99 percent on MNIST and 92 percent on Fashion-MNIST test sets; a loss-curve gallery showing one failure you diagnosed |
| 5 | Fine-tuned image classifier deployed on Hugging Face Spaces with Gradio and Grad-CAM | 5 | Public demo URL; confusion matrix on a held-out set; Grad-CAM overlays for five correct and five wrong predictions with commentary |
| 6 | Character-level language model ladder: bigram → MLP → tiny transformer on names or song titles | 6 | Validation negative log-likelihood decreases at each rung; 20 samples per model shown side by side |
| 7 | GPT from scratch with your own BPE tokenizer on a corpus you care about | 7 | Validation loss below 1.6 nats on character-level Shakespeare (or a documented equivalent on your corpus); a parameter-count table matching the formula from 7.4 Inside the block: residuals, LayerNorm, FFN and embeddings |
| 8 | Inference profiler: measure tokens per second and memory vs batch size, context length and precision for a small open model | 7 | Plots reproduce the KV-cache memory formula from 7.10 Efficient inference: KV cache, GQA, FlashAttention, speculative decoding, quantization within 10 percent; a short note on where decoding is bandwidth-bound |
| 9 | LoRA fine-tune of a 1–8B open model on a custom dataset, with a before/after evaluation | 8 | Held-out win rate vs the base model judged by an LLM judge and by you on 50 examples; merged adapter published to the Hub |
| 10 | Diffusion from scratch (DDPM) on MNIST or CIFAR-10, with a schedule study | 8 | Recognizable samples after training; a figure comparing linear vs cosine schedules; FID or a documented proxy reported honestly |
| 11 | Cited RAG assistant over your own documents with hybrid retrieval and reranking | 9 | A 50-question golden set; recall@5, faithfulness and answer relevance reported; a refusal path for unanswerable questions demonstrated |
| 12 | Evaluation harness: an LLM-as-judge pipeline calibrated against your own labels | 8–9 | Judge agreement with your labels above 80 percent on 100 items; position-bias test included; results reproducible from one command |
| 13 | Tool-using agent with an MCP server and a 20-task evaluation | 10 | Pass rate and cost per task reported; the MCP server works from at least one third-party client; a trace viewer or logs for every run |
| 14 | Single agent vs orchestrator–workers comparison on a research or coding task | 10 | A table of quality, latency and cost for both designs; a written recommendation with the conditions under which each wins |
| 15 | Paper reproduction of a 2023–2025 result (DPO on a small model, RoPE length extrapolation, Self-RAG, a scaling curve) | 11 | One figure reproduced within a stated tolerance; a list of every detail the paper omitted; a short "what I learned" post |
Intuition. The ladder alternates "from scratch" (projects 2, 3, 6, 7, 10) with "with real tools" (1, 5, 9, 11, 13). The first kind proves understanding; the second proves you can ship. Interviewers probe both.
Packaging: the README that passes the five-minute test
- One-sentence summary and a screenshot or GIF at the top. What it does, what the result is.
- Results table with the baseline on the first row. Numbers without a baseline are decoration.
- How to run in three commands, tested on a clean machine or Colab. Pin versions.
- Design decisions as a short list: why this model, this metric, this chunk size. Each decision with the alternative you rejected.
- Failures and limitations stated plainly. Reviewers trust write-ups that admit what did not work.
- Evaluation details: dataset splits, how labels were made, how the LLM judge was validated.
- What next in three bullets.
Keep the repository clean: a src/ package, a notebooks/ folder for exploration only, a tests/ folder with at least smoke tests, a requirements.txt or pyproject.toml, and a license. Commit history with real messages is itself evidence of how you work.
Demos and publishing
A Gradio or Streamlit app on Hugging Face Spaces costs nothing and runs on CPU for small models; for LLM demos call an API and cache responses to control cost. Publish models and datasets on the Hub with a model card that states training data, evaluation and limitations. Write a 600–1,200 word post per major project (a blog, GitHub Pages or a LinkedIn article) that tells the story: question, approach, result, surprise, next step. Posts are what people share; repositories are what they verify.
Presenting projects on a CV
One line each: what, how, result. "Built a cited RAG assistant over 2,000 internal documents (hybrid BM25 + dense retrieval, cross-encoder reranking); faithfulness 0.91 and recall@5 0.84 on a 50-question golden set; deployed on Spaces." Numbers, methods, and a link. Three such lines beat ten vague ones.
Common mistakes
- Starting with the model instead of the evaluation; decide how you will know it worked before you train anything.
- Choosing a dataset nobody cares about, including you; a personal corpus or a domain you know produces better questions and better write-ups.
- Notebook-only projects; reviewers cannot run or test them.
- Hiding failures; a documented negative result is still a result.
- Over-scoping; a finished small project beats an abandoned ambitious one, and the ladder is ordered so that each project is finishable.
Where this shows up later
- Projects 11–14 are the artifacts interviewers ask about in LLM system-design rounds (11.4 ML and LLM system design and interviews).
- Project 15 uses the reading skills from 11.1 How to read (and reproduce) papers and the list in 11.2 Landmark papers: the reading list.
- The agent evaluation skills from 10.9 Evaluating agents and the RAG evaluation skills from 9.8 Evaluating and observing LLM apps are what make projects 11–14 credible.
Common pitfalls (4)
- Reporting a metric with no baseline, or a baseline you did not tune as hard as your model.
- Using an LLM judge without checking its agreement with your own labels on a sample.
- Shipping a notebook as the deliverable; put code in a package with tests and make the notebook a tour.
- Copying a tutorial's dataset and model and calling it a project; change the question, the data or the evaluation.
Recap: all 7 key points
- A portfolio project needs a question that can fail, an evaluation with a baseline, a runnable demo and a write-up with decisions and failures.
- Decide the 'done when' metric before writing model code; it keeps scope finite and makes the result defensible.
- Alternate from-scratch projects (understanding) with real-tool projects (shipping); interviewers probe both.
- The README must pass a five-minute test: summary + screenshot, results table with baseline first, three-command run, decisions, limitations.
- Publish demos on Hugging Face Spaces with Gradio; cache API calls; write a model card for anything you upload.
- On a CV, one line per project: what, how, numbers, link.
- Finish small projects in order rather than abandoning one large one; the ladder is sized to be completed.
Watch
The template for project 7 and the standard for a from-scratch write-up.
Read
Exercises
- Write the README first
Pick your next project from the ladder and write its README before any code: summary, planned results table with the baseline row, run commands, decisions you expect to make. Expected: a one-page document that defines "done when" with a number.
- Audit an existing project
Take a project you already built and score it on the seven README items and the four portfolio requirements (question, evaluation, demo, write-up). Fix the two lowest-scoring items. Expected: a before/after diff of the README and at least one new evaluation number.
- Ship a Space open ↗
Deploy any model you have trained as a Gradio app on Hugging Face Spaces with an example input preloaded. Expected: a public URL that a friend can use on their phone, and a model card with limitations.
Quiz
Flashcards
These cards are now in your spaced-repetition queue. Review all due cards →