CSCI 4845/5845 · Fall 2026 · Homework Assignments

Four graded assignments (10% each) plus an ungraded setup assignment. Each pairs programming with short written analysis, and each ends with a brief “reflection” question connecting the work to the current lecture arc. Due dates are Thursdays 11:59 PM on Canvas.

Design principles: Units 1–2 homework is build it from scratch (the algorithms are the learning objective); Unit 3 homework is use the modern stack well (the judgment is the learning objective). Everything is sized for Google Colab.

AI-tool tiers per the syllabus: HW1–HW3 are Tier B (permitted with disclosure, but functions marked ★ must be written by you without AI assistance); HW4 is Tier A/B (LLM use required in parts, disclosure everywhere).

Dual listing (CSCI 4845 vs. 5845): each of HW1–HW4 includes a marked graduate extension (“G part”) — a short additional experiment or analysis tied to a primary paper. Required for CSCI 5845 (worth 10 of the 100 points; core parts scaled to 90); optional extra credit for CSCI 4845 (up to +3 points). G parts are listed with each assignment below.

HW0 — Setup & background survey (ungraded)

Out Thu Aug 20 · “due” Tue Aug 25

HW1 — Language models & classification from scratch

Out Thu Aug 27 · Due Thu Sep 10 · Covers L2–L5

  1. Text processing: regex exercises; build a word tokenizer; Zipf plot on a real corpus.
  2. N-gram LM ★: implement bigram/trigram models with add-k and interpolation; report perplexity on held-out data; generate samples (save your samples — they return as a punchline in Unit 3).
  3. Classification ★: Naive Bayes sentiment classifier from scratch (e.g., SST-2 or IMDB); compare against scikit-learn logistic regression; error analysis on 10 misclassified examples.
  4. Written: smoothing math; precision/recall trade-off scenario.
  5. G part (5845): implement Kneser-Ney (or Good-Turing) smoothing and compare held-out perplexity against add-k and interpolation; ~1 page of analysis.

HW2 — Embeddings & sequence labeling

Out Thu Sep 10 · Due Thu Sep 24 · Covers L6–L9

  1. Sparse vectors: build TF-IDF and PPMI vectors from a corpus; nearest-neighbor analysis.
  2. word2vec: complete a scaffolded skip-gram-with-negative-sampling implementation ★ (loss
    • gradient functions); train on a small corpus; analogy and similarity evaluation; replicate a mini bias probe (WEAT-style) and discuss.
  3. HMM tagger ★: implement Viterbi decoding for POS tagging on Universal Dependencies data; compare against a provided most-frequent-tag baseline; where does each fail?
  4. Written: why negative sampling works; BIO tagging exercise.
  5. G part (5845): extend the WEAT-style bias probe to a second embedding space (e.g., pretrained GloVe vs. your skip-gram vectors) and discuss the differences with reference to Caliskan et al. (2017).

HW3 — Build a Transformer

Out Thu Sep 24 · Due Tue Oct 13 (before midterm) · Covers L11–L15 · ~3 weeks

  1. Neural LM: train the Bengio-style feedforward LM on the HW1 corpus; compare perplexity with your trigram model (the semester’s central plot: same metric, new eras).
  2. Mini-GPT ★: implement a decoder-only Transformer LM from a heavily-scaffolded skeleton (nanoGPT-style): attention, multi-head, causal masking, one block, then stack. Train at character level on a small corpus; plot loss curves; sample generations at several temperatures.
  3. Fine-tuning: fine-tune DistilBERT on a classification task with the HuggingFace Trainer; compare with your HW1 Naive Bayes on the same test set — accuracy, compute cost, and 5 examples where they disagree.
  4. Written: attention-weight hand calculation; parameter counting for your mini-GPT.
  5. G part (5845): run one ablation on your mini-GPT (e.g., remove positional encodings, or single- vs. multi-head attention) and report the effect on loss curves and samples, connecting the result to Vaswani et al. (2017).

HW4 — The LLM toolkit

Out Thu Oct 29 · Due Thu Nov 19 · Covers L17–L23 · ~3 weeks

  1. Prompting as experimentation: on a fixed task and test set (e.g., a reasoning or extraction benchmark slice), compare zero-shot, few-shot, and chain-of-thought prompting; report accuracy and cost per query; a structured-output (JSON schema) exercise.
  2. RAG: build a small retrieval-augmented QA system over a provided document set (embed → index → retrieve → generate); evaluate faithfulness and answer quality; show one question where retrieval saves the model and one where it hurts.
  3. LoRA fine-tuning: QLoRA-tune a small open-weight model (1–3B) on a narrow task in Colab; compare against prompting the same model and a larger API model: quality, latency, cost.
  4. Written reflection: decision memo — for three scenarios (startup feature, enterprise internal tool, research experiment), which technique (prompt / RAG / fine-tune) and why.
  5. G part (5845): validate an LLM-as-judge — score your RAG system’s answers with a judge prompt, hand-label 20 of the same examples yourself, and report judge–human agreement and any systematic bias you observe.

API usage: course-provided credits cover the required experiments (~$5–10/student); every part has a local open-weight fallback path documented in the starter repo.

Grading & logistics