CSCI 4845/5845 · Fall 2026 · Homework Assignments

Four graded assignments (10% each). Each pairs programming with short written analysis, and each ends with a brief “reflection” question connecting the work to the current lecture arc. Due dates are Thursdays 11:59 PM on Canvas.

Design principles: Units 1–2 homework is build it from scratch (the algorithms are the learning objective); Unit 3 homework is use the modern stack well (the judgment is the learning objective). Everything is sized for Google Colab.

AI-tool tiers per the syllabus: HW1–HW3 are Tier B (permitted with disclosure, but functions marked ★ must be written by you without AI assistance); HW4 is Tier A/B (LLM use required in parts, disclosure everywhere).

Dual listing (CSCI 4845 vs. 5845): each of HW1–HW4 includes a marked graduate extension (“G part”) — a short additional experiment or analysis tied to a primary paper. Required for CSCI 5845 (worth 10 of the 100 points; core parts scaled to 90); optional extra credit for CSCI 4845 (up to +3 points). G parts are listed with each assignment below.

Setup: HW1’s starter ships check_setup.py, which verifies the Python environment (or the provided Colab) before students write any code. The background survey happens as in-class questions in Lecture 1.

HW1 — Words & n-gram language models

Out Thu Aug 27 · Due Thu Sep 10 · Covers L2–L3 · ~6–8 h

  1. Tokenizer ★: regex tokenizer and word counts; Zipf plot on a real corpus.
  2. N-gram LM ★: bigram/trigram with add-k and interpolation; held-out perplexity; text generation (save the babble — it returns as a punchline in HW4).
  3. Written: smoothing by hand, Zipf/hapax, reading a perplexity number, babble analysis.
  4. G part (5845): Kneser-Ney smoothing; compare held-out perplexity against add-k and interpolation; short analysis of continuation probability.

HW2 — Classification & vector semantics

Out Thu Sep 10 · Due Thu Sep 24 · Covers L4–L7 · ~6–8 h

  1. Naive Bayes ★: multinomial NB with add-1 from scratch on SST-2; compare against scikit-learn logistic regression; analyze disagreements.
  2. Sparse vectors ★: co-occurrence, PPMI, tf-idf, cosine, nearest neighbors; export real vectors from the corpus and run the gender bias probe on them.
  3. Written: NB by hand, NB vs. logistic regression, PPMI, bias in their own vectors.
  4. G part (5845): WEAT effect size ★; compare bias across two vector spaces built from the same corpus.

HW3 — Sequence labeling & the first neural LM

Out Thu Sep 24 · Due Tue Oct 13 · Covers L8–L11 · ~6–8 h

  1. HMM tagger ★: transition/emission counts, add-k emissions, Viterbi; evaluate on UD English-EWT against a most-frequent-tag baseline.
  2. Bengio neural LM: the 2003 feedforward LM in PyTorch; compare held-out perplexity against their own HW1 trigram.
  3. Written: Viterbi by hand, tagger on real data, counting → learning, and the edit-distance/Viterbi correspondence (one DP, two costumes).
  4. G part (5845): greedy vs. Viterbi decoding experiment and analysis.

HW4 — Build a Transformer

Out Thu Oct 29 · Due Thu Nov 19 · Covers L12–L18 · ~6–8 h

  1. Attention ★: scaled dot-product → causal masking → multi-head → the block.
  2. MiniGPT ★ + fine-tuning: assemble and train a character-level mini-GPT on the HW1 corpus; generate with temperature/top-k; fine-tune DistilBERT on SST-2 and compare against their HW2 Naive Bayes number.
  3. Written: attention by hand, parameter census, step-zero-loss sanity ritual and the perplexity ladder, encoder vs. decoder.
  4. G part (5845): head-count ablation (4 heads vs. 1, everything else fixed) and analysis.

Grading & logistics