Course Schedule
17 weeks | Spring 2026
Note: Monday & Wednesday sessions are lectures (Jin). Thursday sessions are labs (Ricky). Schedule subject to change.
| Week | Dates | Topics | Project Milestone | Assignments |
|---|---|---|---|---|
| 1 | Jan 12, 14 | Lecture 1: Course Intro | Lecture 2: Annotation Fundamentals | — | HW 0 assigned |
| 2 | Jan 21 (MLK Day Jan 19) | Lecture 3: When to Annotate | Tools & Formats | Project introduced | HW 0 due |
| 3 | Jan 28 (Jan 26 snow day) | Lecture 4: Corpus Selection & Data Sourcing | Project idea brainstorming | HW 1 assigned |
| 4 | Feb 2, 4 | Lectures 5 & 6: What Models Learn from Annotation | Conceptualize task | HW 1 due |
| 5 | Feb 9, 11 | Lecture 7: The Annotation Design Pipeline | Lecture 8: Inter-Annotator Agreement I | Form groups, Choose dataset | — |
| 6 | Feb 16, 18 | FEBRUARY BREAK - No Classes | ||
| 7 | Feb 23, 25 | Lecture 9: Inter-Annotator Agreement II | Lecture 10: Inter-Annotator Agreement III | Draft initial annotation scheme | HW 2 assigned |
| 8 | Mar 2, 4 | Lecture 11: IAA in the LLM Era | Lecture 12: Annotator Reliability | Complete Draft 0 guidelines, Annotate pilot batch | HW 2 due |
| 9 | Mar 9, 11 | Class Group Presentations | Project proposal presentation, Peer feedback incorporated | HW 3 assigned |
| 10 | Mar 16, 18 | Lecture 13: Annotator Reliability II | Lecture 14: Annotation Projects | Draft 1 guidelines, Pilot annotations, Compute pilot IAA, Error analysis, Guideline revision iterations | HW 3 due |
| 11 | Mar 23, 25 | Lecture 15: Annotation Projects | Lecture 16: Supervision Engineering | Finalize annotation guidelines, Lock label schema | HW 4 assigned |
| 12 | Mar 30 (Passover Apr 1) | Lecture 17: Instruction Annotation | Full-scale annotation begins, Adjudication protocol finalized | HW 4 due |
| — | Apr 6, 8 | PASSOVER BREAK - No Classes | ||
| 13 | Apr 13, 15 | Lecture 18: Preference Annotation & RLHF | Lecture 19: Reasoning & Process Supervision | Full IAA computation, Adjudication complete, Gold standard dataset finalized | — |
| 14 | Apr 20, 22 | Lecture 20: Synthetic Annotation | Lecture 21: Active Learning & LLM Annotators | Model experiments | — |
| 15 | Apr 27, 29 | Lecture 22: Evaluation Annotation | Lecture 23: Multilingual & Cultural Annotation | — | — |
| 16-17 | May 7, 9:00a.m.–12:00p.m. | FINAL EXAMINATION PERIOD | Project report write-ups, Project presentations | — |
Key Dates Summary
Holidays & Breaks
- Jan 19: MLK Jr. Day - No Class
- Feb 16-18: February Break
- Apr 1: Passover - No Class
- Apr 6-8: Passover Break
Assignment Due Dates
- Jan 21: HW 0 due
- Feb 4: HW 1 due
- Mar 4: HW 2 due
- Mar 18: HW 3 due
- Mar 30: HW 4 due