2026
RAG quiz generation with LLM-as-judge grading
Studying from a document is passive. Turning it into practice questions is the part that actually builds recall, and it is the part nobody has time for. Auto-generated questions usually hallucinate content that was never in the source, and grading a free-text answer by string-matching marks a correct answer wrong because the wording differed.
A production REST API that ingests a PDF or text file and turns it into an adaptive quiz session grounded entirely in that document. It generates questions, grades free-text answers against a three-criterion rubric with partial credit, tags the specific knowledge gaps behind each wrong answer, and aggregates a session into a report with a grade and a targeted study recommendation.
POST /ingest pypdf → 800-char chunks (100 overlap)
→ all-MiniLM-L6-v2 (384-dim) → FAISS
POST /quiz retrieve by document ID → GPT-4o
→ structured JSON questions
POST /eval GPT-4o as judge → 3-criterion rubric
→ partial credit + knowledge-gap tags
GET /report aggregate → grade + ranked gaps
FastAPI · Pydantic v2 · LangChain
Docker container → Railway · React/Vite frontendinstead of A hosted embeddings API
Ingestion embeds every chunk of every uploaded document, the highest-volume operation in the system. Running a small model on CPU takes embedding cost to zero and removes a network dependency from the slowest path, at a quality level that is more than sufficient for retrieving within a single document.
instead of Top-k nearest-neighbour search
Top-k answers 'what is most similar to this query'. Quiz generation needs 'cover this whole document'. With similarity search, a quiz would silently over-sample whatever the query happened to resemble and never ask about the rest of the material.
instead of String or keyword matching against a reference answer
A student can be right in words the reference never used, or half-right in a way that deserves partial credit. A rubric across accuracy, completeness and terminology produces a defensible score plus the diagnostic feedback that makes the grade useful. It is the same technique used in reward modelling and model evaluation.
instead of Parsing prose responses
Every generation and grading call feeds directly into typed application code. JSON mode with an explicit schema plus Pydantic validation means a malformed response fails loudly at the boundary instead of corrupting a quiz halfway through a session.