Most workers now use a large language model (LLM) every week, but few can explain why it gives a different answer each time, why it invents a citation, why a long file gets ignored, or why one model costs ten times more than another. This instructor-led course opens the box. Trainees learn how LLMs work: text becomes tokens, the model predicts the next token, and attention lets it use context. They learn how pretraining, instruction tuning, and reasoning training shape behavior and why hallucinations happen. They then apply that knowledge to real decisions. They read a model card, compare closed and open-weight models, estimate token costs, and fit a job to a context window. They tell a meaningful benchmark from a contaminated one, and choose between prompting, retrieval-augmented generation (RAG), and fine-tuning. The final session covers tool use, the Model Context Protocol, the OWASP Top 10 for LLM Applications, the NIST Generative AI Profile, and Massachusetts data rules. Hands-on labs use the employer's approved AI tool plus a provider sandbox with model playgrounds and a small open-weight model, all with fictional Massachusetts business data. No coding is required. Optional technical extensions are provided for developers. Trainees leave with a model selection scorecard and a one-page LLM solution brief for a real task type. For analysts, IT staff, product and project owners, team AI champions, and other workers who choose, configure, or explain AI tools.
What you will be able to do
By the end of the course, trainees will be able to:
- Explain in plain language how an LLM turns text into tokens, predicts the next token, and uses attention over its context, and use that to explain variability and hallucination.
- Describe the training pipeline (pretraining, instruction tuning, preference tuning, reasoning training) and predict how base, instruct, and reasoning models will behave on the same task.
- Estimate tokens, context-window fit, and cost for a workplace task, and choose decoding settings (such as temperature) for accuracy or variety.
- Compare at least three models (closed, small, and open-weight) on a 10-item task-specific test, read their model cards and data terms, and recommend one with evidence.
- Choose between prompting, RAG, fine-tuning, and tool use for a business need, and name at least three OWASP LLM risks with matching controls.
Who this course is for
Business, data, and operations analysts; IT support and systems administrators; Project, product, and program managers; Team AI champions and super-users; Software developers new to LLMs; Compliance, risk, and quality staff
Before you start
Regular use of any AI chat tool at work, or completion of FAST-01 or FAST-02. Comfortable with spreadsheets and basic arithmetic (for cost estimates). No coding required; developers can take the optional technical extensions.
Hands-on activities
- Lab 1 Tokens, probabilities, and temperature: tokenize a fictional Cape Harbor Inn FAQ, estimate monthly cost, view next-token probabilities, and compare outputs at different temperatures.
- Lab 2 Base vs instruct vs reasoning, and a hallucination hunt: compare model types on the same prompts and log invented facts about fictional Bayside Precision Parts with and without a fact sheet.
- Lab 3 Model selection scorecard: test three models on a 10-item eval for fictional Back Bay Accounting Partners, with cost per 1,000 documents, context fit, and data terms.
- Lab 4 Grounded assistant and injection test: ground an assistant in a fictional North Shore Builders warranty handbook, test it, plant a hidden instruction, and write an LLM solution brief.
- Explain-it-back: each session ends with trainees explaining one mechanism (tokens, attention, RLHF, RAG) to a partner in two minutes.
How your learning is measured
- Attendance at all four sessions (start and end check).
- Four lab worksheets (W1 to W4) scored complete / incomplete against written criteria.
- Pre- and post-course knowledge check from the 30-item quiz bank; gain reported per trainee; 80% on the post-check to meet.
- Final LLM solution brief scored Meets on every rubric criterion (one resubmission allowed).
- Post-course confidence survey and a 60-day follow-up on whether the trainee used the scorecard or brief at work.
