Session 3 · Analysis

Week 2 · Requirements discovery

The agent has never
met your user.

Everything it knows about them comes from what you find out and write down.

Building Real Systems with AI · Gonzaga University

Lids down

Warm-up

Your Lab 2 app is on the internet.

Open your live URL on your phone. Add something.

Now open it on your laptop.

Where did your data go?

Hold that thought. Week 3 explains it.

Why this matters

Building is cheap now.

An agent can build almost anything you describe, in an afternoon.

So the expensive mistake is no longer building it wrong.

It is building the wrong thing.

Discovery reduces uncertainty before you spend a sprint on an answer.

Same work, two vocabularies

Discovery is requirements elicitation

In the readingIn Systems Analysis and Design
Interviews and observationRequirements elicitation
Jobs to be done, personasUser and stakeholder analysis
What they do today, journey mapAs-is process model
Riskiest assumptionFeasibility and risk analysis
MVP scope: in and outRequirements baseline

Discovery is truth-seeking

What you believe vs. what is true

Venn diagram of what you believe and what is true, split by an evidence line into six regions.

Two ways most products fail:

False Confidence

"I know users want this." Believed, wrong, and never checked.

Available but Unseen

The truth was one interview away. You built on a guess instead.

Today's running example

The dues tracker

A student club collects dues. Members pay the treasurer on Venmo.

Every Sunday, the treasurer matches payments, one by one, against a sign-up sheet to see who still owes.

A student thinks: "One page that shows who has paid would save that Sunday."

Is that true?

Does the Sunday matching hurt enough to switch? Can the app even get the Venmo data? Would anyone use it twice?

The example is invented for teaching.

Part 1

Describe the problem
without naming a product

Jobs to be done

Why did people buy the milkshake?

The question

A fast-food chain improved the recipe. Sales did not move. So they asked: what job are people hiring the milkshake to do?

The answer

Most were bought in the morning by commuters: something to make a boring drive interesting and keep them full until lunch.

The real competitors

Not other milkshakes. Bananas, bagels, and donuts. The fix: thicker, fruit chunks, faster at the drive-through.

People don't want your product. They want progress in a specific situation.

Christensen et al. 2016, Competing Against Luck

Jobs to be done

The job statement

When I'm situation, I want to motivation, so I can outcome.

Dues treasurer: When I'm closing out dues on Sunday night, I want to know who still owes without checking every payment, so I can remind the right people and be done.

Trap  The middle slot invites a feature: "I want to scan Venmo with an app." Write progress the person would recognize even if your product never existed.

Turn to a neighbor

Try it · 90 seconds

What is wrong with each one?

Job statementThe mistake
"When I'm studying, I want to use an AI flashcard generator, so I can learn faster."Names a product
"When I'm hungry, I want to eat, so I can not be hungry."Too vague You can't observe "hungry." When, where, doing what?
"When I'm running a club event, I want a Slack integration for RSVPs, so I can track attendance."Names a feature Where the progress should be.

Map the whole job

The pain is usually not where you assumed

Customer journey map for the club treasurer: collect, match, chase, report, with mood dipping during matching and chasing.

Opportunities sit under the dips. This is your as-is process, and it goes in your requirements.

The real competitor

You're not competing with other apps. You're competing with the habit.

Four forces: push and pull toward switching, anxiety and habit against it.

Push: Sunday spent matching Venmo to a sheet

Pull: one page that shows who has paid

Anxiety: "What if it's wrong and I chase someone who paid?"

Habit: everyone already Venmos; the sheet mostly works

So in every interview, ask: "What do you do today?"

Forces of progress: Moesta and Engle 2020
Lids up · 10 min

Lab 3 · Part A

Choose your project's user

Open the problem discovery worksheet. For three rows, fill in:

  • Who: a real person you can talk to this week
  • Frustration: in their words
  • Job: When I'm ___, I want to ___, so I can ___
  • Frequency, intensity, willingness to pay (1 to 5)

The access test

Pick the row with the strongest evidence and a user you can actually reach.

Who can you talk to by Thursday?

"Fix campus parking" fails. "The club treasurer I sit next to" passes.

Part 2

Ask about what they did,
not what they'd do

The Mom Test

Your mom will say your idea is great.

She loves you. Her feedback is useless.

People lie to be nice about your idea. They rarely lie about what they did last Tuesday.

  1. Talk about their life, not your idea
  2. Ask about specifics in the past, not generics about the future
  3. Talk less, listen more: under 20% of the time
Fitzpatrick 2013, The Mom Test

Rewrite the question

Bad question → good question

BadWhyGood
"Would you use an app that does X?"Hypothetical. Everyone says yes."Tell me about the last time you tried to do X."
"Do you think this is a good idea?"Opinion, not fact."What have you tried? What happened?"
"How much would you pay for this?"They'll guess to be nice."What are you paying to solve this now?"
"Would you recommend this to friends?"A promise, not behavior."Have you told anyone about this problem?"

The most useful question

"Walk me through the last time this happened."

How often

it really happens

What they did

not what they say they'd do

What they tried

and why it didn't work

Live demo · 3 min

Watch what changes

Two interviews, one volunteer

Interview 1

"So I'm building an app that tracks club dues. Would you use something like that?"

"Do you think other treasurers would want it?"

"What would you pay for it?"

Interview 2

"Tell me about the last time you had to keep track of who paid you for something."

"Walk me through what you did, step by step."

"What was the most annoying part? What did you do about it?"

What did you learn from each?

How much can you trust it?

The evidence ladder: said, did, paid

Evidence ladder: said costs nothing, did costs time and effort, paid costs money. Each rung is harder to fake.

Each rung costs the person more to fake. "Users want this" with only said evidence is an assumption dressed up as an insight.

Call it out

Try it

Said, did, or paid?

"I'd definitely pay for something like this."Said
She showed you a Google Sheet with 14 tabs she maintains by hand.Did
He asked if he could beta test it.Said until he shows up
Three people put down a $10 deposit on a pre-order page.
"I hate doing this every week."Said ask about last week
He forwarded your email to two friends with the same problem.Did risked his reputation
Lids up · Pairs

Lab 3 · Part B

Practice interview: 8 minutes each way

Round 1: Partner A interviews
8:00

Then 4 minutes: mark each line of your notes

said did

Rules

  • Ask about the past, not the future
  • Don't describe your idea. Caught pitching? Stop.
  • Ask what they do now and what it costs
  • Write down their exact words
  • Initials only, never names

Starters

  • Tell me about the last time you dealt with ___.
  • What do you do about it now? Who's involved?
  • What does it cost you in time or money?
Lids up · 15 min

Lab 3 · Part C

Synthesize with AI, carefully

Paste your notes into Claude Code with the Lab 3 prompt. It must quote your notes for every finding.

The AI will invent findings if you let it. It has read a million interviews, just not yours.

Check every line

Treasurer is frustrated by matching payments "I spend every Sunday on this"

Treasurer wants automatic reminders

↑ No quote supports it. Strike it.

Write how many you struck at the top of synthesis.md.

Part 3

Test the riskiest thing first,
then build the smallest thing

What could make it fail?

Find the riskiest assumption

Two by two grid of uncertainty and fatality. Value and Venmo feasibility are in the riskiest quadrant.

Value: will they want it enough to switch?

Usability: can they figure it out?

Feasibility: can we build it?

Viability: does it work as a business?

An agent settles feasibility in an afternoon. A working app is not proof anyone wants it.

Four risks: Cagan 2017

Before you build

Fake it, do it by hand, or sell it first

Smoke test

Dropbox posted a demo video before the product existed. The waitlist jumped from 5,000 to 75,000.

Concierge

Do the job by hand for a few people. Food on the Table started with one customer.

Wizard of Oz

Looks automated; a person does the work. Zappos bought shoes at full price, one order at a time.

For the dues tracker: a concierge test

Three treasurers send you their Venmo export each Sunday. You send back a "who has paid" list built by hand. In two weeks you know if they keep sending it, with no code at all.

Ries 2011, The Lean Startup

MVP scope

What is the one thing users must be able to do?

MoSCoWMeans
MustThe MVP fails without it
ShouldImportant, but it works without it
CouldOnly if time permits
Won'tOut of scope, on purpose

Most "musts" are "shoulds" at best.

Your requirements baseline: In = the musts. Out = everything else, and why.

Cut to the minimum, then make what's left excellent. A barely-working app gets polite answers that teach you nothing.

Lids up

Lab 3 · Part D

Discovery ends in a page the agent can build from

discovery/requirements.md

  • Problem statement and job to be done
  • Users and stakeholders, and who it is not for
  • Persona and current process (as-is)
  • Journey map
  • Functional and non-functional requirements, each with an ID and a source
  • Riskiest assumption
  • MVP scope: in and out

Why it matters

Tell an agent "build a dues tracker" and it fills every blank with a guess from the average app it has seen.

Your requirements replace its guesses with your evidence.

Privacy and usability are requirements too.

Before you leave

Exit ticket

Who is your project's user, and what is one thing they did, not said, that shows the problem is real?

By Thursday

Two real interviews · Lab 3 due before class · Sprint 1 plan discussion post

By Tue, Nov 24

Three interviews total for Sprint 1 · names removed from all notes

← → or click to advance · F full screen · T starts the timer