Asking whether I can stand behind a line I have not heard myself Going back to the recording or transcript personally, every time, so nothing machine-produced reaches a team unchecked.
Asking whether I can stand behind a line I have not heard myself Going back to the recording or transcript personally, every time, so nothing machine-produced reaches a team unchecked.
Scrub back and listen to anything load-bearing, because the machine may have heard it wrong and the meaning shifts
So anything load-bearing, I scrub back and listen.
Never hand the AI's output straight to a team
I never hand its output straight to a team.
Re-listen to the recording for anything I quote, because right nine times out of ten isn't right every time when the quote is going into a deck with my name on it
there's a difference between right nine times out of ten and right every time, when the quote is going into a deck with my name on it. So I re-listen to the recording for anything I quote
Never take its quotes: if it quotes a participant at me, I go back to the transcript and find the line myself, every time, even when it's boring, especially then
Ever. If it, you know, quotes a participant at me, I go back to the transcript and find the line myself. Every time, even when it's boring. Especially then.
Don't trust machine transcripts for quotes; check against the recording
you can't trust them for quotes, you check against the recording
Trust the transcripts more when I've checked them
I trust the transcripts more when I've checked them?
Make it a rule to find every quote a machine gives me in the transcript and read the lines around it
So now it's a rule. Every quote a machine gives me, I go find it in the transcript, read the lines around it.
Check any quote I pull against the recording, at least the bit around it
So I do a pass on any quote I pull, against the recording, at least the bit around it.
Answering personally for anything that goes out under my name Feeling that a number, quote or report carrying my name makes any error mine, including one a machine made.
Keep in mind that day to day it's my judgment, reports go out with my name on, and people decide things off them
Day to day it's my judgment, reports go out with my name on, and people decide things off them.
Insist that whatever I hand over is numbers I actually believe, because the counts go in a deck
Because the counts go in a deck? And we've got a new manager who wants, like, weekly numbers, so whatever I hand over better be numbers I actually believe.
Think that if some AI codes a whole document wrong and I put it in front of Priya and she makes a decision off it, that comes back to me, since it's my name on the analysis
it's my name on the analysis, so if some AI codes a whole document wrong and I put it in front of Priya and she makes a decision off it, that's, um. That comes back to me.
Hold that my name's on any quote I hand over, especially one a machine pulled, because a machine's mistake I passed on is on me twice
if I hand over a quote, my name's on it, in a way. Even if a machine pulled it. Especially if a machine pulled it, actually, because if I'm wrong, I'm wrong, but if the machine's wrong and I passed it on, that's, that's on me twice.
Weighing what the checking costs against what it protects Counting the time that checking takes, often unseen and unscheduled, and deciding it is still worth doing.
Notice the checking still takes the time, nobody schedules it, and it's invisible
The checking still takes the time. And nobody schedules it. It's invisible.
Weigh reading screeners by hand, an hour, hour and a half a week, as not that bad, faster than explaining a bad sample to a stakeholder
It's not that bad. It's, you know, an hour, hour and a half a week. Faster than explaining a bad sample to a stakeholder, I'll tell you that.
Want to check every code against the source, which might take as long as doing it myself
I'd want to check every code against the source, and that might take as long as doing it myself.
Think I spend more time checking things than making things, then ask would I rather hand over numbers I never checked? No
Sometimes I think I spend more time checking things than making things. But then I think, well, would I rather hand over numbers I never checked? No.
Know I spent longer checking the sorting than the sorting would've taken, and can't not
and then I spent longer than the sorting would've taken, checking the sorting. I know. But I can't not.
Feel it's not that slow to click every View source link, it's just boring
It's not that slow, honestly. It's just boring.
Noticing a quote has come loose from who said it and when Seeing that a quote on a sheet or sticky no longer carries the participant, the moment or the question that produced it.
Make peace, mostly, with the quote sheet losing the recording, the moment and the tone
You lose the recording, the moment, the tone. The quote says "that was confusing" and you can't hear whether the person laughed or sighed. I've made peace with that, I mean, mostly.
Regret the times I don't paste the participant ID on the sticky
Sometimes I paste the participant ID on there and sometimes I don't, and I always regret the times I don't.
Feel embarrassed that, ten years in, I lost my own source and couldn't tell the stakeholder which participant said the quote
I could not tell him which participant it was. I had to go re-read transcripts to find the sentence again. That was embarrassing. Ten years in and I lost my own source.
Weigh that the quote sheet is transparent, anyone can open it and filter, but the price is no link back to the recording and the quote floats free
It's transparent, is the thing. Anyone can open it and filter, sort of see where a quote came from. The price is there's no link back to the recording — the quote floats free.
Notice a quote on a sticky just floats there, not saying who, or when, or what I asked to get it
a quote on a sticky is just, it's floating there. Doesn't say who, or when, or what I asked to get it.
Figure the AI can't tell who said a sticky either, since a sticky doesn't tell you who said it
Because a sticky doesn't tell you who said it, so the AI can't either, right
Keeping one vivid quote or a loose count from passing as the finding Watching a single striking quote, a sloppy tag or a meaningless count turn into what the team treats as true.
Hold that I'm supposed to see across the participants and notice what's a pattern and what's one person having a bad day
I'm supposed to see across the participants, notice what's a pattern and what's one person having a bad day.
Expect the team to take a good quote as the finding, whether three people said it or one
the team sees a good quote and it becomes the finding, whether three people said it or one
Figure that if I can count how many people actually said the thing, the quote sheet gets more honest
If I can count how many people actually said the thing, the quote sheet gets more honest.
Realize that when Copilot dumps half the quotes into an "other feedback" category, the count means nothing
sometimes it invents a category that's just, like, "other feedback" and dumps half the quotes in it. Then the count means nothing.
Keep thinking that a tag is not the truth but turns into the truth by Friday, even one I put on sloppily on Tuesday because I was tired
That's what I keep thinking about. The tag is not the truth but it turns into the truth by Friday.
Watching a caveat fall off as findings travel Seeing a limitation or hedge written into a finding go unheard or get dropped between the sessions and the slide.
Feel that "this round" is doing a lot of work in my findings sentence, and the stakeholder doesn't hear it
"this round" is doing a lot of work in that sentence. The stakeholder doesn't hear it.
Can't bring myself to write that employees who already knew the workaround struggled with the promo code field, because then the study sounds pointless, and it wasn't pointless
I can't bring myself to write "employees who already knew the workaround struggled with the promo code field," because then the study sounds pointless, and it wasn't pointless.
Notice the further findings travel from the sessions, the more confident everyone gets, while my confidence goes down with distance
the further the findings travel from the sessions, the more confident everyone gets, and my confidence doesn't go up with distance. It goes down.
Know the word proxy falls off somewhere between my notes and the slide, and I'm the one who takes it off
It falls off somewhere between my notes and the slide, and I know exactly where it falls off, because I'm the one who takes it off.
Wonder whether I should put in one slide saying everything on this page is three degrees removed from a customer, weight accordingly, which I have never once done
Some weeks I think the honest output would be one slide that says "everything on this page is three degrees removed from a customer, weight accordingly," and I have never once put that slide in. Maybe I should.
Live off the word directionally for months, a good word that absorbs a lot
and I've been living off that word for months. Directionally. It's a good word. It absorbs a lot.
Take nobody ever asking about the limitation as telling you how the reports get read
Not once. Which tells you how the reports get read.
Know the write-up will get quoted without the limitation
I already know it'll get quoted without it.
Working out what will survive contact with the people who use the findings Shaping the work around what stakeholders will actually open, read and act on, and wanting it still to be a real answer.
Feel deflated when the stakeholder asks for just the top quotes after my week of careful synthesis
something in me deflated
Realize the stakeholder was right: they wanted the raw material and their own look at it
he was right. They wanted the raw material, their own look at it.
Make what I produce a sheet, because it has to go to a product team who aren't researchers and won't open a special tool; they just get a sheet without me standing there explaining software
And everybody has Excel, that's the real thing. Whatever I make has to go to a product team, and they're not researchers, most of them, and they're not going to open a special tool. But a sheet, they just get it without me standing there explaining software.
Hold myself to this: if it doesn't land, it doesn't count; a finding has to survive contact with a product team or it's not a finding, it's just me being clever somewhere
if it doesn't land, it doesn't count. The finding has to survive contact with a product team or it's not a finding, it's just me being clever somewhere.
Don't think anyone above me is lying awake about whether I did a proper analysis, they just want the answer
I don't think anyone above me is lying awake about whether I did a proper analysis, they just want the answer.
Insisting there is still a why to find when a number is wanted Holding that the reasons are not yet known while a stakeholder treats them as settled and asks for a survey count instead.
Keep coming back to this: we do not, in fact, know the why
That we do not, in fact, know the why. That's the thing I keep coming back to.
Think the survey number is noise: it's the same people, the loyalty panel, answering a question they don't understand
the number comes back, and the number is noise — it's the same people, the loyalty panel, answering a question they don't understand
Keep losing to this: a percentage with a decimal point in it beats twelve good conversations every time, apparently, in every room I've been in
I keep losing to that. A percentage with a decimal point in it beats twelve good conversations every time, apparently, in every room I've been in.
Feel the survey conversation eating at me, more than the tools stuff
It's eating at me, honestly. More than the tools stuff.
Noticing their thanks take away my reason to go deeper Finding stakeholder satisfaction with thin work worse than complaint, because thanks removes any push to go deeper.
Feel the quote sheet is thin, and know it's thin, while the team is happy with it, which is worse than them being unhappy
the quote sheet feels thin, and I know it's thin, and the team's happy with it, and that combination is worse than them being unhappy
Feel it gets me that they're happy with raw quotes
That's the part that, that gets me, actually. They're happy.
Weighing whether to push back and what it will cost me Deciding whether to argue with a decision, knowing one may look like one is defending one's turf or may simply lose again.
Expect that if I say it's redundant, I'll look like I'm defending my turf instead of giving an honest read
now if I say it's redundant, I look like I'm defending my turf instead of giving an honest read
Feel cornered: if I say nothing, we end up paying for two tools that do one job
Then we end up paying for two tools that do one job. That's the corner I'm in.
Suspect we'll keep both, because cancelling is a conversation nobody wants to have
I suspect we'll keep both, um, because cancelling is a conversation nobody wants to have.
Hear "it's not that simple" as the worst possible sentence the second it leaves my mouth, because it sounds like a researcher defending her moat
it came out as "it's not that simple," which is the worst possible sentence, because it sounds like a researcher defending her moat, and I heard it the second it left my mouth.
Brace for standing in the corner of delivering a policy I can't explain in one message, as the person who benefits from the manual work staying manual
A policy I can't explain in one message, delivered by me, the person who benefits from the manual work staying manual. That's the corner I'm going to be standing in.
Pick my battles, and I've picked this one before and lost
because you pick your battles, and I've picked this one before and lost
Try to figure out what the move is: keep arguing, or give him his survey and run the interviews quietly on the side, which is dishonest in its own way
So now I'm trying to figure out what the move is. Do I keep arguing, do I give him his survey and run the interviews quietly on the side, which is dishonest in its own way — I don't know.
Wonder if it's unwinnable: nineteen years and I still don't know how to win that conversation
I've been doing this nineteen years and I still don't know how to win that conversation, so maybe it's unwinnable.
Avoid being the person who brings the AI argument to a meeting, with colleagues loud about not using it
I don't want to be the person who brings that argument to a meeting.
Deciding whether the machine is seeing something or making it up Judging whether a machine-made theme, category or summary reflects what participants said, especially when it looks finished or fluent.
Realize a machine doesn't know what's interesting yet, it only knows what's similar
A machine doesn't know what's interesting yet. It knows what's similar.
Stay suspicious of anything that looks done, looks finished
I'm suspicious of anything that looks done. Looks finished.
Take it that the whole job now, with these things, is deciding whether the machine is seeing something or making something up
That's the whole job now, with these things. Deciding whether the machine is seeing something or making something up.
See the danger in it producing things that look exactly right: if I'd been rushing, and I'm always rushing, a stitched-together quote is on a slide Thursday and nobody ever catches it
And that's the danger, right, it produces things that look exactly right. If I'd been rushing — and I'm always rushing — that's on a slide Thursday and nobody ever catches it.
Argue with the AI's 'budget concerns' theme, because 'can't afford it' and 'waiting for a sale' are different people doing different planning
I argue with it? Like it'll make "budget concerns" one theme and I'm like, no, that's two things.
Don't trust the AI themes exactly, don't hate them either: they're like a colleague who's fast and kind of wrong in ways I have to check
I don't trust it, exactly, but I don't, I don't hate it either? It's more like a colleague who's fast and kind of wrong in ways you have to check.
Feel scared of an AI summary that sounds so good and fluent, more confident than I do on my best day, while missing the one person who said the interesting thing
that's exactly the thing I'd be scared of, right, because it sounds so good. It's fluent! It sounds more confident than I do on my best day. And it was missing the one person who said the interesting thing.
Hold that the payment screen complaint and the shipping thing aren't the same: one was about trust, she said the word trust
they're not the same, one of them was about trust, she said the word trust
Wonder how anyone would catch a summary blending two people into one except by me going back to the transcript
how would anyone catch this except by me going back to the transcript.
Sizing up what a machine's first pass is worth to me Judging a machine's themes, sorting or summary as a starting point: useful, sometimes better than one's own, but only a step.
Treat the AI themes like a second pair of eyes, a rough map of the territory, and then go read properly
I treat it like a second pair of eyes. It gives me a rough map of the territory, and then I go read properly.
Want a first pass I could check; I'm not precious about the manual labour, it's the checking I care about
A first pass I could check. I'm not precious about the manual labour, genuinely. It's the checking I care about.
Hate that as a first pass Copilot's categories were better than what I'd have jotted in that first hour
But as a first pass, and I hate that this is true, it was better than what I'd have jotted in that first hour.
Take the AI themes as a starting point, since some I wouldn't have thought of, like people reusing leftover paint
It's a starting point. Like some of the themes it comes up with I wouldn't have thought of
Think the FigJam AI sorting was fine: it grouped my whole wall in a second, stuff that would've taken me forty minutes
And, I mean, it was fine? It grouped my whole wall in like a second, stuff that would've taken me forty minutes.
Accept as fair that the help page says the sorting should be a step, not the whole job
it says somewhere that the sorting should be like a step, not the whole job, which — fair.
Admit the AI Insight Summary I read early on was good, better than my notes
it was good, that's the thing, better than my notes.
Worrying where the judgment goes if the machine does the reading Looking for the line between letting a machine read and letting it quietly take over the judging, sorting talk or reading where insight happens.
Hold that the team's talk about the clusters is the point, where the insight actually happens
we talk about the clusters, that conversation is the point, that's where the insight actually happens.
Hold that if you automate the sorting, you hand people conclusions they never earned
how the sorting is where people start to see the patterns themselves, and if you automate it you hand people conclusions they never earned
Wonder if I'm part of the problem: if the ten-minute pass exists and I check and fix it in three days instead of five, isn't that just better, and keep going back and forth on that
And maybe I am part of the problem? Like — if the ten-minute pass exists, and it's wrong, and I check it and fix it, and the fixing takes three days instead of five, isn't that just better? I keep going back and forth on that.
Don't know where the line is between the machine doing the reading while I do the judging and rubber-stamping the machine while the judgment quietly leaves the room
The honest version is I don't know where the line is between "the machine does the reading and I do the judging" and "I rubber-stamp the machine and the judgment quietly leaves the room."
Expect that if the ten-minute pass does arrive here for real, that hour is the thing I'd lose first, and it's the only part I trust
And if the ten-minute pass does arrive here for real, that hour is the thing I'd lose first, and it's also the only part I trust.
Realize I'm not defending the tools, which are mostly fine and mostly beside the point; I'm defending a way of working that only exists because the tools are slow
I'm not defending the tools, they're mostly fine and mostly beside the point. I'm defending a way of working that only exists because the tools are slow.
Seeing my slow work measured against a machine's seconds Feeling days of manual work set against a machine that does it in seconds or minutes, and expecting the comparison to be used against the work.
Sit with it that now the ten-minute version is the reference point and my five days is the disappointment
So now the ten-minute version is the reference point, and my five days is the disappointment. That's what I sat with.
Realize the comparison isn't with another researcher anymore, it's with a machine I can't switch on
the comparison isn't with another researcher anymore. It's with a machine I can't switch on.
Figure I'd ask too if I were Dan, since ten minutes versus five days isn't a hard call
If I were him, I'd ask too. Ten minutes versus five days, it's not a hard call.
Expect the next question won't be "can the AI do a pass" but "why isn't the AI doing all of it," and my answer will be a security policy and a shrug
The next question won't be "can the AI do a pass," it'll be "why isn't the AI doing all of it," and my answer will be a security policy and a shrug.
Realize that's my Tuesday night, and Copilot did it in a second
that's my Tuesday. That's literally my Tuesday night, and it did it in a second.
React to the word 'faster': everything is faster
That word. Everything is faster.
Wondering what it would say about me if I admitted I want or use AI Not saying out loud that one uses or wants the machine, and the story told to oneself about it.
Feel there's a version of me that wants it, and that's the part I don't say out loud
there's a version of me that wants it. That's the part I don't say out loud.
Feel that keeping quiet about using Copilot sounds worse out loud than it felt in my head
I haven't mentioned it to anyone. Which, writing that out loud, sounds worse than it felt in my head.
Tell myself it's different because it's my own rough material and never leaves my machine, and that's the story I'm going with
I told myself it's different because it's my own rough material, it never leaves my machine. That's the story I'm going with, anyway.
Looking for someone to check my own judgment Noticing nobody else checks one's codes or groupings, and seeking pushback, even from a machine, as the missing check.
Do my codes first, my own way, in the sheet, then on purpose ask the chat thing to argue against my own grouping: what did I miss, where would it cut it differently
And then, here's the part I do on purpose, I ask it to argue against my own grouping. Like, I do my codes first, my own way, in the sheet. Then I paste it in and say, tell me why this grouping is wrong. What did I miss, where would you cut it differently.
Value the pushback from the chat thing, not the answers
that's worth something. Not the answers. The pushback.
Realize nobody else here is going to push back, so I'm the only check on my own work, which isn't a real check
nobody else here is going to push back. Rachel would have, but she's out, and the researchers come to me for answers, they don't argue with me, and the stakeholders definitely don't. So I'm sitting there being the only check on my own work, which is — if you've done this job you know that's not a real check.
Prefer an AI that makes its case and tells me why, so I can say no, you're wrong, over a summary that just asserts stuff
I'd want it to make its case, honestly. If it codes something and it tells me why
Doubting whether my own coding caught what is there Wondering whether a theme or a pattern is real or only reflects one's own attention, tiredness or coding.
Wonder whether this is a real theme or just where my attention happened to be that day
is this a real theme or just where my attention happened to be that day?
See that I'd smoothed over one participant who's off in their own world because I was tired, which the chat thing noticed
it'll notice one participant who's off in their own world and I'd smoothed over that because I was tired
Catching myself in the bias I would fault in others Noticing one's own rut, inconsistency or convenient belief while doing the analysis, and naming it.
Notice I end up trusting the clean transcripts more than the messy ones, a bias I try to watch
you end up trusting the clean transcripts more than the messy ones, which is its own bias I try to watch
Realize noting something and moving past it is the same move as the proxy thing: note it, move past it, keep the schedule, and I do it, everyone does it
It's the same move as the proxy thing, I realize. Note it, move past it, keep the schedule. I do it, everyone does it, the wheels keep going.
Know I'm being inconsistent: two weeks arguing the AI can't do the pass for Dan while I run my quote column through Copilot at nine at night and tell no one
It's just me being inconsistent, I know. Two weeks arguing the AI can't do the pass for Dan, meanwhile I'm running my quote column through Copilot at nine at night and not telling anyone.
Know, after nineteen years, my own ruts: I over-code the thing that confirms what the team wants to hear, because it's easier to deliver that
Nineteen years, I know my own ruts. I know I over-code the thing that confirms what the team wants to hear, because it's easier to deliver that, right, Thursday's coming.
Think, knowing it was a bad thought, "at least the machine transcript's already there for the fifteen minutes", like that's a consolation
And the worst thought I had — and I knew it was a bad thought while I was having it — was, well, at least the machine transcript's already there for the fifteen minutes. Like that's a consolation.
Recognize the bad thought is sort of the same instinct as the survey guy: something exists, so it counts
it's, sort of, the same instinct as the survey guy, isn't it. Something exists, so it counts.
Realize I'm coding employees telling me how customers behave as if it's evidence of customers, even though I wrote the limitation
the transcripts are full of employees telling me how customers behave, and I'm coding it as if it's evidence of customers. I know. I wrote the limitation.
Decide the guy was overselling, because I don't want that to be true
I don't want that to be true, so I decided the guy was overselling. That's my honest process.
Arguing with myself over whether handing over quotes is enough Going back and forth between seeing the quote handover as a fair compromise that works and seeing it as having stopped doing the analysis.
Feel a bit like a quote vending machine, handing it over while they make of it what they want
Sometimes I feel a bit like a quote vending machine, I hand it over and they make of it what they want.
Wonder what we lose, now that skipping the synthesis layer is faster and the teams are happy
But I do wonder what we lose.
Feel a little bit of guilt, like I trained for the part I'm skipping
a little bit of guilt. Like I trained for the part I'm skipping.
Swing between thinking I'm adapting and thinking I've slowly stopped doing part of my job because nobody complained
Some weeks I think I'm adapting, other weeks I think I've just slowly stopped doing part of my job because nobody complained.
Make peace with handing over quotes, on some days
Some days I've made peace with it.
Hear a voice right before I hit send: this isn't analysis, you're a quote delivery service
this isn't analysis, you're a quote delivery service.
Hear the other voice say they made a good decision off my quote sheet last month, so on paper it works
they made a good decision off your quote sheet last month. The checkout thing. So on paper it works.
Facing what years in the job have come to Stepping back to see, with discomfort, that the work after a decade or more has become something one would not have called research.
Find it a strange sentence to say after ten years that the quote sheet is the core artefact of my job
The quote sheet is the core artefact of my job at this point, which is a strange sentence to say after ten years.
Realize you can't do research that way, and I'm doing research that way
You can't do research that way — and I'm doing research that way.
Feel that after eighteen years this is what it's come to, me and a highlighter at midnight
And then I think, eighteen years, and this is what it's come to — me and a highlighter at midnight.
Feel validated in a way I didn't love, hearing the podcast on research debt
It was validating in a way I didn't love.
Think if the debt's the point, the reading I do at midnight is just the interest
If the debt's the point, then the reading I do at midnight is just the interest.
Treasuring the immersed reading there is less and less room for Valuing the time spent properly inside the transcripts, where insight arrives, and noticing it is getting rarer.
Feel like I'm actually doing the job when I'm properly in it, patterns jump out and I know what to tag
Some days I'm properly in it and I feel like I'm actually doing the job — patterns jump out, I know what to tag.
Feel the transcripts are happening to me and I'm just moving text around
Other days it's more like the transcripts are happening to me and I'm just moving text around.
Know all my actual insights arrive while I'm re-reading a transcript at nine at night that nobody asked me to re-read
all my actual insights arrive while I'm re-reading a transcript at nine at night that nobody asked me to re-read.
Enjoy finding the sentence where somebody tells you the whole thing in nine words; it's still the best part of the job
Some nights the reading is genuinely good — you find the sentence where somebody tells you the whole thing in nine words. That's still the best part of the job.
Notice the good reading is rarer than it used to be, because there's more of everything else
It's just rarer than it used to be, because there's more of everything else.
Protect the sitting with it; everything else is just logistics
That's the part I'd protect, if anything. The sitting with it. Everything else is just, it's logistics.
Notice there's less and less of the half where I'm actually in it, following what people said
I'm actually in it, following what people said, and when it's good it's good. There's just less and less of that half.
Knowing I am skimming past what I should be reading Catching oneself scanning instead of reading, and relying on going back later to recover what was missed.
Feel it's worse that I know I'm scanning, once I stop reading and start scanning around transcript six
I stop reading and start scanning, and I know I'm scanning, which is worse.
Hate skimming, but the team wants numbers by Friday, so I skim
you start skimming, which I hate, but the team wants numbers by Friday
Tell myself I'll come back to the ones I skimmed, and only sometimes do
you tell yourself you'll, um, you'll come back to the ones you skimmed
Judging whether my makeshift method counts as real work Downplaying or defending one's own spreadsheet, colour or whiteboard habits as not a real system, or as dumb but one's own.
Feel the Excel work is dumb but mine, and nobody has to approve it
It's dumb work but it's mine, and nobody has to approve it.
Treat my yellow and blue row coloring as not a real system
it's not a real system, don't write that down.
Wondering whether the slowness is my own fault Suspecting the squeeze comes from being slow or missing some system others have, and feeling steadied to hear others have the same week.
Find it weirdly steadying to hear a stranger describe what was near enough my week
And I found that weirdly steadying, hearing it from a stranger.
Assume by default it's a me problem: that I'm slow, that other researchers have some system I missed
my default is to assume it's a me problem — that I'm slow, that other researchers have some system I missed.
Wonder whether the two weeks got shorter, the studies got closer together, or I got slower, and not know which
somewhere along the way the two weeks got shorter, or the studies got closer together, or I got slower, I don't know which.
Doubting whether participants are who they claim to be Suspecting a participant is fake, a repeat or not a real person, and second-guessing every panel and every good quote because of it.
Feel embarrassed that I'd vouched for the sample and told the product team it was fine, and it wasn't
Embarrassed, mostly. I'd, uh, I'd vouched for the sample. Told the product team it was fine, and it wasn't
Wonder if dscout isn't catching him or he's better than its fraud checks
either it's not catching him or he's better than it
Second-guess every panel I field
every panel I field, I'm second-guessing
Wonder if the polished checkout-diary response is a person, or I don't even know what
so polished it read like marketing copy, and I sat there thinking — is this a person, or is this... I don't even know what.
Can't get over that he still shows up on camera, whoever it is
He shows up on camera. Whoever it is. That's the part I can't get over, honestly.
Trust nothing, and give every good quote a second look: is this real
It makes you trust nothing, honestly. Every good quote gets a second look now. Is this real.
Stop at every third highlight and think, who is this person
every third highlight I stop and think, who is this person
Get a feeling mid-session that I know this guy, from a session and not the neighborhood
mid-session I get this feeling, I know this guy. Not from the neighborhood, from a session.
Feel bugged that I can't decide if it matters that I'm pretty sure he's in a study from last spring under a different name
And the bit that bugs me is I can't decide if it matters.
Wonder, sitting there that night, how many of the other participants are the same, because you don't know
But I did sit there that night wondering how many of the others — because you don't know.
Think at least the loyalty program people are actual customers, while the panel is just whoever
At least the loyalty program people are actual customers. The panel is just, whoever.
Weighing what stand-in participants can tell me about real customers Judging what employees or trained staff reveal and hide when they speak in place of the customers the study is about.
Think they're still users, they hit the same screens, and they caught real navigation stuff
they're still users, they hit the same screens, and they caught real navigation stuff.
Know we don't see all the friction normal people hit, because they know the returns policy backwards and don't call support when something breaks
But they also know the returns policy backwards, and they don't call support when something breaks, because they sit next to support. So the friction normal people hit, I don't — I know we don't see all of it.
Feel it sitting badly that the study is supposed to be about customers and I'm interviewing employees
that's the thing sitting badly with me, because the study is supposed to be about customers
Tell myself employee answers are directionally useful, since they watch customers all day and know the workarounds
I tell myself it's directionally useful. They watch customers all day, they know the workarounds, they know what people ask at the desk.
Hear that an associate's answer is what training told them, not what a customer would do
the answer is what training told them, not what a customer would do.
Think I'm coding a cleaned-up version of a too-fluent account of somebody else's experience, with the actual answer in there somewhere, probably, and I can't get at it
So I'm coding a cleaned-up version of a too-fluent account of somebody else's experience, and somewhere in there is the actual answer, probably, and I can't get at it.
Doubting what the machine transcript actually says Carrying doubt about misheard words and missing hesitations in machine transcripts, from amusement to worry once a mistake reaches a stakeholder.
Carry a layer of doubt under everything because the transcripts are machine transcripts
the transcripts themselves are machine transcripts, so there's a layer of doubt under everything
Miss the hesitations, because the machine transcript doesn't have hesitations, it has text
And I miss the hesitations, because the machine transcript doesn't have hesitations. It has text.
Take the machine transcripts as mostly okay, though they mangle the odd word
Mostly okay. They mangle the odd word.
Find a mangled word like 'I'd a band in it' funny until it's in the quote sheet and a stakeholder is reading it out in a readout
Which is funny, until it's in your quote sheet and a stakeholder's reading it out in a readout.
Think 'well. Thanks for the honesty' on finding the vendor says in the help pages that machine transcripts aren't guaranteed accurate
I remember finding that and thinking, well. Thanks for the honesty.
Seeing a participant who was giving me something good lost and counted as nothing Dwelling on a willing participant cut off mid-session, the thin or orphaned material left behind, and a system that treats them as a slot.
Feel it gets to me when someone drops off partway and you can't get them back into the same test
The part that gets me is what happens when someone drops off partway. You can't get them back into the same test.
Feel that either way the session is shorter, the transcript is thinner, and I'm coding thin material
either way the session is shorter, the transcript is thinner, and I'm coding thin material. That's the part I actually feel.
Can't get the dropoffs out of my head this week
But the dropoffs. That's the thing I can't get out of my head this week.
Feel it get to me that she emailed within ten minutes, wanted to finish, willing to do it that same hour, and I can't bring her back into the same test
And here's the part that gets me: she emailed within ten minutes, wanted to finish, willing to do it that same hour. And I can't bring her back into the same test.
See the system as built like every participant is interchangeable, like a participant is a slot, not a person
The system's built like every participant is interchangeable. Like a participant is a slot, not a person.
Understand why it's that way, as the ops person: panels, logistics, whatever
And I get it, panels, logistics, whatever, I'm the ops person, I understand why it's that way.
Feel she evaporated and her fifteen minutes are gone, not in the transcript in any way that counts, when she'd already told me more than half the people tell you in an hour
But this woman had already told me more in fifteen minutes than half the people tell you in an hour, and the system just — she evaporated. Her fifteen minutes are gone. They're not in the transcript in any way that counts, not in the analysis, nothing.
See it's not one woman, it's every dropoff, it's the no-shows, it's the pattern
Because it's not one woman, right, it's every dropoff, it's the no-shows, it's the pattern.
Feel it hurts when a participant drops out mid-session and the platform won't bring them back, as with the best session of the batch cut off at the payment step
That's the one that actually hurts. Technical problems — their mic dies, the app crashes, whatever — they're just gone, and the platform won't bring them back into the same test. You can't reschedule them into it. There's no "come back in an hour." The transcript just stops. Mid-sentence, sometimes. I had one last month, the best session of the batch, cut off right at the payment step. The exact question Theo cares about.
See 'session ended early' as all the machinery lets a partial be in the report
In the report it's "session ended early." That's all the machinery lets it be.
Doubting a secondhand sense that the sessions got worse Holding a feeling, borrowed from a colleague and never measured, that the platform or the pool got less reliable.
Carry around a feeling that the sessions got less reliable with the move, a feeling I haven't measured and got from Devi, as if it were mine
That's not something I've measured, it's a feeling. Devi's feeling actually, I got it from her, and I've been carrying it around as if it were mine.
Wonder whether dropoffs like this happened on the old setup and I didn't notice, since Devi ran half the sessions
Or maybe it did and I didn't notice, because Devi ran half the sessions — that's the honest caveat.
Believe Devi that the pool feels thin this year, even though I can't point at a number
But when she says the pool feels thin this year I believe her, even though I can't point at a number.
Working out what the time and sessions will stretch to Doing the arithmetic of days, sessions and slack against the work that has to fit into them.
Watch the clock, since the team wants findings in two weeks and reading alone eats five days of it
And the clock, because the team wants findings in two weeks and reading alone eats five days of it.
Figure the crunch version isn't the catastrophe people assume, since by Thursday I know that material cold, though everything else falls off that week
it's the same six transcripts, and by Thursday I know that material cold, so the crunch version isn't the catastrophe people assume. It's just everything else falls off that week.
Feel frustrated: two weeks to do two months of work
Frustrated, honestly. Two weeks to do two months of work.
Make it work: cut scope, do what I can
you make it work. Uh, you cut scope, you do what you can.
Count every screener burned on him as a real participant we didn't reach, since we don't have sessions to spare
And we don't have sessions to spare, so every screener burned on him is a real participant we didn't reach.
Count the small rebuilt things as adding up to time I don't have
Small stuff, but it all adds time I don't have.
Count every no-show as a burned slot and a live session I'm supposed to be moderating, with no slack across five in two days
But you burn the slot, and if you're running five across two days there is no slack. Every no-show eats a live session I'm supposed to be moderating.
Wonder at ten at night if I can read all twelve before Thursday, doing the same clock math again
there I am at ten at night doing the same math. Can I read all twelve before Thursday.
Judging that refusing costs more than doing what I know falls short Proceeding with a sample or study design one knows cannot fully answer the question, because the timeline and the stakeholder leave no other option.
Feel it was over before it started: the privacy review for actual customers was quoted at three weeks and the team wanted answers in two
So that was that, kind of. It was over before it started.
Don't feel great about the deal I made: I said this at the kickoff, got 'yes, noted,' and the schedule held
I said this in the kickoff and the answer was basically "yes, noted," and the schedule held. So that's the deal I made. I don't feel great about it.
Figure refusing is worse: the timeline slips a week and the team builds the thing anyway with no research at all, and I've seen that version
There's a version of this where I refuse, the timeline slips a week, and the team builds the thing anyway with no research at all, and I've seen that version. It's worse.
Know the difference after eighteen years, and do it anyway because the calendar says so
Eighteen years, I know the difference, I can feel the difference, and I do it anyway because the calendar says so.
See that a quick usability test on a new flow can't tell you why people abandon carts at the payment step
That's not a usability question. A quick test on a new flow tells you whether people can find the button. It will not tell you why anybody abandoned anything — the people who abandoned aren't in your test.
Let that participant become a row like everybody else, since I've got the two weeks
I've got the two weeks, so that participant becomes a row like everybody else.
Tracing the squeeze back to constraints nobody is fixing Seeing one time window and a few seams, not the tools or oneself, behind everything that goes wrong, and seeing no one fix them.
See everything else as downstream of the two-week thing
Mostly the two-week thing. Everything else is downstream of that.
See that the two weeks decides how many sessions I run, not me
The two weeks decides, not me.
Think the work is fine and the seams are bad; the tools mostly do what they say
The honest summary is the work is fine and the seams are bad. The tools mostly do what they say.
Name the seams as the two weeks, the privacy review, the panel, and learning anything new coming out of my sleep
It's the two weeks, and the privacy review, and the panel, and the fact that learning anything new comes out of my sleep.
Figure any one of those fixed would give me a month back
Any one of those fixed would give me a month back.
See that nobody's fixing any of them
Nobody's fixing any of them.
Realize it's all the same thing, the two weeks: the quotes instead of the real synthesis and the checking at night all get squeezed by the same window
it's all the same thing, isn't it. The two weeks. It comes back to that. Everything I do, the quotes instead of the real synthesis, the checking at night, it all gets squeezed by the same window
Asking whether to trust that whoever set a rule had grounds I cannot see Taking a security rule, a consent form or a deadline on someone else's word, wondering if the ground is solid, and carrying on.
Wonder whether anyone actually consented to the recording being transcribed automatically, going into my Excel sheet and decks, and being summarised by AI
But then the recording gets transcribed automatically, the transcript goes into my Excel sheet, quotes go into decks, bits of dscout get summarised by AI. Somewhere in that chain, did anyone actually consent to that?
Assume somebody upstairs checked because it's easier, while knowing that's not a reason
It's always easier to assume somebody upstairs checked, and I know that's not a reason.
Sit with the uncertainty and keep working, because it's not my form, I can't change it, and the two people who own it are drowning
it's not my form, that's the thing. I didn't write it, I can't change it, and the two people who own it are drowning. So I sit with the uncertainty and keep working, which I recognise is not great when I say it out loud.
Agree with the security reasoning for the AI being off, without thinking about it much
And I agreed with the reasoning without thinking about it much.
Wonder where the two weeks comes from, though I've never seen it written down and could be completely wrong that it's a leadership thing
I don't even know where it comes from, someone told me it was a, like a leadership thing, everyone gets the same window. Could be completely wrong. I've never seen it written down anywhere.
Losing hold of which explanation goes with which problem Mixing up which restriction or which platform applies, and understanding each explanation only briefly.
Read Devi going quiet as meaning wrong, after I told people the Germany thing explained the AI being off
I've told people the Germany thing explained the AI being off, and Devi just went quiet, which with her means wrong.
Think of the whole thing as Germany, not sure whether the AI part is the storage part; I'd have to ask Dana
Something about everything sitting in Germany — that's why the web version was out, so in my head the whole thing is Germany. Whether the AI part is the storage part — I'd have to ask Dana.
Wondering where participant text goes once it is in a tool Keeping identifying material away from AI tools because one does not know where the text goes.
Don't know where participant text goes in Copilot, and I'd rather not find out the hard way
Participant text going into — I don't know where that goes. I'd rather not find out the hard way.
Noticing tool decisions get made on grounds that miss the work Seeing tools chosen, split or renewed on money, licence count or habit, without anyone asking what they do for the research.
Feel the tool decision arrived over my head, when tool decisions are meant to be part of my ops role
it's a tool decision that arrived over my head. I do the tool decisions, sort of, it's meant to be part of the ops role.
Think nobody in this room has asked whether the sessions got worse, while the renewal conversation is all about the money and the licence count
nobody in this room has asked whether the sessions got worse. The conversation is all about the money and the licence count.
Notice we've got one vendor selling reliability and another one apparently delivering less of it
so we've got one vendor selling reliability and another one apparently delivering less of it
Notice nobody ever decided the vendor split on purpose, it's just where things landed, and the renewal looked at it only through the money lens
Nobody ever decided that split on purpose. It's just where things landed, and the renewal was the first time anyone sat in a room and looked at it, and even then it was through the money lens.
See only two of us in a project at a time, for twelve researchers, as a bottleneck, not a tool
Which meant only two of us could be in a project at the same time, and for twelve researchers, that's not a tool, that's a bottleneck.
Wonder what we're basing the vendor renewal on: vibes?
what are we basing that on? Vibes?
Trying to judge a tool I have never seen work Forming, doubting or being unable to form a view of a tool or feature one has never used or been shown.
Keep thinking I must be missing something, maybe it's better at the video side, maybe the sentiment thing is better
I keep thinking I must be missing something. Maybe it's better at the video side specifically, maybe the sentiment thing is better.
Feel bothered, more than I should, that I genuinely don't know what the trial gives us
I genuinely don't know, and that bothers me more than it should.
Think VoxPopMe and dscout overlap almost completely
So they overlap almost completely
Feel embarrassed that I'm arguing from a demo I saw in a tweet, same as Dan, having never worked with a good AI pass
I've never worked with a good AI pass, so I'm arguing from a demo I saw in a tweet, same as Dan, and that's embarrassing.
Think VoxPopMe is just another logo someone paid for, though I never formed a view
The trial window closed before anybody ran a real study in it, so I never formed a view. It's just — another logo someone paid for.
Wonder if calling it another logo someone paid for is unfair, since nobody showed me anything in it
That's unfair maybe. I don't know. Nobody showed me anything in it, so.
Feel frustrated that they won't even let us try the AI on fake data
they won't even let us try it on fake data. That's the part I can't, um.
Have no idea if AI Coding is good; I'm taking their word for it, and nobody here knows
I'm taking their word for it! I have no idea if it's good. Nobody here knows if it's good.
Want to at least see what I'm saying no to, even once
I'd like to at least see what I'm saying no to. Even once.
Wondering what the links between my codes could show me Suspecting that co-occurrence or query features would show relationships the codes alone do not, and feeling the loss of not getting at them.
Suspect the actual analysis lives in the relationships between codes, not the codes
I suspect that's where the actual analysis lives — the relationships between codes, not the codes.
Feel a nagging sense the tool is laughing at me
I just have this nagging sense the tool is laughing at me.
Feel it's a shame, kind of, about Atlas.ti: the basic coding part was fine, I didn't hate it, and I could see the point of the co-occurrence tables and the query thing
It's a shame, kind of, because the basic coding part was fine, I didn't hate it. I wanted the co-occurrence tables, the query thing — I could see the point of those.
Want to know what a co-occurrence table of our checkout studies would say, four years of them sitting in folders
I would genuinely like to know what a co-occurrence table of our checkout studies would say. Four years of them, sitting in folders.
Seeing that learning a tool would come out of time I do not have Counting the learning a new tool or feature would take against days and hours already spoken for by the transcripts.
Keep deferring the decision to read the query tool's manual, because there's never an hour where nobody wants anything from me
But there's no hour. There's never an hour where nobody wants anything from me, and the query tool has a manual, and the manual is a decision I keep deferring.
See that learning the tool happens on my own time, and my own time is the transcript pile
Learning happens on my own time, and my own time is the transcript pile.
Admit it sounds bad that there's no time to be curious, though it's true
That sounds bad. It's true, though.
Want to know, don't want to learn it, no time to learn it anyway
want to know, don't want to learn it, no time to learn it anyway
Think the co-occurrence stuff is completely gone and I'd have to redo the whole training, and I don't have a day
the co-occurrence stuff? Gone. Completely gone. I'd have to redo the whole training, and I don't have a day.
Figure learning a platform is a study in itself and I'd need a study that fits it, which is why I've never opened dscout
Time, mostly — learning a platform is a study in itself, and I'd need a study that fits it.
SYNTHETIC DATA PROTOTYPE Not research evidence · six fictional listening sessions
Germinal question Tell me what research you have been doing these last few weeks, and what goes through your mind doing transcript analysis work using our current tools?
How to read a window: the quote the participant said, then the summary. The line above names its thinking style, cognition type and window ID; the caption names the participant, and the transcript turn the quote came from.
White card at the top of each tower: what the tower's windows focus on, and its counts.
TS-01 · Carry the gap between the work and what I hand over I measure what I deliver against the fuller work I know it should be, and I tell myself a story that makes the thinner version acceptable without fully believing it. I do not settle the gap; I carry it as guilt, as two voices, or as feeling the difference, and keep working. People: Tomasz Wilk, Hollis Grant, Desmond Achebe.
TS-02 · Withhold trust until it's earned I notice when something starts to count as true because it looks finished, sounds sure, or simply exists: a tag, a survey number, a fluent summary, a finding that reached a slide. The more others trust it, the less I do, and I hold back trust until the thing has earned it, whether or not I act on that doubt. People: Nisha Rao, Jamal Whitfield, Rafael Duarte.
Cream capability: supports the tower above it.
Grey capability: partially supports it.
Blue-grey capability: leaves it unsupported, because the feature is switched off, on trial, blocked by policy, or not designed for it.
Pink capability: may harm it, for example by handing back answers that discourage the researcher's own reading.
Black street: cognition above, the tools' documented capabilities below.
In the transcripts below the skyline, each highlight is a window's quote, coloured by its cognition type. Its number matches the number on the window card.
TWO LENSES ON ONE BODY OF EVIDENCE Both lenses read the same reviewed concept records. Neither produces the other. The Skyline built every tower on this board; the thinking-style colors were layered on afterwards and did not build the towers.
Question it answers Skyline: What are people paying attention to when they face this purpose? Thinking styles: What distinct ways of reasoning exist among these people?
What gets grouped Skyline: One concept summary at a time, detached from who said it. Thinking styles: One person at a time, through their whole set of cognition cards.
Grouped by Skyline: Focus of Mental Attention — what occupied the mind. Thinking styles: Shared essential cognition — the inner logic a person runs.
A group is Skyline: A tower: summaries sharing one focus. Thinking styles: A style: people who reason alike.
Where a person sits Skyline: Scattered — one person's summaries land in many towers. Thinking styles: In exactly one style.
How it scales Skyline: Grows with evidence, then flattens as attention territory is covered. Thinking styles: Stays small. This board has 2, fewer than the three to five the method usually finds.
What it is for Skyline: Seeing the whole problem space, and what the organization supports. Thinking styles: Seeing that one solution will not serve everyone.
Decision it drives Skyline: Where support is thin, absent, or misaligned. Thinking styles: Who you are designing for, and who you are failing.
Must never group by Skyline: Person, role, chronology, topic, or style assignment. Thinking styles: Behavior, action taken, outcome, or stated preference.
On this board Skyline: 218 windows formed 39 towers in 9 blocks, with 28 capabilities and 46 alignments below. Thinking styles: 2 provisional styles across 6 participants, layered on as colour only.
HOW THEY MEET Two people can share one thinking style and act in opposite ways, so their summaries can sit in different towers. Crossing the two lenses is a later, deliberate step, and the crosswalk is many-to-many: a tower may hold every style, and a style may appear in many towers.
READ THE SHAPE 218 windows · 39 towers · 28 documented capabilities
18 towers have no aligned capability RT-T07 Watching a caveat fall off as findings travel; RT-T11 Wondering what it would say about me if I admitted I want or use AI; RT-T14 Catching myself in the bias I would fault in others; RT-T19 Doubting a secondhand sense that the sessions got worse; RT-T20 Treasuring the immersed reading there is less and less room for; RT-T21 Knowing I am skimming past what I should be reading; RT-T22 Arguing with myself over whether handing over quotes is enough; RT-T23 Facing what years in the job have come to; RT-T24 Noticing their thanks take away my reason to go deeper; RT-T26 Insisting there is still a why to find when a number is wanted; RT-T27 Weighing whether to push back and what it will cost me; RT-T28 Judging whether my makeshift method counts as real work; RT-T29 Wondering whether the slowness is my own fault; RT-T32 Tracing the squeeze back to constraints nobody is fixing; RT-T33 Asking whether to trust that whoever set a rule had grounds I cannot see; RT-T35 Wondering where participant text goes once it is in a tool; RT-T37 Noticing tool decisions get made on grounds that miss the work; RT-T40 Seeing that learning a tool would come out of time I do not have
12 towers have one aligned capability RT-T02 Answering personally for anything that goes out under my name; RT-T04 Weighing what the checking costs against what it protects; RT-T10 Seeing my slow work measured against a machine's seconds; RT-T12 Looking for someone to check my own judgment; RT-T13 Doubting whether my own coding caught what is there; RT-T15 Doubting whether participants are who they claim to be; RT-T16 Weighing what stand-in participants can tell me about real customers; RT-T18 Seeing a participant who was giving me something good lost and counted as nothing; RT-T25 Working out what will survive contact with the people who use the findings; RT-T30 Working out what the time and sessions will stretch to; RT-T31 Judging that refusing costs more than doing what I know falls short; RT-T34 Losing hold of which explanation goes with which problem
Densest support pile: 5 alignments RT-T08 Sizing up what a machine's first pass is worth to me
Tallest tower: 11 windows RT-T15 Doubting whether participants are who they claim to be
5 towers have a capability that may harm them RT-T02 Answering personally for anything that goes out under my name; RT-T03 Deciding whether the machine is seeing something or making it up; RT-T05 Noticing a quote has come loose from who said it and when; RT-T06 Keeping one vivid quote or a loose count from passing as the finding; RT-T09 Worrying where the judgment goes if the machine does the reading
12 towers are left unsupported by a capability that exists RT-T05 Noticing a quote has come loose from who said it and when; RT-T08 Sizing up what a machine's first pass is worth to me; RT-T09 Worrying where the judgment goes if the machine does the reading; RT-T10 Seeing my slow work measured against a machine's seconds; RT-T12 Looking for someone to check my own judgment; RT-T13 Doubting whether my own coding caught what is there; RT-T16 Weighing what stand-in participants can tell me about real customers; RT-T18 Seeing a participant who was giving me something good lost and counted as nothing; RT-T31 Judging that refusing costs more than doing what I know falls short; RT-T34 Losing hold of which explanation goes with which problem; RT-T36 Trying to judge a tool I have never seen work; RT-T39 Wondering what the links between my codes could show me
Read colour concentration carefully A concentrated style colour can suggest a perspective to investigate. It is not a population estimate.
Ask what an empty foundation means It may be a support opportunity, an intentional boundary, or evidence that the capability inventory is incomplete.
Use the shape to choose the next question Do not turn card counts directly into priorities. Investigate support quality, harm, organizational intent, and missing evidence.
58 summaries were filtered out before the towers, as not essential: 12 supporting detail instead of key point; 8 preference without decision rule; 5 scene-setting; 5 action without inner cognition; 2 duplicate or near-duplicate of P03-MM12; 2 duplicate or near-duplicate of P05-MM04; 1 duplicate or near-duplicate of P01-MM07; 1 duplicate or near-duplicate of P01-MM08; 1 duplicate or near-duplicate of P01-MM12; 1 duplicate or near-duplicate of P01-MM26; 1 duplicate or near-duplicate of P01-MM32; 1 duplicate or near-duplicate of P01-MM43; 1 duplicate or near-duplicate of P02-MM59; 1 duplicate or near-duplicate of P02-MM01; 1 duplicate or near-duplicate of P02-MM61; 1 duplicate or near-duplicate of P03-MM13; 1 duplicate or near-duplicate of P03-MM17; 1 duplicate or near-duplicate of P03-MM23; 1 duplicate or near-duplicate of P03-MM30; 1 duplicate or near-duplicate of P03-MM43; 1 duplicate or near-duplicate of P03-MM47; 1 duplicate or near-duplicate of P04-MM31; 1 duplicate or near-duplicate of P04-MM12; 1 duplicate or near-duplicate of P04-MM43; 1 duplicate or near-duplicate of P04-MM19; 1 duplicate or near-duplicate of P04-MM51; 1 duplicate or near-duplicate of P05-MM06; 1 duplicate or near-duplicate of P06-MM11; 1 duplicate or near-duplicate of P06-MM31; 1 duplicate or near-duplicate of P06-MM36. Each is listed with its reason in the run's affinity file.
Expect I was always going to be faster with a sheet
But I was always going to be faster with a sheet.
Think it's the incentives: we pay people, we pay quickly, word gets around
What goes through my mind is the incentives. We pay people, we pay quickly, word gets around.
Wonder what exactly the evidence is if I can't fully trust the transcript and can't fully trust the participant
If I can't fully trust the transcript and I can't fully trust the participant, what exactly is the evidence.
Keep telling myself the next study will be the one with room in it, though there's always a next study
I keep telling myself the next study will be the one with room in it. There's always a next study.
Feel I haven't gotten anywhere with Atlas.ti, obviously
I haven't gotten that far. Obviously I haven't gotten anywhere.
Feel a slightly bad taste anyway, even though Dan was happy, and try to work out at whom, though it's not him
And it left a slightly bad taste anyway, and I've been trying to work out at whom, and it's not him.
Suspect the actual problem is that I spend real energy on debates that never land anywhere, and the week just goes
And I've thought about it more since, which is maybe the actual problem: I spend real energy on these debates that never land anywhere, and the week just goes.
See the survey push as not exactly a him problem, with the PM: nice guy, don't get me wrong
nice guy, don't get me wrong, it's, uh, it's not exactly a him problem.
Brush off two logins and two places to lose things as just admin
Two logins, two places to lose things. It's just admin.
SYNTHETIC DATA PROTOTYPE — NOT RESEARCH EVIDENCE Every participant, organization, product, event and quotation is fictional. This board demonstrates an analytical shape and makes no claim about a real population.
METHOD BOUNDARY The 218 windows formed 39 towers and 9 blocks without any thinking-style input; the styles were found separately, from each person's cards, and added only as colour. 58 summaries were filtered out as not essential. The capabilities below the street are the tools' documented features at a fictional organization. Nothing here has passed a researcher's review.
QUOTE TO SUMMARY CRITERIA What makes a candidate admissible and a summary faithful.
Select language that expresses inner thinking, an emotional reaction, or a guiding principle. Scene setting, reported facts, explanations, and opinions are not interior cognition just because they sit beside it.
Represent one concept per record. When a passage carries distinct concepts, separate the phrases that support each.
Assign each phrase to exactly one record. A record may hold several phrases; a phrase may not appear in several records. This does not mean one sentence to one phrase to one record. If neighbouring phrases support different concepts, split at the phrase rather than repeat the wording in two records.
Label every record inner thinking, emotional reaction, or guiding principle, including when the cognition is implied rather than named.
Begin the summary with a verb that carries the action of the cognition. For an emotional reaction, use an emotion verb, or feel plus the emotion the quote supports.
Follow the verb with its key point, found by asking in the participant's voice, what am I verb-ing? For an emotional reaction, ask what occasions the emotion.
Extend the quote when it cannot support the key point, taking only as much nearby transcript as the key point needs.
Retest the verb and the cognition type against the key point. If the key point does not sit clearly under them, try another verb or another type instead of forcing the first reading.
Write from the participant's viewpoint, restating what they were conveying rather than explaining them from outside.
Prefer their memorable words. Do not swap in researcher, organizational, or domain terminology.
Borrow a word the quote lacks only from language the participant is likely to use, and preferably language they used elsewhere in the session.
Express the summary in first person, present tense, with the subject omitted so it opens on the verb. This holds when they narrate in past tense or say you meaning themselves.
Reorder and prune freely. Keep their language without keeping every word or its original order, so the cognition reads quickly.
Condense into a verb and key point that can stand in for the quote later. Not analytical notes, not a topic label, not a distillation of what the passage is about.
WORDS TO TRY — NOT A CONTROLLED VOCABULARY Prompts to widen the field before choosing, and to make the misfires recognizable. Never a permitted-words list; that would defeat QS-010.
Reach for a verb that names a thought process. Guiding principles lean on the commitment end, inner thinking on the working-it-out end.
Use feel plus the emotion, or one of the verbs that already carries its emotion. Feel worried and worry are equally good.
Search an emotion list when the emotion is implied, and brainstorm two to five before choosing. Move from the general emotion to the specific one the quote actually supports.
Hook supporting details onto the key point as a prepositional phrase, never as a second sentence. More than one phrase is fine.
Distrust these four classes. They are not banned words, they are signals that the interior cognition has not been found yet. The session-mode four are legitimate when the comparing or remembering happened inside the remembered moment.
HOW THE WORK GOES Rules that govern the work but are stated only in passing, or shown through worked examples rather than stated. Recoverable from watching the work, not from reading the method.
Take a verb the participant already used, and skip the brainstorm. Brainstorming exists for when no such verb is there.
Brainstorm two to five verbs or emotions, and generate more the less certain the reading. The wider brainstorm is what produces the more evocative word.
Let the candidate verbs decide the cognition type, not the reverse. The same passage resolves differently depending on which verb carries it best.
Prune before typing. Cutting back to the words that carry the concept is what makes the type legible, exposes tangled second concepts, and surfaces the words worth keeping.
Recover a pruned phrase once the key point is known and the cut phrase turns out to hold the memorable word. Pruning is provisional, not destructive.
Ask three questions before keeping an implied concept. How much do I trust myself to know this? Why did they mention it? Are they in memory mode or session mode?
Grade your confidence, and let the threshold move with the role. A looser reading can support a detail on an already-strong quote; the concept itself needs more. No supporting context anywhere means drop it.
Untangle by asking each candidate emotion what it is about. Two emotion words sharing a key point are one concept. Then ask when and where the thought happened; different moments are different records.
Join a repeated concept to the record it belongs to rather than opening a new one. You need not capture every repetition once the record holds the words.
Add supporting details only if they help you recall the concept and the person later. If the verb and key point alone bring the moment back in three years, no details are needed.
Repair the key point when writing details surfaces a dangling referent. Who is he? belongs in the key point, not the details. The details step is a check on the key point.
Read the summary aloud. Tripping over the wording is the signal; collaborators and stakeholders will trip in the same place. Reading silently lets fog pass as clarity.
Keep the personal context. Stripping it happens later when groups are labelled. A summary anyone in the study could have said was abstracted too early.
Hold one summary per record. Two verbs with two key points is two concepts wearing one row.
Search the transcript for vivid wording when a summary is correct but too thin to recall the person. Enrich from what they said; do not embellish. This is why every record needs a stable participant ID.
Swap the key point and the supporting details when the memorable material is sitting in the details. The front of the sentence should carry the cognition.
JUDGMENT CALLS Where the method does not resolve to one answer and the reviewer chooses. These are the calls that most need reasoning recorded beside the summary.
Accept that several summaries can be right for one quote. Clarity and recall decide, not correctness. Preferring different wording is not by itself grounds to reject a faithful summary.
Hold the cognition type loosely. The same cognition can be cast as inner thinking, emotional reaction, or guiding principle, and grouping does not depend on which. DStL's structural use of type is an open divergence, recorded rather than resolved.
Choose the verb that matches the shape of the cognition, not only its content. Proposing and discarding phrasings is figuring out, not concluding.
Cite where else in the session the participant used a borrowed word. The licence is how often and how saliently they said it, never that it sounds right.
Keep their word even when it is uncomfortable. Substituting a preferred term inserts the researcher's worldview.
Add a discriminator only to make the concept findable among others later. It adds no meaning, only the qualifier that keeps it distinguishable.
Drop one verb of a double verb and demote it to supporting details. Feel plus emotion plus to plus verb is not a double verb. If dropping removes something real, look behind it in the transcript.
Test try to by asking whether they are struggling, working at it, or aiming at it. If not, lead with the second verb.
Replace an adjective that is not an emotion with the emotion behind it. An imperfect emotion word that still reads clearly in three years is acceptable; do not burn time on it.
Look behind feel plus emotion plus by someone. Those name another person's action, not your emotion. Keep the surface form if the emotion beneath it is unrecoverable.
Allow self-description emotions to stand. Try a more precise word, keep the original if the recast is a struggle.
Demote somatic responses to supporting details. The body responding is not an emotion and may not fill the emotion slot.
Split before omitting when the brainstormed emotions scatter. Scattering usually means two concepts, not one unguessable emotion.
Use wish and hope freely, and do not agonize over their type. The only condition is that neither is a disguised preference.
Favor clarity over conformance. The rules exist to reach clarity quickly, not to be satisfied. A clear faithful summary slightly off formula is a lesser finding than an on-formula summary nobody can read.
Resist the generous reading. Crediting the participant with a consideration they never expressed is the characteristic failure. Ask what the whole session shows before granting a motive.
Prefer an over-split set to an over-merged one. Splits regroup harmlessly later; conflations do not. Expect to find most of the concepts in a transcript, never all of them.
FAIL CONDITIONS Conditions to drop or send back rather than polish.
Drop session mode. The participant addressing the listener now, rather than speaking from inside the remembered moment, was never a candidate.
Omit the exterior layer. Approval, preference, description, and statements of fact are not interior cognition. Where a feel-phrase hides real inner thinking, convert the verb; where it is opinion, drop it.
Return a summary with no key point. A verb with no object of attention has not been written yet.
Recast a gerund as a plain verb. The -ing form nominalizes the cognition and opens the slide into third person.
Split a compound sentence, or re-hook its second clause as a prepositional phrase. Complex sentences are fine; two subject-verb-clauses joined by a conjunction are two summaries in one record.
Rewrite third person into first person, present tense, subject omitted. Third person puts a step between the reader and the participant's mind.
Enrich a generic summary from the transcript, not by editing what is there. A well-formed summary that could have been said by anyone has lost the person.
Reject researcher notes and rewrite from the quote. Prose that distills what the passage is about, in words the participant never used, is analysis rather than synthesis.
Loosen a summary that follows their word order too faithfully. Their words and their voice, your ordering.
Record insufficient evidence as a legitimate outcome. Sometimes the listener did not dive in and the material is not there. Dropping is correct practice, and a pipeline that cannot say so will manufacture candidates.
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
CAPABILITIES ALIGNED BELOW
Mental model skyline: a synthetic data prototype
The Mental Model Skyline and Thinking Styles are Indi Young's methods. This board is George Jensen's interpretation of them, made for the Research CoP session with Indi Young.
Germinal question: “Tell me what research you have been doing these last few weeks, and what goes through your mind doing transcript analysis work using our current tools?”
Six fictional listening sessions with researchers and ResearchOps leads. Every person, organization and quotation is invented.
Towers, blocks, thinking styles and capability alignments are AI-proposed and not yet reviewed. Capabilities below each street are the tools' documented features.
03 — Making sure what I hand over is true
RT-B01 · 4 towers · Attention on whether each quote, code or number that leaves one's hands matches its source and can be answered for.
04 — Holding the line on what the research means with the people who use it
RT-B02 · 6 towers · Attention on what stakeholders take from the work, and on keeping its weight, caveats and open questions intact on the way to them.
05 — Deciding how far to let the machine into the analysis
RT-B03 · 5 towers · Attention on what machine output is worth, whether it can be believed, and what part of the researcher's judgment it would take over.
06 — Keeping a check on my own judgment
RT-B04 · 3 towers · Attention on one's own analysis as a possible source of error, and on the lack of anyone else to check it.
07 — Questioning what the job has become
RT-B05 · 6 towers · Attention on whether the work as now done is still the analysis one trained for and values.
08 — Doubting the material the analysis stands on
RT-B06 · 5 towers · Attention on whether the participants, sessions and transcripts are sound enough to build findings on.
09 — Working inside time I do not control
RT-B07 · 3 towers · Attention on a fixed window that decides what can be done, and on going ahead within it.
10 — Making sense of decisions made out of my reach
RT-B08 · 4 towers · Attention on rules and tool choices made by others whose grounds one cannot see, hold onto or influence.
11 — Sizing up tools I cannot get close to
RT-B09 · 3 towers · Attention on tools and features one wants to judge or use but has not seen work or has no time to learn.
15 — Quote-to-summary rules: George Jensen's reading of Indi Young's Emergent Data Synthesis, 62 rules
Reference only. These rules were not applied to the windows on this board; its summaries followed the pipeline's skill specs.
16 — Transcripts: each highlight is a window's quote, numbered to match its card