Research CoP whiteboard

Tracing concrete grounds

Notice the prototype looks impressive yet still leaves me unable to tell what it changed.

it was impressive, but I could not tell what it had changed.

Feel more nervous when a cleaner screen hides the evidence of what the creator did.

that made me more nervous, not less, because the mess had been evidence of what the creator had done.

Try to reconstruct the path the system took through the workspace.

I was trying to reconstruct its path.

Treat the feature as not ready if I cannot show its path in the ordinary interface.

if we could not show the path in the ordinary interface, then the feature was not ready

Conclude we are designing from the agent's ability outward instead of from the person's ability to stay oriented.

we have designed from the agent's ability outward instead of from the person's ability to stay oriented.

Realize recovery labels do not matter once the creator loses their map of the workspace.

The technical distinction between archive and delete was irrelevant in that moment. She had lost her map of the workspace.

Shift from treating the activity log as the fix to treating a proposed change set as the fix.

We needed a proposed change set.

I notice when the panel hides which system each fact came from and when it was last updated.

did not show which system each fact came from or when it was last updated.

I see those checks as persuasion, not evidence.

those checks are persuasion, not evidence.

I need the exact records, access path, and rule that make the recommendation acceptable.

I need to know the exact records, access path, and rule that made the recommendation acceptable.

I recognize that this is exactly the kind of mundane failure I worry about.

That was exactly the kind of mundane failure I worried about.

I treat a claim without a durable source as residue, not evidence.

a claim without a durable source is not evidence; it is residue.

I say the product cannot present the conclusion as reviewable if the reviewer cannot inspect the source.

the product cannot present the conclusion as reviewable if the reviewer cannot inspect the source.

I feel a little jolt when every individual fact is real but the custody of time is wrong.

I felt a little jolt. Every individual fact was real, but the custody of time was wrong.

I am trying to keep a small trace so I can learn which part actually needed correction.

Now I am trying to make a little trace for each beta action: input message, proposed slots, user edit, final action. Not to build a huge audit product, but so I can learn which part actually needed correction.

Surfacing usable reasons

Require the interface to show the inferred rule beside each group instead of hiding behind a polished action summary.

show the inferred rule beside each group, not just the resulting action.

I feel humiliated by having "the agent had three green checks" as the explanation months later.

“The agent had three green checks” would be humiliating.

I insist that the product be able to reveal the chain without reconstructing it later.

the product must be able to reveal the chain without reconstructing it later.

I want to hear whether the trace is actually useful or just satisfying to me.

I wanted to hear whether the trace was actually useful or just satisfying to me.

I need the chain to arrive at language a person can use in a conversation with their manager, customer, or auditor.

The chain has to arrive at language a person can use in a conversation with their manager, customer, or auditor.

I want wording that helps customers predict the interaction.

I wanted wording that helps them predict the interaction.

Keeping uncertainty visible

I feel that source-indifferent confidence is operationally upside down.

Operationally it felt upside down.

I realize the dangerous cases will look tidy, not absurd.

the dangerous cases would not look absurd. They would look tidy.

I fear the agent will reduce suspicion precisely when suspicion is needed.

I was afraid the agent would reduce the analyst's suspicion precisely when suspicion was needed.

I want the interface to preserve contradictions and missing evidence instead of compressing them into a confidence number.

I wanted the interface to preserve contradictions and missing evidence, not just compress them into a confidence number.

I want the evidence package captured at the moment of approval, including uncertainty.

I want the evidence package captured at the moment of approval, including uncertainty.

I feel calmer because uncertainty has somewhere to go.

I felt calmer because uncertainty had somewhere to go.

I feel almost irritated when we flatten a jagged reality into one clean number.

feeling almost irritated that we had made one clean number out of something I knew was jagged.

I want the agent to hold multiple commitments and keep the tension visible instead of quietly dropping the value that is harder to quantify.

I wanted the agent to hold multiple commitments and make the tension visible. If no option satisfies both, it should say that. A product should not manufacture resolution by quietly dropping the value that is harder to quantify.

Making contest easy

Judge the review by whether the creator can describe the change in their own words before it touches the workspace.

The creator could describe the change in their own words and notice a wrong assumption before it touched the workspace.

Aim to make disagreement cheap rather than making the agent's work unquestionable.

It was to make disagreement cheap.

See systems look accurate by default when correction feels like a fight and mismatches go unrecorded.

People stop correcting systems when correction feels like a fight. Then the system looks accurate because nobody records the mismatch.

Want the creator to argue with the reason and reshape the result, not just inspect mechanics.

I wanted the creator to argue with the reason, not inspect a pile of mechanics.

Check whether an explanation makes the action more contestable instead of merely making it sound reasonable.

I kept checking whether our explanation made the action more contestable or merely made it sound reasonable.

I do not want analysts to become a rubber stamp.

I did not want them to become a rubber stamp.

I ask whether the badge helps a person investigate or helps them stop investigating.

I ask whether the badge helps a person investigate or helps them stop investigating.

I want support people to have an easy way to say the agent is missing what a case is about.

I wanted the support people to retain an easy way to say, “The agent is missing what this case is about.”

I want both people to see and contest shared-money interpretations before those interpretations become advice.

I wanted us to redesign the moment so any interpretation about shared money could be seen and contested by both people before it became advice to either one.

I want contesting a suggestion to update assumptions, not trigger self-defense from the agent.

The agent should update the assumptions for this goal and show the revised options. It should not defend itself or generate a more persuasive explanation.

I feel angry when the model sounds helpful while refusing to listen.

I was angry because the model sounded helpful while refusing to listen.

I ask what human disagreement the agent makes easier or harder.

I also want to ask what human disagreement the agent makes easier or harder.

I want the agent's influence to remain visible, answerable, and subordinate to the people the data describes.

I want the influence to remain visible, answerable, and subordinate to the people whose lives the data describes.

I support an agent when everyone affected can see what it uses and says about them, add context before interpretation becomes action, and keep it as a source of options rather than an authority.

I can support it when everyone affected can see what it is using and saying about them, add context before interpretation becomes action, and keep the agent as a source of options rather than an authority over the relationship.

Marking action thresholds

See a dangerous crossing when the product moves from proposing into deciding without clearly marking it.

the product had crossed from proposing into deciding, but the interface had not marked the crossing.

Make the problem manageable by sorting actions into suggest, prepare, and commit.

I drew three columns on the whiteboard: suggest, prepare, and commit.

Want the human to cross the commit line unless they deliberately delegated that kind of action.

I wanted the human to cross the commit line unless they had deliberately delegated a specific kind of action.

Feel guilty for inviting someone into a test without a clear boundary.

I felt guilty, because we had invited her to test something without giving her a clear boundary.

Prefer explaining why the boundary exists over showing magic that depends on nobody asking where the work went.

I would rather explain why the boundary exists than show magic that depends on nobody asking where their work went.

Distinguish hidden consequential actions from ordinary suggestions by whether the consequence can happen outside my attention.

the consequence can happen outside my attention.

Use deciding what did not matter as a warning sign that a capability needs a visible boundary and a recovery path.

The creator saying the agent decided what did not matter. That is the phrase I use as a check.

I get excited that this is finally an agent in the actual sense, not a chatbot.

I was excited because it was finally an agent in the actual sense, not a chatbot.

I think the agent should prepare options and hold them temporarily, but let the owner send the message.

So I thought the agent could prepare options and hold them temporarily, but let him send the message.

I judge automation by whether mistakes are easy to repair or costly in trust or money.

I think about whether the mistake creates a conversation people can easily repair or a consequence that costs trust or money.

I delegate slower when the consequence travels farther from the screen.

The farther the consequence travels from the screen, the slower I want to delegate.

I want each person to have room to respond before a first reaction becomes a plan change.

The product needed room for each person to respond before turning the first reaction into a plan change.

I want the agent to slow the conversion from signal to optimization.

I wanted the agent to slow down the conversion from signal to optimization.

Scoping delegation deliberately

Want delegation scoped in the same terms people use to think about their work.

The scope needed to match how people think about their work.

Hold the rule that the person should knowingly choose how much control to hand over.

the person should knowingly choose how much control to hand over.

Try to preserve speed without making speed depend on surrender.

I was trying to preserve the speed without making speed depend on surrender.

Hold firm that delegation must stay deliberate and revisable even if the exact control mechanism changes.

What I would not give up easily is the requirement that delegation be deliberate and revisable.

I want a few explicit operating modes tied to recognizable responsibility.

I wanted a few explicit operating modes tied to recognizable responsibility: gather only, recommend with required review, or resolve within a customer-defined policy.

I do not want one simple switch to stand in for too much delegation.

I did not want “let the agent manage scheduling” as one switch. That sounds simple but means too much.

I want the product to earn permission through use without making people study the system.

I was asking how the product could earn permission through use without making them study the system.

I like the agent having to be useful before it has broad permission.

It also meant the agent had to be useful before it had broad permission, which I liked.

Keeping recovery within reach

Want each movement to feel like something the creator could have done and undone.

I wanted each movement to feel like a move the creator could have made and could reverse.

Feel annoyed when reversibility gets treated like polish instead of a core condition.

I was annoyed because I had brought up reversibility earlier and it had been treated like polish.

Decide the trapped feeling matters because a customer will feel it even more than I do.

If I feel trapped while I have engineering access, a customer will feel much more trapped.

Treat traceability and reversibility as the line between operating a feature and merely releasing it.

If we could not trace and reverse its important actions, I did not feel we had really chosen to operate it; we had just released it.

Become more open to delegated action when recovery can happen in a comprehensible human-sized unit.

She did not need to supervise each action; she needed a comprehensible recovery boundary.

Need wrongness to stay visible and recoverable rather than pretending it can be eliminated.

I needed certainty that wrongness would remain visible and recoverable.

I want fine-grained rollback because a single on-off switch makes teams hesitate to intervene.

A single on-off switch makes teams hesitate to intervene because the cost is too high.

I need a bad case, because without one I do not know whether the recovery path works.

if a pilot has no bad case, I do not know whether the recovery path works.

I want the last change to be visible and reversible when the agent misreads something.

I just made the last change visible and reversible because that was what my friend needed when the agent misread something.

Keeping responsibility attached

See after-the-fact logging and reliability benchmarks as missing the real question of orientation and authority.

A log after the fact is not the same as being oriented before something happens, and outperforming an intern does not answer who has authority.

Broaden control beyond approval to prediction, visibility, correction, and recovery.

Control could be prediction, visibility, correction, or recovery.

Feel exposed once I become accountable for whether the boundary actually works for creators.

I also felt exposed because I had become the person accountable for whether the boundary worked.

Evaluate agent capability by what authority leaves the person's hands and how they can understand, contest, or take it back.

I ask what authority leaves the person's hands and how they can understand, contest, or take it back.

Wish we had started by defining what the creator remains responsible for and what they need to stay responsible.

What does the creator remain responsible for, and what do they need to stay responsible?

Support a powerful agent only when the person can tell what authority they gave it, see its meaning, disagree without a fight, and recover without needing us.

the person can tell what authority they gave it, see the meaning of what it is doing, disagree without a fight, and recover without needing us.

I see the feature as borrowing authority from the user if we cannot preserve the chain.

the feature was borrowing authority from the user.

I come back to accountable workflow rather than catastrophic risk.

That brought me back to accountable workflow rather than catastrophic risk.

I realize automation can change the apparent performance of the human doing the remaining work.

Automation could change the apparent performance of the human doing the remaining work.

I start thinking about the operational record as protecting the analyst too.

I started thinking about the operational record as protecting the analyst too.

I feel embarrassed that we have been discussing accountability almost entirely as legal defensibility.

I felt embarrassed that we had been discussing accountability almost entirely as legal defensibility.

I want the system to make assistance visible in both directions.

I wanted the system to make assistance visible in both directions

I feel naming an owner is almost more important than model choice.

It felt almost more important than model choice.

I want responsibility to follow the decision being made, not float upward into an AI committee that cannot see the work.

Responsibility should follow the decision being made, not float upward into an AI committee that cannot see the work.

I can support probabilistic reasoning when the responsibility path is crisp.

I can support probabilistic reasoning when the responsibility path is crisp.

I ask what promise the human is still expected to keep.

I ask what promise the human is still expected to keep.

I resist leaving the promise on the human while moving the visibility into the agent.

What I resist is leaving the promise on the human while moving the visibility into the agent.

I can support the agent when every consequential action is tied to accessible evidence, an explicit rule, a bounded permission, and a person or process that truly owns the result.

every consequential action can be tied to accessible evidence, an explicit rule, a bounded permission, and a person or process that truly owns the result.

I take responsibility for foreseeable relational effects instead of hiding behind user responsibility.

I felt responsible for foreseeable relational effects, even if we could not control what couples did. Design cannot guarantee a healthy relationship. But we choose who sees which inference, whose framing becomes the default, and whether disagreement is treated as input or resistance. I did not want to hide behind “users are responsible for their own conversations” after designing an agent that enters with privileged data and a confident voice.

Matching access to audience

I keep the agent's workspace resembling the human's assigned workspace.

My rule is that the agent's workspace should resemble the human's assigned workspace.

I want access control before reasoning, not redaction after reasoning.

I wanted access control before reasoning, not redaction after reasoning.

I ask what permission someone gave for this interpretation, not just whether the data was accessible.

I wanted to know what permission Jordan had given for their individual transactions to be interpreted that way, not merely whether the shared account made the data technically accessible.

I need symmetry so the product does not quietly amplify whoever controls the settings.

I needed symmetry. Not because partners always have equal power—they do not—but because the product should not quietly amplify the person who happens to control the account settings.

I separate what the plan needs to know from what each partner is entitled to know.

I was separating what the plan needs to know from what each partner is entitled to know.

I feel angry when consent language ignores the social consequence of an inference.

I felt angry at the word consented because the onboarding checkbox did not describe this social consequence.

I want consent to attach to the meaning and audience of an inference, not just to data access.

I wanted consent to attach to the meaning and audience of the inference, not just data access.

I keep “not now” valid because treating missing permission as a hurdle makes the design coercive.

I also wanted “not now” to remain a valid answer. If the agent treats missing permission as a problem to overcome, the whole design becomes coercive.

I ask whose context serves which purpose and is visible to whom.

My question was whose context, for which purpose, visible to whom.

I support a pilot only if both partners separately enroll and can see exactly what the agent will use and share.

I supported a small opt-in pilot if both partners enrolled separately and could see exactly what the agent would use and share.

I prefer losing some activation to creating an asymmetric product by default.

I preferred losing some activation to creating an asymmetric product by default.

I treat audience as part of provenance because a fact can be accurate and still be wrongly disclosed.

In a shared space, provenance includes audience. A fact can be accurate and still be wrongly disclosed.

I make privacy legible so it does not look like secret negotiation.

Privacy itself needed to be legible so it did not look like secret negotiation.

I learn that hidden protective machinery can destabilize people as much as hidden agent reasoning.

The lesson was that hidden protective machinery can be as destabilizing as hidden agent reasoning.

I expose complexity when it changes what someone can reasonably believe about another person's knowledge or power.

When the complexity changes what a person reasonably believes about another person's knowledge or power.

I keep the feature bounded because separate consent may not be freely given in imbalanced relationships.

Separate consent may not be freely given.

Stopping inference from ruling

I want confidence to inform the decision without silently defining acceptable risk.

I wanted confidence to inform the decision, but I did not want it to silently define acceptable risk.

I trust explicit conditions more than a general confidence threshold.

I trusted explicit conditions more than a general confidence threshold.

I want the product claim to match the actual authority boundary.

I wanted the product claim to match the actual authority boundary.

I feel my body tense when spending data becomes a judgment.

I felt my body tense.

I resist letting the system offer one person's transaction judgment to the other as authority.

The system had turned one person's transactions into a judgment and offered it to the other person as authority.

I treat an unseen, unanswerable story about me as something that changes the relationship.

The agent had created a story about me in a space where I could not see or answer it.

I reject calling that output neutral when it recruits one person into managing the other.

this is not a neutral insight. It recruits one person into managing the other.

I feel lonely when everyone else stays on tone while I am worried about whose perspective the product privileges.

I remember feeling lonely in the room because everyone else was still focused on whether “allowance” was the wrong word.

I assume money data can make the agent's story arrive with extra weight.

Money data looks objective, so an agent's story can arrive with more weight than a partner's story.

I worry the conversation can start as an accusation that the person has to disprove.

If the agent says overspending, the conversation starts with accusation and the person has to disprove the system.

I worry softer copy can leave the same one-sided architecture in place.

worried the architecture would stay the same: observe one person, advise the other.

I realize a neutral shared observation still does not guarantee a neutral conversation.

I remember thinking that a neutral shared observation does not guarantee a neutral conversation.

I do not want the agent to become a third partner whose judgment carries extra authority.

I wanted it to increase the number of discussable options without becoming a third partner whose judgment carried extra authority.

I guard against the agent making one preference look factual because it speaks with data and confidence.

Because the system has data and a confident voice, it can make one preference look factual.

I know an agent has too much authority when its interpretation changes what one person can do to another before both can challenge the premise.

When its interpretation changes what one person can do to another before both can see and challenge the premise.

I reject making a policy decision sound like it naturally emerged from the data.

If the company has decided debt reduction outranks everything, it should own that policy. It should not make the model sound as if the conclusion emerged naturally from the data.

I do not let the agent's access become social authority by accident.

Do not let the agent's access become social authority by accident.

Enforcing hard boundaries

Realize I am not trying to keep the agent weak; I am trying to keep its power inside a boundary a person can understand.

I realized I was not trying to keep the agent weak. I was trying to keep its power inside a boundary a person could understand.

Treat trust as something shaped into the capability itself, not layered on afterward.

trust was not a layer we would add after capability. The shape of the capability determines whether trust is possible.

I need custody of the consequential boundary to be deterministic.

I needed custody of the consequential boundary to be deterministic.

I need a feature to prove not only that it can act, but that we can govern the act.

I need a feature to prove not only that it can act, but that we can govern the act.

I place my confidence in the system around the model, not in the model itself.

It was confidence in the system around the model.

I stop asking the model to become deterministic and focus on making custody around it deterministic.

I had to stop asking the model to become deterministic and focus on making custody around it deterministic.

I believe the cleanest security boundary is capability, not prompting.

the cleanest security boundary was capability, not prompting.

I trust systems more when the failure boundary is physical or enforced by code.

I trust systems more when the failure boundary is physical or enforced by code.

I enjoy turning the stable part of a probabilistic failure into a guard.

This is the part I enjoy: you observe a probabilistic failure and turn the stable part into a guard.

I do not treat blocked policy attempts as proof that the agent is safe.

A blocked policy attempt counted because the sandbox saved us; it did not prove the agent was safe.

Preserving authorship

See opaque rearrangement of unfinished work as taking over authorship without saying so.

If our system rearranges that and they cannot tell how, we have taken over a part of authorship without saying we did.

Understand the mess as the creator's memory of alternatives, not as waste.

The mess was not waste to her. It was her memory of alternatives.

Clarify that I am protecting continuity of authorship and orientation, not manual control for its own sake.

It was not manual control. It was continuity of authorship and orientation.

Want ambitious automation without asking people to become passive inside their own work.

I want to be able to support ambitious automation without asking people to become passive inside their own work.

Value surprise when the person can see a proposal as a proposal, choose it, and make it theirs.

She chose it and then made it hers.

I do not want the agent to make the owner generic.

I did not want the agent to make him generic.

Keeping judgment human

I want the agent to surface signals that help a person recognize the situation and then leave the response to their judgment.

I wanted the agent to surface signals that help the person recognize the situation, then let their judgment set the response.

I reject humane-sounding language that lets the system avoid humane judgment.

I did not want the model to supply humane language that lets the system avoid humane judgment.

I want the agent to remove work around judgment and surface evidence for it, not automate that judgment too early.

The agent could remove work around the judgment and make the evidence for it visible. If we automated the judgment itself too early, we would make the product look efficient while erasing the reason experienced support people are valuable.

I distinguish detecting a condition the team has decided matters from making the repair judgment itself.

I would be careful about calling it judgment. It is detecting a condition the team has decided matters.

I do not want our agent to absorb customer emotion so the organization does not have to take responsibility.

I did not want our agent to become a layer that absorbs customer emotion so the organization does not have to.

I support the agent when it carries known burdensome work and makes the team's value-creating judgment easier to see.

what work is known and burdensome, and what work is how this team creates value? I could support the agent when it carried the former and made the latter easier to see.

I try to separate messages where relationship matters from messages that are just mechanics.

I was trying to separate the messages where his relationship matters from the messages that are just mechanics.

I see that a simple reschedule can turn personal, so neat categories do not fully work.

The problem was a simple reschedule can become personal if the person has already moved it twice. So the category did not fully work.

I do not want the agent to become less humane in the name of consistency.

I do not want the agent to be more consistent by becoming less humane.

I watch for places where only the owner can add something and support that judgment instead of impersonating it.

Watch where the owner still adds something only they know. That is probably where the product should support judgment instead of impersonating it.

I avoid emotional interpretations because people can name their own emotions.

We also avoided emotional interpretations like “You seem anxious about money.” People can name their own emotions.

Bringing context forward

I make myself follow her choice points instead of rushing to complete a framework.

I made myself follow the next ticket and ask what was going through her mind at each choice instead of trying to complete a framework.

I want the agent to bring relevant past into the present instead of declaring emotions and prescribing empathy.

I wanted it to bring the relevant past into the present, not declare that the customer was frustrated and then prescribe empathy.

I want early usefulness to come from making the shared situation legible, not from helping one person police the other.

I wanted the first release to help both people see the plan, not help one person police the other. The usefulness had to come from making the shared situation legible.

I see the agent as useful when it translates priorities instead of judging which spending is rational.

I was thinking the agent could be useful as a translator between priorities, not a judge of which spending is rational.

I want priorities to come from the people in the moment, not from model guesses about identity.

the priorities should come from the people in that moment, not from demographic guesses or a model assigning them a financial personality.

I dislike labels because they turn a relationship pattern into a person.

I disliked them because they turn a relationship pattern into a person.

I want the agent to make the shared situation easier to see and generate options with explicit tradeoffs.

To make a shared financial situation easier to see, help each person bring forward what matters to them, and generate options whose tradeoffs are explicit.

I avoid adding more personality inference and prefer better option generation under explicit constraints.

I would not add more personality inference. The next useful intelligence might be better option generation under explicit constraints, not a deeper model of the people.

Following work to completion

I feel a little tired because every agent feature seems to create a new maintenance obligation that the sales story does not mention.

I also feel a little tired because every agent feature seems to create a new maintenance obligation that the sales story does not mention.

I refuse to measure agent activity and call it value.

I did not want us to measure agent activity and call it value.

I check whether the agent leaves the engineer with a better model of the failure.

What I was actually checking was whether the agent left the engineer with a better model of the failure.

I realize we may have automated the visible task while missing the work she is actually trying to finish.

we may have automated the visible task and missed the work she is actually trying to finish.

I realize speed is not the first problem here; the bigger risk is making the customer repeat a bad experience.

Speed was not her first problem. She was trying not to make the customer repeat a bad experience. The draft could actually hurry her into doing that.

I need to know whether the agent shortens the whole path to a resolution instead of just moving work around.

If the agent only saved typing but added a verification chore, we had moved work around. I needed to see whether it shortened the whole path to a resolution she felt good sending.

I wonder whether we built around the artifact because it is easy to show.

whether we had built around the artifact because the artifact was easy to show. A drafted reply is visible. Restoring a strained relationship is not one field in the interface.

I define progress by the support lead's completed purpose, not by queue movement or agent activity.

if the agent makes the queue move while customers come back, that is false progress. I wanted to define value from the support lead's completed purpose, not from the number of agent actions.

I worry an autonomous-resolution story will push us toward deflection when a human moment could teach the product something.

I worried we would design toward deflection even when a human moment was the product's chance to learn something.

I fear we will claim efficiency while moving effort to the customer.

We would claim efficiency while moving effort to the customer.

I refuse to add intelligence unless it changes useful work.

Do not add intelligence unless it changes useful work.

I feel relieved that the feature has a concrete place where it helps.

I was relieved that the feature had a concrete place where it helped.

I fear a throughput metric will redefine success so every uncertain case starts to look like leakage.

The fear that a throughput metric would redefine success. Once leadership promises an autonomous-resolution percentage, every uncertain case looks like leakage.

I know a feature belongs when I can point to a recurring moment where it improves work in a way the people doing it immediately recognize.

I can point to a recurring moment where it changes the work for the better, and the people doing that work recognize the improvement without needing the demo explained.

I do not feel I have a product decision when all I have is a list of capabilities.

If I only have a list of capabilities, I do not feel I have a product decision yet. I have inventory.

I treat a rollout as dangerous if it is locally useful but organizationally harmful.

A feature can be locally useful and organizationally harmful. I want the rollout decision to include how the surrounding system responds, not just how the interface performs.

I start from what the person is trying to bring to completion and where their judgment changes the outcome, not from which part the model can perform.

Do not start from which part the model can perform. Start from what the person is trying to bring to completion and where their judgment changes the outcome.

I see the agent as a way to save people annoying back-and-forth.

it would save people a lot of annoying back-and-forth.

I realize the task is not just finding a slot.

I realized the task was not just finding a slot.

I feel a rush when the feature reduces effort without asking for trust in a whole automated conversation.

I felt this rush because it was the first time the feature reduced effort without needing him to trust a whole automated conversation.

I tell myself to ship the part that makes the next action easier, then learn.

I should follow that feeling: ship the part that makes the next action easier, then learn.

I check whether the next automation removes named pain or just makes the agent story sound better.

I asked whether the next automation removed a pain he had named or just made the agent story sound better.

I choose the moment of a real interruption over the comparison page.

I had to choose whether I was designing for a comparison page or for the moment someone is standing outside a client's house trying to answer a reschedule. I kept coming back to the second one because that is where a small product can be obviously better.

I judge success by whether the agent removes juggling and leaves a meaningful choice instead of a draft to rebuild.

The question is whether the agent removed the juggling and left a meaningful choice, or created a draft that the person had to rebuild.

Carrying known paths

I count declining a case as part of the behavior under test.

If the agent declined a case, that counted too; knowing when not to act was part of the behavior we were testing.

I see that “billing ticket” is a topic bucket, not a useful rollout boundary.

“billing ticket” felt like a topic bucket, not a useful boundary.

I feel relief that the feature does have a place.

My first thought was relief. The feature did have a place.

I see that fit depends on known-good, reversible work with stable meaning, not just on whether a ticket looks simple.

the difference was not just complex versus simple. She already knew what good looked like, the action was reversible, and the account context did not change the meaning. The agent removed steps without removing judgment she cared about.

I think the agent belongs where it can carry a known path and support work whose path is still being defined.

the agent belonged where it could carry the known path and support her where the path was still being defined.

I refuse to let a complexity score decide boundaries when history can make an ordinary case meaningful.

I did not want a complexity score to decide that automatically, because the history made the apparently ordinary billing ticket meaningful.

I ask what part of the work people refuse to let become generic, because that tells me where the agent should stop or ask.

I started asking, what part of the work are you unwilling to let become generic? The answer told us where the agent should stop or ask.

I decide yes to a staged helper and no to a general autonomous inbox.

I decided yes to the agent as a staged helper, no to a general autonomous inbox.

I want the agent to handle uncertainty visibly instead of needing me to rescue it.

I wanted the agent to handle uncertainty visibly: if it cannot understand the time, ask; if the calendar changed, refresh; if it cannot send, leave a draft.

I want the agent to hold the tedious state in the background, not become the new thing the owner has to think about.

The agent should hold the tedious state in the background, not become the new thing they have to think about.

Spending attention carefully

I picture the agent doing broad gathering while the analyst spends attention where evidence conflicts or the policy boundary is close.

I pictured the agent doing the broad gathering and the analyst spending attention where the evidence conflicts or the policy boundary is close.

I need variation and exceptions to stay visible enough to wake judgment up.

So we needed variation and exceptions to remain visible enough to wake judgment up.

I fear wasting engineers' attention more immediately than breaking code.

The more immediate fear was that it would waste engineers' attention.

I become skeptical when “feel agentic” means hiding the machinery and making it fast.

I become skeptical because “feel agentic” often means hide the machinery and make it fast.

I accept a little friction when it is the instrument that lets us see what is happening.

For an experimental trajectory, a little friction may be the instrument that lets us see what is happening.

I treat some friction as the work when judgment matters.

the friction might be the work in the cases where judgment matters.

I fear support people will lose the context that lets them see recurring problems if automation hides routine volume.

I also feared support people would lose the context that lets them see recurring problems. If the agent absorbs all routine contacts, the remaining queue looks anomalous and nobody sees the buildup.

I treat a metric as decoration if I cannot name the decision it changes.

If I could not name a decision, the metric was probably decoration.

I feel good that the feedback loop works, but I also see how quickly noisy attention signals can train people to ignore the feature.

I felt good because the feedback loop worked, but I also saw how easily a noisy “needs attention” badge could train people to ignore the whole feature.

I treat attention as part of the product budget.

Attention is part of the product budget. Every alert spends some.

I keep signals in the background unless they can lead to a different action.

If a signal was not connected to a different action the person might take, it should remain in the background.

I resist turning assistance into a second queue of agent work that people have to supervise.

assistance can become another queue. The product should reduce the number of things people must hold, not manufacture a second layer of agent work to supervise.

Earning setup through use

I worry that we will give customers a control panel so complex that nobody can configure it.

Part of me worried we would give them a control panel so complex that nobody could configure it.

I worry the explanation loop will die if naming judgment turns into extra documentation.

If that becomes extra documentation, they will stop.

I want configuration without turning every customer into the product designer.

I want configuration, but not a rule engine that makes each customer become the product designer.

I want the magic of “just handle it,” but I do not want people to train a second product before it saves them time.

I wanted the magic of “just handle it,” but I did not want to create a second product that people have to train before it saves them time.

I try to find the smallest version that still feels useful when things get complicated.

Usually when something gets complicated I try to find the smallest version that still feels useful.

I think setup should start from one real thread and the step the owner wishes had gone faster.

I started to think the setup should begin by watching one real thread and asking what the owner wishes had happened faster.

I realize I have built homework when setup asks too much before giving value.

I thought, of course, I have built homework.

Writing memory explicitly

Prefer inspectable local rules over a personality model that quietly infers who someone is.

A local rule felt inspectable. A personality model felt like something we would use later in ways they could not anticipate.

Want the creator to decide when a correction graduates into reusable memory.

The person should decide when a correction graduates into reusable memory.

I do not want to treat every edit as a preference.

He might edit a message because he is in a hurry, not because that is his preferred style. I did not want to call every edit a preference.

I want automation to grow from draft-only patterns that the owner explicitly accepts.

I thought maybe the product could start with draft-only, then offer automation for patterns it has seen repeatedly and let him accept the rule explicitly.

I would rather give owners a simple handle they can control than a hidden style profile.

I would rather give them a simple handle they can control than a hidden style profile that sometimes imitates them badly.

I trust a memory more when it can be written as a rule the owner might have entered manually.

I would ask whether the memory can be written as a rule the owner might have entered manually.

I want the system to remember explicit priorities tied to a goal and context, not infer a stable type from transactions.

I wanted the system to remember explicit priorities tied to a goal and context, not infer a stable type from transactions.

Narrowing intimate data paths

Treat remembered corrections around unfinished work as intimate enough to feel invasive even when encrypted.

We could say we encrypt it and still make a product choice that feels invasive.

I feel that training on past messages as a default is heavy and a bit creepy.

I considered training on their past messages, but that felt heavy and a bit creepy as a default.

I choose a narrow data path I can explain.

I chose a narrow data path I could explain.

I feel nervous that clear consent language can make the feature sound scarier.

Nervous, because clear language can make a feature sound scarier than the usual vague “we use service providers.”

I treat a no as useful evidence that the saved effort is not worth the data path for that person.

If someone says no, that is useful. It tells me the saved effort is not worth that data path for them.

I want the system to degrade gracefully toward less inference instead of making privacy feel like refusing to collaborate.

That was important to me: the system should degrade gracefully toward less inference, not make privacy feel like refusing to collaborate.

I want stronger deletion and memory controls and unmistakable audience cues.

I would strengthen deletion and memory controls, make the audience of every input and output unmistakable, and study moments where one person declines to share.

Bounding learning exposure

I follow a rule of naming the failure, the harmed responsibility, and the smallest boundary that contains it.

My personal rule is to name the failure, the harmed responsibility, and the smallest boundary that contains it.

I support a narrower pilot with stronger measurement.

It made me support a narrower pilot with stronger measurement.

I try to find the smallest exposure that teaches beyond offline runs without making the customer absorb unknown failure modes.

I was trying to find the smallest exposure that would teach us something our offline runs could not, without making the customer absorb unknown failure modes.

I say yes when the experiment can surprise us and the cost of surprise is bounded.

That is usually what I need before I say yes: not a promise that the system is good, but an experiment where the outcome can surprise us and the cost of surprise is bounded.

I remind myself that customer failures are not a slot machine for our learning.

I had to remind myself that customer failures are not a slot machine for our learning.

I keep the blast radius no larger than my learning.

My discomfort does not mean stop; it means do not make the blast radius larger than my learning.

I look for speed inside a boundary, not speed by skipping the boundary.

I was looking for speed inside a boundary, not speed by skipping the boundary.

Testing claims against reality

Want a concrete failure the team cannot easily wave away.

I wanted a concrete failure that the team could see.

Refuse to borrow excitement from the demo and make customers repay it later.

I did not want us to borrow excitement from the demo and make customers repay it later.

I focus on the case where all the signals agree for the wrong reason.

My mind went immediately to the one case where all the signals agree for the wrong reason.

I ask what we would know when it succeeds incorrectly.

I was asking what we would know when it succeeds incorrectly.

I judge the pilot by whether analysts can explain and challenge recommendations and whether responsibility gets clearer, not by match rate alone.

I did not want to call the pilot successful because the model had a high match rate. I wanted to know whether analysts could explain and challenge recommendations, and whether the system made their responsibility clearer or more confusing.

I worry that we are calling a stack of poorly understood error rates intelligence.

we were combining four systems whose individual error rates we did not understand and then calling the combination intelligence.

I trace how reliability decays across a correlated multi-step trajectory.

If each step is ninety-five percent reliable, a five-step trajectory is not ninety-five percent reliable. And the errors are correlated

I keep asking which forty-one percent the benchmark is really solving.

I kept wondering which forty-one percent.

I refuse to treat a right outcome reached for the wrong reason as trust evidence.

If the system got the right outcome through the wrong reason, I did not want to count it as evidence that we could trust the trajectory.

I stay suspicious of gates built on ten good-looking runs because early agents can be cherry-picked unconsciously.

I was suspicious of a gate based on ten good-looking runs because early agents can be cherry-picked unconsciously.

I feel happy, but I resist the urge to generalize from one easy success.

I was happy, but I remember resisting the urge to generalize.

I fear the shape where a patch passes and still damages understanding.

That was my feared shape: a passing patch that damages understanding.

I want opt-out instrumented as a valid outcome, not treated as churn we should prevent.

I wanted opt-out to be instrumented as a valid outcome, not treated as churn we should prevent.

I need claims to encounter reality.

I think I need claims to encounter reality.

I want us to state in advance what counts against the feature.

I wanted us to state in advance what would count against the feature.

I distrust precision that has not crossed environments.

Since then I distrust precision that has not crossed environments.

I want quantitative monitoring plus recurring qualitative review.

I want quantitative monitoring plus recurring qualitative review.

I keep asking what the next test can teach us without pretending it proves more than it does.

During the decision I mostly kept asking what the next test could teach us without pretending it proved more than it did.

I support an agent when the claim, the authority we give it, and the evidence we have are the same size—and when reality has a cheap way to tell us we are wrong.

I can support it when the claim, the authority we give it, and the evidence we have are the same size—and when reality has a cheap way to tell us we are wrong.

I feel embarrassed that our demo would reward a draft that sounds right but drives the wrong next step.

I felt embarrassed because our demo would have scored the original draft highly for accuracy and tone.

I worry we are designing around one person.

Then I worried we were designing around one person.

I decide to watch more support leads and measure the whole resolution before turning my interpretation into workflow.

I decided we needed to watch more support leads and measure the whole resolution, including reopens and handoffs, before turning my interpretation into the workflow.

I let vivid stories open attention, but I wait for recurrence before turning them into features or rules.

The story should open attention, not close the decision. If several support people independently say they need to recognize a broken promise, then we can design a signal or workflow.

I want broader patterns to grow from specific cases without losing the cases that make them meaningful.

I wanted broader patterns to grow from specific cases without losing the cases.

I feel more comfortable with uncertainty when it is tied to observed behavior instead of a vague sense that the model may do anything.

They feel workable because they are connected to observed behavior. I am less comfortable with uncertainty framed as “the model may do anything.”

I feel stupid for widening the rule after maybe six examples.

I felt stupid because I had widened the rule after maybe six examples.

Growing scope after proof

I experience the narrower pilot as earning the next scope.

I experienced it as earning the next scope.

I want autonomy to expand through observed responsibility, not through a general belief that the model is getting better.

I wanted autonomy to expand through observed responsibility, not through a general belief that the model was getting better.

I need the decision to be renewable.

The decision has to be renewable.

I stay unresolved about how quickly people normalize the recommendations.

How quickly people normalize the recommendations.

I need the renewal decision to be part of operations, not a launch document everyone forgets.

The renewal decision needs to be part of operations, not a launch document everyone forgets.

I want the agent's permission to expire into reconsideration.

The agent's permission should expire into reconsideration, not persist forever because the original pilot went well.

I feel relief because revisiting authority makes the problem finite.

Relief, because it makes the problem finite.

I feel uneasy that review conditions could harden into permanent rules and miss new kinds of relationship harm.

I was also uneasy that our review conditions might become permanent rules and miss new kinds of relationship harm.

I keep a live question attached to rollout because shipping is not the end of the research.

I like having a live question attached to the rollout. It reminds the team that shipping is not the end of the research. The agent changes the work, and we have to keep listening to the changed work.

I decide I have enough examples when approval annoys me more than error scares me.

I probably decide when the same flow works enough times that I am more annoyed by the approval than afraid of the error.

I can support an agent when it removes juggling quickly, leaves the owner's judgment intact, and earns broader permission through visible, recoverable corrections.

I can support it when it removes a real piece of juggling quickly, leaves the owner to add the judgment that makes their business theirs, and earns broader permission through corrections people can actually see and recover from.

I keep the first release from gaining relational authority before we know how people experience it.

That the agent did not gain relational authority before we knew how people would experience it.

Checking my own pull

I protect the possibility of shipping something useful.

The possibility of shipping something useful.

I think we are about to encode our comfort as the customer's policy.

we were about to encode our comfort as the customer's policy.

I know my responsibility can make me over-index on containment.

That can make me over-index on containment.

I make myself name a path to yes.

I try to make myself name a path to yes.

I talk to the person doing the work because their concerns cut through my abstract scenarios.

Their concerns usually cut through my abstract scenarios.

I feel briefly vindicated, then worry that vindication will make me overreact.

I felt vindicated for about half a second, then worried that vindication would make me overreact.

I tell myself not to build the decision around being right.

Do not build the decision around being right.

I sometimes feel the same feature would be judged more calmly if we did not call it an agent.

I sometimes felt that if we did not call it an agent, the same feature would be judged more calmly.

I feel fear of being fooled by my own work.

There was some fear of being fooled by my own work.

I want outside friction before I become too good at defending my own work.

I wanted outside friction before I became too good at defending it.

I become less attached to autonomous repair as the destination.

That made me less attached to “autonomous repair” as the destination.

I keep extending the experiment because there is always another slice to inspect.

I tend to keep extending the experiment because there is always another slice to inspect.

I bounce between excitement, fear of looking slow, and curiosity about the runs.

I was bouncing between excitement, fear of looking slow, and curiosity about the runs.

I notice my habit of converting her story into workflow states before she finishes telling it.

I had a product manager habit of converting her story into workflow states while she was still talking.

I feel defensive when I am the one arguing for more clicks instead of fewer.

I felt defensive because product managers are usually the ones removing clicks.

I feel responsible for making a choice that teaches us, not one that merely makes everyone comfortable.

I felt responsible for making a choice that could teach us, not one that made everyone comfortable.

I want the feature to make my solo product feel much bigger than it is.

it could make the product feel much bigger than it is.

I feel pressure because everyone is adding AI.

Everyone is adding AI, too, so there is pressure.

I feel good having something I can finish, even while it feels like a compromise of the big vision.

I felt good because I had something I could finish. I also felt like I was compromising the big vision, but the users were more excited about the small loop than the big story.

I feel firm and a little anxious when friction could be remembered as slowing launch.

Firm and a little anxious.

I feel humbled when my safety mechanism introduces suspicion.

I felt humbled.

Mental model skyline: a synthetic prototype, not research evidence

“Think back to the most recent time you had to decide whether an LLM agent belonged in the product. What was going through your mind as you worked through that decision?”

Six fictional interviews. Every participant, organization, product and quotation is invented.

MM-B01 · Revealing grounds for action

CAPABILITIES ALIGNED BELOW

MM-B02 · Bounding authority before it spreads

CAPABILITIES ALIGNED BELOW

MM-B03 · Supporting work without replacing its owner

CAPABILITIES ALIGNED BELOW

MM-B04 · Earning adaptation on explicit terms

CAPABILITIES ALIGNED BELOW

MM-B05 · Growing scope from tested evidence

CAPABILITIES ALIGNED BELOW