Playbook.Architectures & Artifacts.

A model judging a model returns a probability, not a boundary. You cannot audit a probability, you cannot reproduce it, and you cannot tell a customer or a regulator what the system will do the next time the same request arrives.

Deterministic rules are more work to write, and they only catch what you thought to anticipate. I would rather ship a boundary that holds plus a gap I can see than a judgement I cannot reproduce.

The policy gate this produces, in Spctre

Nobody hands a new contractor root keys because they are fast. The question was never competence or speed, it is blast radius and attribution. An agent changes neither — it just removes the person you would otherwise have asked.

Gates add latency and teams feel it. That trade is real: some work runs slower so the expensive class of mistake stays impossible rather than merely unlikely.

The agent governance work

Speed arguments are almost always proxy arguments. Two people who do not agree on what they are building will never agree on how long it takes to build it, and no amount of process fixes that downstream.

Sending it back looks like losing a week and it is unpopular in the room. It is still cheaper than the month spent shipping the wrong thing at high velocity.

Dual-track discovery and squad topology

Teams are visible and queues are not, so the blame lands on whoever is holding the work when someone looks. Mapping the value stream moves the argument off opinion: the wait between two steps is a number, and a number can be fixed. Almost every time, the delay is sitting in a hand-off nobody owns rather than in the hours anyone is putting in.

Mapping is unglamorous and it takes a cycle to do honestly, and it usually indicts a hand-off that a senior person designed. Adding capacity is faster to announce and does nothing, which is why it stays popular.

Product operations, delivery cadence, and the discovery loop behind them

The job changes register several times a day — a schema question in the morning, a pricing call after lunch. Someone anchored at one end handles half of it well and quietly drops the other half. AI raised that bar instead of lowering it: the tactical output and the deep analysis are both cheap to generate now, so the scarce skill is knowing which question to ask and being able to tell whether the answer is right. A PM who only operates tactically is competing with a tool that is faster. A PM who only operates strategically cannot judge what the tool produced.

It is a demanding bar and it narrows the pool. It also means telling someone plainly which end they are short on, which is a harder conversation than a rating.

A direct report on how this worked

Jungle Scout's defensibility was its data — quality, coverage, accuracy. Jungle Scout Market was a two-sided services business alongside it: we vetted freelancers on one side and Amazon sellers on the other, matched them for copywriting, photography, design, and translation, and took a commission. It worked. It reached roughly $1M. It also made the data no better, and it ran on the people who would otherwise have been making the data better.

I did not start Jungle Scout Market — it launched in April 2018, before I joined — but I expanded it before I shut it down. I read the revenue as validation and put more people behind it, and it was another two quarters before I accepted that a business can be working and still be the wrong business. Shutting it down meant giving up $1M and telling both sides of a marketplace we had asked to trust us that we were leaving.

The product, as it was announced (opens in new tab)

Before the Downstream Impact acquisition, enterprise retention was the worst number in the business. The data was not the problem: it was the best marketplace data in its category and customers said so. It could tell a brand exactly what had happened on Amazon. It could not tell them why, or what to spend differently on Monday, and enterprise buyers do not renew six figures for a description of their own quarter. Engagement hid it — people used it constantly and left anyway.

I read strong usage and strong data quality as product-market fit, and spent too long improving the layer we were already good at — more coverage, more accuracy, faster refresh — when the actual gap was that we answered one question out of three. By the time that was clear, it was not something we could build quickly from where we stood. It took acquiring Downstream Impact to get the advertising data the why needed and the levers to act on it. Retention recovered on the other side of that. Buying the missing half costs more than seeing it earlier.

What customers said after