How to measure work when AI makes the clever part invisible

So this is the one I haven't worked out yet, and I'd genuinely like other people's take, because I think we're about to have a measurement problem nobody's set up for.

We are very good at measuring effort

Here's the thing about manufacturing. We are really, really good at measuring it. Output per hour. Units per shift. Yield. Scrap rate. OEE. Cycle time. The whole discipline is built on the fact that the work is visible and countable. You can see effort turn into stuff. That's the entire game.

The clever part went invisible

Now drop an agent into the middle of it. It forecasts the demand. It optimises the schedule. It runs the design-for-manufacture and hands you a better part before you've finished your coffee. And here's the problem. All of that was the clever part. And all of it is now invisible.

So what exactly am I measuring? The line still runs at the speed the line runs. The units per shift might look identical. But the decision that made those units the right units, made at the right time, in the right quantity, that happened upstream, in a few seconds, inside a model. The value moved. The metric didn't move with it.

And this is where it gets messy. Because if the output number looks the same, someone, somewhere, is going to conclude the AI did nothing. And someone else is going to conclude the human did nothing. And honestly, both of those are wrong, but our measurement tools can't tell the difference, because they were built to count effort, and the thing that changed wasn't effort. It was judgement.

Think about that line from a couple of years back, every hire is six hires now with agents. Fine. But six hires measured how? Six times the output? Six times the value? Six times the decisions? Those are three completely different claims, and we just say the number like it settles something.

Measuring judgement, not throughput

So here's where I land, at least for now. We're going to have to start measuring the thing we never had to measure before. Not how much got made. Whether the right thing got made. Not throughput. Judgement. Not effort, because effort is the bit the machine just took over. The expensive, scarce, human bit is being right, and we have almost no idea how to put a number on that.

And I think the companies that figure that out, that work out how to see and reward the invisible upstream decision instead of just the visible downstream output, are going to run circles around the ones still counting units while the real value happens somewhere they're not looking.

So how are you measuring it? Genuinely asking. Because I don't think output went up is going to cut it for much longer.