When new isn't enough - #441

September, 19 2025 by Nathanael Yellis

In my years as a consultant, I cooked up lots of interesting concepts for people to use their software tools to work differently. The work of identifying the root cause of solvable challenges, finding the conceptual ingredients for a solution, and then demoing the new way kept me busy. It's really engaging work! But coming up with a new (easier!, better!, etc.) way of working wasn't enough. You had to make the ideas resonate. It takes work to translate cool new solutions into real world language. Using demos, interactive proofs of concept, etc. go a long way in landing new ideas in a way that makes people want to adopt them. But even with the best communications and change management, new ways of working often failed to stick. A quick example: a short few months after configuring everyone's Outlook to talk with their CRM automatically, I'd learn the sales team returned to using whiteboards to track conversations and emailed Excel sheets to project new sales.As a consultant, you end up with a big list of failed projects: failures to launch, failures to stick, and failures to scale. Hunting around for root causes gives you a fair amount to learn (see above: communicating effectively, managing change). But sometimes there's just a mismatch between what you're able to recommend or the concepts you're able to demonstrate, and the reality in which the people you're trying to help work. I'm thinking less about software limitations, because those are almost always solvable, and more about the inability of any software to address the work that actually matters. In software consulting land, we'd often blame the victim, so to speak. The customer was just too backwards to work in the way we recommended. That is often true, but I think it's more true that context dooms improvement projects before they start. The first essay below offers a sense of what I mean. The work it takes to run a small upstate inn isn't the type of work that is readily or usefully addressable by the hottest trend in software, AI. They're trying to price the food menu based on piles of physical receipts and keep their collection of old buildings maintained enough for guests to enjoy. Do we really think that lack of AI game-changed is the innkeeper's fault?

The second essay, a workplace memoir of an engineer's 18-month run at OpenAI, tells us why AI doesn't yet work for so many types of work. Here we have an all-in software culture, where permissions don't really exist, people forge their own way, and bring something entirely new to market really fast. The whole entity and all of the work it contains are both classic software (SaaS economics! Free food! Build fast/break things!). This is all eminently addressable by AI and it's the context for how AI tools get delivered to the broader world. Is it any wonder that AI tends to work best for software concerns? Or, put more charitably, that current AI tools are best at doing differently and better what people used past softwares to do? Some of the coolest AI-ish stuff I've advised on has been using unstructured data (think emails and meetings logged to a CRM) to accurately update structured data (think company health score in an associated CRM record). This is the stuff software has been used for since the '80s. It's fascinating to see how the newest software is AI is scarily good at this work. But we're not quite seeing the work of our innkeeper revolutionized by the new tool.

The other week, we had a fun showcase of how people on our team use AI. There was some really interesting analysis of email threads, call transcripts, and notes. People were using nifty LLM prompts to consider and refine their approaches to the next email, conversation, or presentation. The coolest bit was, if you will, the democratization of what high executive function folks have been doing all along. Here a single knowledge worker can have the level of assistance (and number of assistants) previously reserved for top-level executives. I asked the question: is this really helping us slow down, be more thorough, and perform better? Is the big value individual use of little robot assistants that they force us to break down complex work into discrete steps, ensuring we understand exactly what we're doing as we host meetings, reply to emails, and contribute to working groups? If yes, that doesn't diminish the value of these new tools. An 'AI chief of staff' may be an absurdly over-hyped way to describe a harmless little robot, but helping people like me slow down and work more effectively is no small thing.