Large language model AI is getting a lot better. I’ve written here before about a bunch of the problems with it, and many of those have been ironed out, but that may actually make our systems worse.
I’m obsessing about AI behaviors and systems right now. Specifically, what are the behaviors that people need to use if they are going to use large language model AI effectively, and how does the system support those behaviors?
Here’s an example — if I have an intern, and I delegate a task, the feedback loop might look like this:
Basically, if I delegate a task and the intern does it well, it will increase my trust in the intern, which in turn will increase the likelihood that I delegate more tasks to the intern, and it might mean that I start to ease back on supervision a little because I’m not worried about them getting it wrong. This is all pretty normal stuff. If I delegate five similar tasks successfully, I’m definitely going to feel more trust, delegate more, and supervise less. To do anything else would be illogical and a waste of my time.
So does this same feedback loop work with AI? Maybe if you are using the expensive top end versions (I don’t have enough experience or tokens to know about that for sure), but the average-ish AI that I and lots of normal folks frequently are using isn’t so consistent. Here are some examples:
- I’ve been using AI to format my research references for my new book on skills development and after formatting dozens of references correctly, it just fabricated an incorrect book author for one. I pointed this out, it apologized and gave me a reference with a different wrong author. It got it right on the third go round. This is after getting references right dozens of times.
- I was talking to a colleague who used AI to draft emails to let clients know they would be unavailable for engagements on certain dates. My colleague sent five successful similar emails. On the sixth, the AI decided to offer my colleague as a resource to help the client find somebody else to do the engagement, which committed them to possibly hours of free work.
Now, somebody will probably tell me that the AI is getting better and more reliable, and I’m sure that it is, or that interns can still make mistakes even after doing things a long time, which is also true. But I don’t think anyone would dispute the idea that ultimately the human doing the delegating is the responsible party in these scenarios. I caught the error of the author misattribution, but if I missed a different one along the way, it’s still on me if the book goes out with that error. And my colleague still feels an obligation to help that client.
What I am saying is that we may have a mismatch between what our personal experience tells us about the system feedback (if it gets it right several times, the next instance is likely to be right) and what technology does (gets it right many times and then still has unexpected errors). The responsible thing is that you (the human) are still checking on instance 52, even if it got the first 51 instances correct, but that’s not how our internal feedback system works. Our internal feedback system says you can dial down vigilance after instance 4 or 5 because staying vigilant is a waste of effort.
What I am also saying is that we have technology that doesn’t match our psychology in many ways, and sometimes that will create an embarrassing mistake or a little extra work, and sometimes it will likely have much more serious consequences. We will need to ensure our systems are set up to support our psychology if we want to avoid some really bad outcomes.
PS – when I asked ChatGPT (Medium) about this, here’s what it told me about the reliability of its own information:
For the current medium reasoning level, the best practical summary [of its hallucination rate] is: much better than it was a year or two ago, but still not dependable enough to trust unsupported factual claims automatically.
And literally while I was writing this post, Josh Cavalier posted this on LinkedIn about the same issue:
“Why should L&D care? Because reviewing AI work is a skill, and research shows it’s very cognitively demanding. Staying alert when the machine is right 97% of the time takes training, practice, and deliberate work design. That’s not an IT problem. That’s a people/performance problem.“
Added: And here is Markus Bernhardt talking about the The Decisions Your AI Is Making Without You
Shameless marketing plug: If you want to learn more about systems mapping for learning and development, I’m doing a 2hr workshop on August 13. If you are interested, you can learn more here: https://usablelearning.com/systems-thinking-virtual-workshop/