AI is turning the corner
...but there are still plenty of issues
A quick post this week, entirely human-generated.
Over the past six months or so, I’ve become a regular user of AI tools, particularly Anthropic’s Claude in various forms. As regular readers will appreciate, I haven’t always been a fan of the over-hyped technology, but it has become a valuable toolset.
Let me explain: When ChatGPT (a generative pre-trained transformer) hit the news, I gave it a shot—what I found was disturbing, because, although it gave intelligent-sounding responses, the intelligence was a facade. It went in circles, and occasionally off into parts unknown.
As I’ve often said, when confronted with new technologies, pay more attention to the adjective than to the noun: Artificial Intelligence is Artificial, first and foremost. It seemed intelligent but fell back on credible bullshit (aka “hallucinations”) when the algorithm should have said “I don’t know,” or “I’m not sure.” Like many humans, it has a blind spot where it is unaware of what it does not know. Unlike many humans, it doesn’t feel the slightest bit guilty when lying.
Indeed, as I noted recently, not only does AI bullshit effectively, it also exhibits characteristics of human bullshitters: The less it knows about a topic, the more emphatic its assertions become. As I’ve become more adept with the tools, I gauge my response based on the confidence of the assertions to prompt follow-up probes on extremely confident responses that require clarification or specific sources, frequently asking, “Are you sure?”
The Use Cases
I’ve found AI algorithms handy in places where language dominates, but their work product can be easily verified. First, I’ve been using Adobe Acrobat’s analysis tool to rapidly read scientific papers directly from PDF, and it highlights specific facts on the pages. I often want to know the technical details of how a measurement was done and what the experimental results are. But these nuggets are usually buried in a pile of justification and explanation that don’t matter to me. AI can read the entire paper and pull out key facts much faster than I can. Those facts can be cross-referenced and checked, so hallucinations don’t affect the results.
Second, I’ve been writing a lot of code in programming languages that I am not fluent in, including Python, SQL, JavaScript, HTML, Unix shell scripting, and others. This is an ideal use case because there is a massive amount of open-source code that is exact in its syntax, and AI can translate the analysis I want to perform into the code that will execute it very efficiently. It’s not perfect, but it can also debug efficiently, given the often lengthy and unintelligible error messages that computers spew out. I think of it as making programming much more egalitarian. You do need to know how computers work (they follow instructions to the letter, regardless), but you don’t need to know the arcane syntax rules involved with computer processing. Because testing can be done rapidly, hallucinations also don’t affect the results.
Finally, and perhaps most surprisingly, I’ve taken up golf for the third or fourth time in my long life, and I’m using AI to analyze my swing. I still suck, but I’m improving this time around. It turns out that there are indoor simulators that produce reams of data from every swing, including video. I’ve plugged these feeds into Claude, and the algorithm tells me which data and which swings to review to discriminate a good shot from a bad one. And my swing is becoming more consistent as I become more aware of my tics. It may be a hallucination, but I’m finding value.
The Downside
The intelligence is still artificial. I’ve found Claude to be a profoundly persistent ass-kisser. I’m regularly “absolutely right” when I point to a flaw, while the results it generates are highlighted by self-serving declarations of “Perfect!” and “I FOUND IT”. Needless to say, these assertions now lead me to examine the results even more critically.
Another feature I’ve noted is that, as the exchanges become longer and longer, unlike interactions with humans, the AI becomes progressively less intelligent. It has ONLY long-term knowledge, and no memory of any other communications, or even of previous exchanges in the same thread, which it must read with fresh eyes every time through. It does not learn on the fly.
To an AI, you are just the “human”. The longer you interact with the AI, the less and less it understands about what you’re looking for in an answer. My analysis: Because the algorithm takes the ENTIRE conversation stream and uses the series of words to formulate its response, any miscommunication becomes amplified in more prolonged interactions. So, if the artificial conversation isn’t taking the direction you want, it’s better to start fresh than to correct. A helpful approach I’ve found is to ask the AI to generate a prompt for a new chat to take over from the current one. That way, it analyzes and summarizes the entire exchange in a way that is easily edited to enhance clarity, and the next bot starts from a much more concise place.
The bottom line
AI is developing rapidly, but it should be considered as an aid to thinking, not a replacement for it. If you let a machine think for you, you can (and will) be replaced. So long as you are actively engaging with an AI work product, it can help you learn faster. As soon as you disconnect your brain in deference to a computer, you’re toast.