"This tool replaces thinking," is a technology problem we (humans) have faced before. It's a snark that I've seen pro-AI contenders take as well: I bet these same people would have complained about calculators! And books!
Well. They did, at the time.
We have records from centuries -- even millennia back -- of scholars at the time complaining that these new-fangled "books" were turning their students lazy; why, they can barely recite any poems in their entirety any more! And there are people still alive today who remember life before widely available calculators, and some of them complained -- then and now -- that bringing them into schools dealt a ruinous blow to math education, and now these young people don't even know how to use a slide-rule.
The human brain can, when called on, perform incredible feats of memorization. Bards and skalds of old could memorize and recite poems and epics that were thousands of lines long. This is a skill that is largely lost to most of the population. It's not needed any more, and so it is not practiced.
There is a definite generational gap, between the people who were trained on slide-rules and reckoning and the generation that was taught on calculators. There came a year, when that first generation grew up and entered the workforce, when you suddenly started encountering grown adults who could not do math -- not even the very basic arithmetic needed to count down from one hundred. I would go into a shop, buy an item for sixteen dollars, give the cashier a twenty and a one because I want a fiver back, and have them stare at the money in incomprehension -- what do? They don't know how to subtract sixteen from twenty-one. They don't know how to calculate a fifteen-percent tip. They did not exercise the parts of their brain that handle this, because they always had a calculator to do it for them.
Nowadays, newer point-of-sale machines compensate for this; they will automatically calculate and dispense the change, no subtraction necessary on the part of the operator. Nowadays everyone carries a phone, and every phone carries a calculator, so if you need to do these calculations, the tool is right there. As more and more transactions go electronic and card, and cash fades further and further out of daily life, these situations happen less and less; it's not a problem that most people can't do math (until it is.)
The people who complained that these tools-that-replace-thinking would reduce the ability of the broad population to exercise these cognitive skills weren't wrong. It's simply that, as the pace of life changed, the environment changed so that in day-to-day life these skills were largely unnecessary.
Isn't this, ChatGPT and Generative AI, just the latest in a long series of tool-replaces-thought that has, broadly, worked out well for us? What's different about this?
Well, two things are different.
1) In the previous instances of tool-replaces-thinking, the cognitive skill that it replaced was a discrete and, on a day-to-day basis, unnecessary outlay of energy. Most people don't need to memorize thousands of lines of poetry, or anything else for that matter. Most people don't need to do more than cursory levels of math on a day to day basis.
This, however, is different. The cognitive skill that is being obsoleted here is more than "how to write essay" or "identify what is the capital of Rhode Island." It encompasses the entire field of being able to generate new thoughts; of being able to consider and analyze new information; of being able to follow logical trains to their conclusions; of being able to order your thoughts to construct rational arguments; or indeed of being able to express yourself in any structured way. These cognitive tools are not occasional use; they are every day, all the time.
2) In the previous instances of tool-replaces-thinking, the tool was good at what it did.
Calculators may have replaced reckoning, but calculators are also pretty good at what they do. The calculator will, as long as you give the right input, give the right answer. ChatGPT cannot be relied on to do this. ChatGPT will tell you, confidently and unhesitantly and dangerously, that 2+2=5, and it will not care that it is wrong.
Books may have replaced memorization, and books certainly could be wrong; but a fact, once in a book, is pretty stable and steady. There is not a risk that the Guy Who Owns All The Encylopedias might wake up one day and decide -- to pick a purely hypothetical example -- that the Gulf of Mexico is called something else, and suddenly all the encyclopedias say that.
Generative AI fails on both these counts. It fails on every count. It's inaccurate, it's unethical, it's unreliable, it's wrong.
I remember some time ago seeing someone say (it was a video about medieval footwear, actually) that "humans have a great energy-saving system: if we can be lazy about something, we are."
This is not a ethical judgment about humans; this is how life works. Animals -- including humans -- will not do something the hard way if they can do it the easy way; this basic principle of conservation of resources is universal and morally neutral. Cognition is biologically expensive, and though our environment is not what it once was, every person still goes through every day choosing what is valuable enough to expend resources on and what is not.
Because of this, I don't know if there is any solution, here. I think pushing back against the downhill flush of the-easy-way-out is a battle both uphill and against the tide.
So I'll just close with this warning, instead:
Generative AI is a tool that cannot be trusted. Do not use it to replace thought.
