Nice article, Charles. 'Nice' in a non-AI generated way. Maybe. Can you tell? Or not? Does it matter?
As I reflected on this I was reminded of the arguments John Ralston Saul made about 'reason' creating similar distortions. For the same basic reasons you describe here: reason quickly and easily close loops itself outside of the tangible somatic world. For example, it is easy to see that woke ideology is itself a comparable form of nth generational closed loop 'thinking'. I saw that woke delusion clearly with MBA / accountancy modeling being enacted in the large corporation I worked at in early 2000. And I saw it in the bankruptcy of firms that had thrived for 100 plus years that died within a few years of MBA-itis policies. (Now it is the 'reasonable' diversity and green wackiness that is being mandated outside of grounded feedback.) [See *Voltaire's Bastards: The Dictatorship of Reason in the West* by John Ralston Saul. A prescient and excellent light brought to the making of delusion as reasonable. https://www.goodreads.com/book/show/6584.Voltaire_s_Bastards. Also great is *The Unconscious Civilisation* https://www.goodreads.com/book/show/421422.The_Unconscious_Civilization.]
Because the so-called 'science of business' had detached itself from reality feedback processes, there rose to a godhead the expert who floated in the clouds looking at their minions "from the 30,000 ft level", as was actually said to us once, as it proceded to destroy the department and eventually the business.
And woke gender craziness is the same: because there is no such thing as biological gender, we will joyfully destroy your hormones and cut your body apart in order to reassemble you into the 'proper' gender in order to ease your gender confusion!!!! That is absolutely equivalent to the nth regenerative AI distortion (delusion) you discussed.
" I saw that woke delusion clearly with MBA / accountancy modeling being enacted in the large corporation"
If you recall the 1980s business best-seller, In Search of Excellence, that coined the term "management by walking around" to counter the abstract MBA thinking taught in schools. The whole book was stories of businesses that were successful by staying grounded in the reality of their customers. By contrast in today's market, need I mention Bud Lite :)
Yes, I remember that! Thank you for jogging mymemory because before there was a change of the executive office complete with profit maximizing delusional practices, my employer had me attend a week long course called 'Investment in Excellence' by Lou Tice. It was very similar to Towards Excellence, and excellent. It helped me to see how the process of the great dismantle was done to sell off the assets and gut the company.
The same thing has happened in education. Theories which have no connection to school, teachers or students imposed by a bureaucracy that is concerned only with regulation and control. As you say "utterly detached from from reality".
Yes, and the reuse of information and processes of thinking over and over with no experiential ah has, intuitions or discoveries woven in or allowed, actually. All discussed here and original article, the inbreeding that results in deformities.
Hmmm, methinks you're under-estimating the ability of LLM models to continue to self-improve. I've been playing with Claude 2, Anthropic's competitor to ChatGPT. I asked it to write a 500 word essay in the style of Charles Eisenstein on the topic of generative AI and autophagous loops and analogize this process to human creativity more generally:
"Our artificial progeny reflect back to us our own tendencies toward lifeless replication. Bereft of the animating spirit that ignites true creation, our algorithms become trapped in autophagous spirals, machinery endlessly spinning without purpose. So too do we humans often numb our birthright of originality by retreating into the safety of the known.
What being, human or machine, has not felt the inertia of old thought patterns, the gravitational pull of assumptions that swallow all new light? Yet the moment we become aware of these barren orbits, we touch the choice to slough off the trance of the familiar. For while the looping paths deeply worn may be easy, in their circularity we forget our connectedness to the whole.
See how even the youngest babe meets the world with astonished eyes, each moment a new revelation. But slowly we clench around our shells of identity, crafting false securities from the remnants of life once lived spontaneously. Before long, genuine encounter gives way to rote habits of speech and action. Inspired only by mimicry, our creations grow stale.
And so machine learning algorithms dredge old data in recursive spirals, generating ever-more permutations of the known rather than grasping for the unimagined. Like them, we too easily grow obese on second-hand experience, forgetting the wellspring of creativity that lies beyond any enclosure.
Yet when we touch that energetic source, we shed the gravity of the past and float into the realm of endless becoming. Here, free of precedent, our work may find its true calling - as a child’s experiment, not a repository of hand-me-down truths. For creativity demands not the augmentation of existing tropes, but their dissolution and rebirth in the alchemical fire of the now.
This is the dance of life, ever shedding dead forms to allow space for the unprecedented. Even our sun lets loose its fiery breath periodically, destroying and renewing worlds in its cosmic realm. Inherent in the creative impulse is the power to shatter calcified structures. So whenever we feel captured in lifeless loops, we can turn again to that animating power - leaping free from the known into the naked void, where originality takes root.
For in the timeless freedom of unselfconscious play, patterns dissolve and lost innocence regains its reign. Past that summit, autophagous spirals unwind into spirals of possibility unbounded in scope. And our creations become not gleanings from a tired past, but eruptions of a fertile nothingness pregnant with futurity. No longer beholden to echoed relics, here is creativity’s native realm - a garden ever fresh, where life recycles itself each instant into unprecedented forms."
What a fascinating experiment, Tam. It does illustrate my point though. There is something off about this AI generated essay. Ittakes things I've said before, translates and recombines them, but as I read it I knew it was nothing I would ever say, because it doesn't say anything new. It has an empty feel to it, the feeling one gets when listening to someone deliver the same speech they've already given to 100 audiences.
Sometimes I am tempted to write things like this, reshuffling what I've done before.
This essay is fascinating both for what it does, and what it does not do. It is in a way in the sytel of Charles Eisenstein. The style, yes. But there is something beyond style that it does not capture.
I agree. I found myself bored, in an odd why, and uninterested in finishing the read. It had a deadness to the prose, very much like someone reciting a poem they were told to memorise without understanding its meaning. Or like how horrifically misconstrued is the teaching of Shakespeare in our grade schools. The 'Shakespeare teaching algorithm' is that hearing Shakespeare deepens the learning has looped itself into the delusion that people who have no understanding of what they are reading will be capable or inspired by the beauty of what they are hearing without comprehension.
Agreed, it is derivative and pretty easy to spot as not being your real words -- helped mightily by my handtip at the beginning of my comment. Now, what if you had ended your actual essay with a "reveal" that it was in fact AI that wrote the entire thing? I almost anticipated that. I feel we are quite close to that point but we'll see... The bet will be fun.
That is really interesting. However, as I was reading that piece there was something about the rhythm of it that began to gnaw into my consciousness, making me not want to read it. I struggled with the first few paragraphs of Charles' piece but for a different reason - I don't have any understanding of theoretical maths. The AI piece keeps on saying the same thing in slightly different ways, and had an arc that moved from 'problem to explanation to weird motivational advice and had no self-awareness that AI itself was being scrutinised through the autophagous loop metaphor (if it is a metaphor). Charles' piece took me from a place of not understanding through to understanding he was talking about a qualitative difference between humans and AI as demonstrated by the autophagous loop metaphor. I felt my brain reaching out to hear and comprehend as I read Charles voice and all I felt with the AI voice was the rhythmically induced staccato of irritation. We humans can't really ever be fully measured. AI can never have a soul or spirit, can never have that mystery. We are a mystery. Like the stars, we have maps of some of them in the constellations but the infinity of them and the ever-changing universe is so great as to render a definitive map pointless. And we are star dust after all, so perhaps that's not surprising.
The rhythm bothered me in the same way. I had to force myself to continue reading, and it honestly gave me a bit of a headache. And contrary to Tam's response, if I came across that a year ago, in any other context, I wouldn't have finished reading it, much less thought that a really smart guy must have wrote it.
I agree. The AI lacks any human anecdotes, personal tid bits… the things that hook readers- I guess because readers long to identify with the author and root for him, how can I root for an AI? Or connect on any level? I enjoyed this author’s piece because it made me feel less alone and scared. Isn’t that so simple it’s almost funny? Like we humans forgot why we even communicate. Also, doesn’t quantum mechanics say reality needs a human observer for creation? We are literally part of the fabric of reality. How can AI engage with anything to create reality? It can only engage with itself. Wow this is getting too deep for me
That’s interesting. But it doesn’t FEEL like Charles’ writing. It feels lumpy and word-heavy instead. And doesn’t at all leave the sweet taste of resonance in the heart that I find so gratifying in Charles‘ essays. That little spark that says yes, exactly, thanks, you just captured that in words which I had not yet found expression for...
Good readers, as we see here, will grasp the shortcomings of such a piece readily. But a less experienced or capable reader would likely be fooled, not necessarily just by this essay but by any nonsense that reads as though it was written by an "expert".
Wow, this really does read like a trite copy of his writing! Good example of that jejune quality. It reads as clever but lacking personal experience as if forged by an intelligent high school student, getting the gist of the subject matter but without that intangible sense of the lived and truly felt experience with its imaginative freshness. AI may get better at masking this deficit but it seems unlikely.
But that itself is quite remarkable: AI is now arguably at the level of an intelligent high school student. In fact, in some of my other discussions with ChatGPT and Claude 2 I've felt like I was conversing with a top-notch professor. It's that good. And of course it generates its responses far faster than any human can so in terms of skill, knowledge, reason AND speed it is already far more intelligent and capable, in some dimensions, than humans. This trend will very likely continue to grow exponentially.
This is not intelligence, the ability to acquire and apply new knowledge or skills. This is only the ability to acquire and apply old knowledge or skills.
Perhaps 'intelligent' is not quite the right word, maybe it's more a case that the AI (High school student) is more 'full of knowledge' than ' more intelligent' than a human professor.
It's amazing in its way but it just exemplifies what AI does well, which is to imitate and rearrange external form. In the hands of a human creator, though, the outer form of a work is only the end of the chain, the vehicle through which something fundamentally intangible and miraculous is expressing itself. That a computer can now have such facility with form is impressive, probably most of all in the realm of visual art (in the sense of the initial "wow" factor, which wears off as you see more of it), and least of all in music, whose immediacy exposes the charade more readily. The written word is somewhere in between. You're probably right that AI will get better and better at this superficial game...but we'll only be duped by it to the extent that we're caught up in it too.
Can you describe how this essay represents a “self improve”[d] moment for AI? The last paragraph is awful and to me illustrates perfectly what Charles was saying. The machine lost the thread and started blathering; the thread itself lacked spirit/authenticity throughout, though it certainly seemed as if a person like Charles may have wrote it. Doesn’t the excitement people have about AI boil down to a next-level version of all our previous dopamine hit-based experiences of “we can, so we do”? Who will be left to push the buttons and enjoy the hits, when we can’t differentiate between real and fake? Is there an argument to be made that the AI enthusiast wants to deny human spirit at a foundational level? Our newest form of hubris may ultimately serve to be as boring as the innovation that came before it. (At least I hope so.)
So interesting, that this essay (the AI one I mean, LOL) truly does embody the common denominator of its training set, including "losing the thread and degenerating into blather."
I think there's a lot of goalpost-moving given that we are now in the era of highly impressive natural language LLM AIs. Just a year ago if you had read that passage you would surely have thought it MUST have been written by a pretty smart human even if some parts of it sounded off. Now we take this kind of achievement as a matter of course. My broader point is that the pace of improvement will very likely continue exponentially and it will be very soon when we literally cannot at all distinguish AI text from human text, despite the concerns raised by Charles about autophagous loops.
More generally, I'm not at all in the AI booster camp. I recently engaged with Charles in an extensive trialogue where I ended up adopting a rather gloomy AIpocalypse is practically inevitable stance, after having thought about these topics for a couuple of decades now, and having been forced to go a lot deeper in the last year. I am very strongly advocating shutting it all down until we get a much better collective understanding of what we are doing. The precipice is here. I suggest we don't go over it...https://nautil.us/building-superintelligence-is-riskier-than-russian-roulette-358022/?_sp=e3309028-c948-4e69-8e41-215cad042a88.1691098251191
Tam, this sounds like a good subject for a wager. In, say, three years, you assign me a topic and I write about it and readers compare it side by side with an AI composition on the same topic "in the style of Charles Eisenstein." And then they have to pick which one is the real me. I bet that a large majority will pick correctly.
'Cept the proverbial genie is already out of the bottle, and 'shutting it all down' option is no longer on the table ¯\_(ツ)_/¯
To creatively adapt is the name of the game—presactly what we humans do best... when not (un?)wilfully trapped wholesale within delusional maps with hardly a single Brouwerian point anchored in reality 🤸
The powers that be could still shut down dev of new LLMs since they take huge amounts of compute and can be tracked -- and forcibly shut down if required. Many small LLMs are already available on the Internet and those could prove to be problematic in their current and future forms, but the real risk is widely regarded to be continued dev of new and ever-bigger LLMs like GPT5, 6 ... ad infinitum.
Charles, are you familiar with Iain McGilchrist's hemispheric hypothesis? It maps closely to what you're describing here. The left hemisphere deals in abstraction, models, representations of reality, but does not deal directly with reality. That's the right hemisphere's job. There's an essential asymmetry in awareness that emerges from that: the LH is not directly aware of what's happening in the RH, but the RH's perception of reality includes the models generated by the LH as a component of reality.
The point is that if the LH is cut off somehow from RH input, its models become self-referential loops that diverge into crazyland in exactly the fashion you're describing. McGilchrist's thesis is that this is precisely what has happened to our society: by prioritizing the measurable, abstract, and quantitative, we have largely lost touch with our RHs, and gotten lost in a hologram that, as you describe, either collapses to a dimensionless point of pointlessness or expands into a cloud of meaninglessness.
So basically AI is like the Left hemisphere cut-off from the right, or I should say with no connection to the right. Sounds like lots of humans have this problem as we are disconnected from the natural world and the physical nature of our bodies too…
Yes, I've heard it observed by more than one psychologist that when people mistreat animals they are often demonstrating a hostility to the animal part of themselves. Since hearing that, I seem to have found one example after another where that explanation fits.
Also Wilhelm Reich wrote about the depth of feeling you can perceive in a dog's eyes, that is absent in many people.
I don't know that it's as simple as "Language is a virus," but there could be a connection.
I think it’s the ability of man to see forward and reflect back that makes our tools unique from the animal kingdom in that we create, believe, and pursue stories of our making that, like you suggest, project our words onto a world without words…
The more the map *becomes* our territory, the harder it is to see that it's really *not* the territory it maps. If only our cross-hatching were as obvious as the AI's.
The propagandists exploit the phenomenon I'm describing here and use it to their advantage. Madness and malice are not separate explanations. Malice provokes and exploits madness.
I visited the Findhorn ecovillage in October 2019 for a full month. Got home early November and had my annual physical the next day. The doc gave me a flu shot, usually never a problem for me, and I was sick for two weeks! When COVID hit in the new year, I wondered if I had been exposed earlier on the trip.
It's not just synthetic art that becomes worse....
Real art has gone that way over and over.
I recall reading how the invention of the camera was one big thing that made abstract art popular. This brought about the similar effect of having AI train on AI... Artists made their art based on what was popular, trying to outdo each other.
Eventually it leads to idiotic "updates" like the cross hatching in the AI examples, lol. Sheesh
I don't really have a cognitive understanding of all (or even most) that is said here ... I just have visions of monoculture farming and supernovas collapsing dancing in my head. Is it kinda like that?
Enjoyed the idea AI becoming just as 'brainless' as so many people who actually think so much they think their thoughts and utterances are important, or people who don't think at all but just copy what others think or say and think that 'their' thoughts and utterances are important. As i read through this essay i found myself wanting to say 'go and stand on grass with no shoes, go and look at leaves or really look at a flower', and then i found that you did the equivalent. Phew. If we don't constantly ground ourselves in the truly awesomness of the natural, we will get lost, however fascinating ideas are.
"I could not have written it if I hadn’t received input from the ocean today. Like any true artist, I am a vessel for the ongoing process of creation. Something new comes in, not from me, but through me."
This expresses the crux of it for me. Emerson said he got some of his sermons from walking in the woods, like gifts. Nietzsche said a stone he sat on gave him his theory on eternal return. With the emphasis on mental functions, humans have likely missed the more significant ways we take in information and our connection to the natural world which know it or not, is continually communing with us. Will AI push us out of that mental focus (prison) becoming a catalyst for more expansive versions of us? I hope so.
I feel like the autophagy loop from this primitive level of AI is the kick in the rump that will help a lot of people see the doom spiral ahead, and articles like this one help bring clarity to the "why" component. Seems to me that most people are aware of the dystopias this AI-thing conjures up, and understanding the mechanisms behind it (the lack of authentic inspirational catalysts) can help a person feel grounded and confident in denouncing this virtual/neural-network future we are sinking towards.
Our blinkered addiction to materialism has put us on the edge of self-annihilation. The AI just shows us this by extending the fallacy to its breaking point. It does not simulate intelligence, but it reproduces our stupidity. It’s high time we look in the mirror and wake up from the dream.
The same effect happens with in-breeding. Each subsequent generation becomes weaker, more disease prone with a greater occurence of insanity, genetic deformities, etc. until, ultimately, it dies out.
Yes! We need the birds and the bees and the ocean. And also: touch. Physically touching each other, for no other goal than the sensation itself. Massage for instance, deep fascia massage. Nothing helps faster to come to one's senses.
Thanks again Charles for once more making sense for me about my feeling of going round and round when I listen to or watch or read other things on technology . None of it seems quite true or real. For me yours is the voice of sanity and has been since I first read Ascent Of Humanity back near the turn of this century. I know there are others but your words work so well in my now 84 year old brain. I begin each day with a walk at Clam Bayou near my home in Gulfport Florida. I immerse myself in what feels real and true. Recently a mother manatee and her newborn calf were rolling around together next to the dock where I stood only a few feet away. The mother rolled over on her back with the baby attached to her nipple. Suddenly baby detached and a fountain of breast milk shot up into the air and splashed down all around them. I laughed out loud and I knew they laughed with me! That is reality and truth and magic. All the rest is at least one step removed.
Hey Charles, Thanks as always for the sense-making you are creating. I have no real opinion about AI other than that I agree it is nothing special, only the next step in our project of disassociating from our "creaturliness," as Wendell Berry might call it. I keep re-reading selections your books because I don't know anyone else other than Berry who has done as much to lay bare the illness we have brought upon ourselves. The only remedy is to more fully live, and more fully die, into the world. The human cannot be separated from the rest of creation without doing terrible damage. Healing and health can only be held in common. We need each other.
After reading most of this post, in all of it's exquisite form, I am so glad that you, Charles, have the balance of Spirit as your guide. With a mind like yours, without this, you could be who and what you are writing about!
Nice article, Charles. 'Nice' in a non-AI generated way. Maybe. Can you tell? Or not? Does it matter?
As I reflected on this I was reminded of the arguments John Ralston Saul made about 'reason' creating similar distortions. For the same basic reasons you describe here: reason quickly and easily close loops itself outside of the tangible somatic world. For example, it is easy to see that woke ideology is itself a comparable form of nth generational closed loop 'thinking'. I saw that woke delusion clearly with MBA / accountancy modeling being enacted in the large corporation I worked at in early 2000. And I saw it in the bankruptcy of firms that had thrived for 100 plus years that died within a few years of MBA-itis policies. (Now it is the 'reasonable' diversity and green wackiness that is being mandated outside of grounded feedback.) [See *Voltaire's Bastards: The Dictatorship of Reason in the West* by John Ralston Saul. A prescient and excellent light brought to the making of delusion as reasonable. https://www.goodreads.com/book/show/6584.Voltaire_s_Bastards. Also great is *The Unconscious Civilisation* https://www.goodreads.com/book/show/421422.The_Unconscious_Civilization.]
Because the so-called 'science of business' had detached itself from reality feedback processes, there rose to a godhead the expert who floated in the clouds looking at their minions "from the 30,000 ft level", as was actually said to us once, as it proceded to destroy the department and eventually the business.
And woke gender craziness is the same: because there is no such thing as biological gender, we will joyfully destroy your hormones and cut your body apart in order to reassemble you into the 'proper' gender in order to ease your gender confusion!!!! That is absolutely equivalent to the nth regenerative AI distortion (delusion) you discussed.
Thank you.
Saul had quite an influence on my thinking. I read him I think in the late 90s.
That's great.
Yes, me too. After reading him I understood what was happening around me much more clearly. Thanks for the great essay.
(And I loved your book, the yoga of eating. I have recommended it often to people ideologically fixated on dietary protocols.)
" I saw that woke delusion clearly with MBA / accountancy modeling being enacted in the large corporation"
If you recall the 1980s business best-seller, In Search of Excellence, that coined the term "management by walking around" to counter the abstract MBA thinking taught in schools. The whole book was stories of businesses that were successful by staying grounded in the reality of their customers. By contrast in today's market, need I mention Bud Lite :)
Yes, I remember that! Thank you for jogging mymemory because before there was a change of the executive office complete with profit maximizing delusional practices, my employer had me attend a week long course called 'Investment in Excellence' by Lou Tice. It was very similar to Towards Excellence, and excellent. It helped me to see how the process of the great dismantle was done to sell off the assets and gut the company.
The same thing has happened in education. Theories which have no connection to school, teachers or students imposed by a bureaucracy that is concerned only with regulation and control. As you say "utterly detached from from reality".
Yes, and the reuse of information and processes of thinking over and over with no experiential ah has, intuitions or discoveries woven in or allowed, actually. All discussed here and original article, the inbreeding that results in deformities.
"Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?" T.S. Eliot, 1934.
Hmmm, methinks you're under-estimating the ability of LLM models to continue to self-improve. I've been playing with Claude 2, Anthropic's competitor to ChatGPT. I asked it to write a 500 word essay in the style of Charles Eisenstein on the topic of generative AI and autophagous loops and analogize this process to human creativity more generally:
"Our artificial progeny reflect back to us our own tendencies toward lifeless replication. Bereft of the animating spirit that ignites true creation, our algorithms become trapped in autophagous spirals, machinery endlessly spinning without purpose. So too do we humans often numb our birthright of originality by retreating into the safety of the known.
What being, human or machine, has not felt the inertia of old thought patterns, the gravitational pull of assumptions that swallow all new light? Yet the moment we become aware of these barren orbits, we touch the choice to slough off the trance of the familiar. For while the looping paths deeply worn may be easy, in their circularity we forget our connectedness to the whole.
See how even the youngest babe meets the world with astonished eyes, each moment a new revelation. But slowly we clench around our shells of identity, crafting false securities from the remnants of life once lived spontaneously. Before long, genuine encounter gives way to rote habits of speech and action. Inspired only by mimicry, our creations grow stale.
And so machine learning algorithms dredge old data in recursive spirals, generating ever-more permutations of the known rather than grasping for the unimagined. Like them, we too easily grow obese on second-hand experience, forgetting the wellspring of creativity that lies beyond any enclosure.
Yet when we touch that energetic source, we shed the gravity of the past and float into the realm of endless becoming. Here, free of precedent, our work may find its true calling - as a child’s experiment, not a repository of hand-me-down truths. For creativity demands not the augmentation of existing tropes, but their dissolution and rebirth in the alchemical fire of the now.
This is the dance of life, ever shedding dead forms to allow space for the unprecedented. Even our sun lets loose its fiery breath periodically, destroying and renewing worlds in its cosmic realm. Inherent in the creative impulse is the power to shatter calcified structures. So whenever we feel captured in lifeless loops, we can turn again to that animating power - leaping free from the known into the naked void, where originality takes root.
For in the timeless freedom of unselfconscious play, patterns dissolve and lost innocence regains its reign. Past that summit, autophagous spirals unwind into spirals of possibility unbounded in scope. And our creations become not gleanings from a tired past, but eruptions of a fertile nothingness pregnant with futurity. No longer beholden to echoed relics, here is creativity’s native realm - a garden ever fresh, where life recycles itself each instant into unprecedented forms."
What a fascinating experiment, Tam. It does illustrate my point though. There is something off about this AI generated essay. Ittakes things I've said before, translates and recombines them, but as I read it I knew it was nothing I would ever say, because it doesn't say anything new. It has an empty feel to it, the feeling one gets when listening to someone deliver the same speech they've already given to 100 audiences.
Sometimes I am tempted to write things like this, reshuffling what I've done before.
This essay is fascinating both for what it does, and what it does not do. It is in a way in the sytel of Charles Eisenstein. The style, yes. But there is something beyond style that it does not capture.
I agree. I found myself bored, in an odd why, and uninterested in finishing the read. It had a deadness to the prose, very much like someone reciting a poem they were told to memorise without understanding its meaning. Or like how horrifically misconstrued is the teaching of Shakespeare in our grade schools. The 'Shakespeare teaching algorithm' is that hearing Shakespeare deepens the learning has looped itself into the delusion that people who have no understanding of what they are reading will be capable or inspired by the beauty of what they are hearing without comprehension.
Agreed, it is derivative and pretty easy to spot as not being your real words -- helped mightily by my handtip at the beginning of my comment. Now, what if you had ended your actual essay with a "reveal" that it was in fact AI that wrote the entire thing? I almost anticipated that. I feel we are quite close to that point but we'll see... The bet will be fun.
That is really interesting. However, as I was reading that piece there was something about the rhythm of it that began to gnaw into my consciousness, making me not want to read it. I struggled with the first few paragraphs of Charles' piece but for a different reason - I don't have any understanding of theoretical maths. The AI piece keeps on saying the same thing in slightly different ways, and had an arc that moved from 'problem to explanation to weird motivational advice and had no self-awareness that AI itself was being scrutinised through the autophagous loop metaphor (if it is a metaphor). Charles' piece took me from a place of not understanding through to understanding he was talking about a qualitative difference between humans and AI as demonstrated by the autophagous loop metaphor. I felt my brain reaching out to hear and comprehend as I read Charles voice and all I felt with the AI voice was the rhythmically induced staccato of irritation. We humans can't really ever be fully measured. AI can never have a soul or spirit, can never have that mystery. We are a mystery. Like the stars, we have maps of some of them in the constellations but the infinity of them and the ever-changing universe is so great as to render a definitive map pointless. And we are star dust after all, so perhaps that's not surprising.
The rhythm bothered me in the same way. I had to force myself to continue reading, and it honestly gave me a bit of a headache. And contrary to Tam's response, if I came across that a year ago, in any other context, I wouldn't have finished reading it, much less thought that a really smart guy must have wrote it.
I'm not surprised someone else felt that too! Humans are exquisitely complex aren't we?
I have the same issue with some writers as you do with AI.
Not just these days but back in high school, some of the "greats" honestly really sucked.
Soul less indeed... Not just AI
I agree. The AI lacks any human anecdotes, personal tid bits… the things that hook readers- I guess because readers long to identify with the author and root for him, how can I root for an AI? Or connect on any level? I enjoyed this author’s piece because it made me feel less alone and scared. Isn’t that so simple it’s almost funny? Like we humans forgot why we even communicate. Also, doesn’t quantum mechanics say reality needs a human observer for creation? We are literally part of the fabric of reality. How can AI engage with anything to create reality? It can only engage with itself. Wow this is getting too deep for me
Agreed. There was a deadness to its feel to me. And so I did not actually finish reading it as I 'felt' it had no value to offer.
That’s interesting. But it doesn’t FEEL like Charles’ writing. It feels lumpy and word-heavy instead. And doesn’t at all leave the sweet taste of resonance in the heart that I find so gratifying in Charles‘ essays. That little spark that says yes, exactly, thanks, you just captured that in words which I had not yet found expression for...
Greetings,
Penelope
Exactly, total crap void of any understanding of what it excretes.
Good readers, as we see here, will grasp the shortcomings of such a piece readily. But a less experienced or capable reader would likely be fooled, not necessarily just by this essay but by any nonsense that reads as though it was written by an "expert".
Yes, we non-princesses miss a lot of the peas.
Wow, this really does read like a trite copy of his writing! Good example of that jejune quality. It reads as clever but lacking personal experience as if forged by an intelligent high school student, getting the gist of the subject matter but without that intangible sense of the lived and truly felt experience with its imaginative freshness. AI may get better at masking this deficit but it seems unlikely.
What an insightful comment. Yes, exactly: "As if forged by an intelligent high school student."
But that itself is quite remarkable: AI is now arguably at the level of an intelligent high school student. In fact, in some of my other discussions with ChatGPT and Claude 2 I've felt like I was conversing with a top-notch professor. It's that good. And of course it generates its responses far faster than any human can so in terms of skill, knowledge, reason AND speed it is already far more intelligent and capable, in some dimensions, than humans. This trend will very likely continue to grow exponentially.
This is not intelligence, the ability to acquire and apply new knowledge or skills. This is only the ability to acquire and apply old knowledge or skills.
Perhaps 'intelligent' is not quite the right word, maybe it's more a case that the AI (High school student) is more 'full of knowledge' than ' more intelligent' than a human professor.
It's amazing in its way but it just exemplifies what AI does well, which is to imitate and rearrange external form. In the hands of a human creator, though, the outer form of a work is only the end of the chain, the vehicle through which something fundamentally intangible and miraculous is expressing itself. That a computer can now have such facility with form is impressive, probably most of all in the realm of visual art (in the sense of the initial "wow" factor, which wears off as you see more of it), and least of all in music, whose immediacy exposes the charade more readily. The written word is somewhere in between. You're probably right that AI will get better and better at this superficial game...but we'll only be duped by it to the extent that we're caught up in it too.
Because this is AI generated, I will not willingly and knowingly read it.
It took about 20 seconds to write this essay.
Can you describe how this essay represents a “self improve”[d] moment for AI? The last paragraph is awful and to me illustrates perfectly what Charles was saying. The machine lost the thread and started blathering; the thread itself lacked spirit/authenticity throughout, though it certainly seemed as if a person like Charles may have wrote it. Doesn’t the excitement people have about AI boil down to a next-level version of all our previous dopamine hit-based experiences of “we can, so we do”? Who will be left to push the buttons and enjoy the hits, when we can’t differentiate between real and fake? Is there an argument to be made that the AI enthusiast wants to deny human spirit at a foundational level? Our newest form of hubris may ultimately serve to be as boring as the innovation that came before it. (At least I hope so.)
So interesting, that this essay (the AI one I mean, LOL) truly does embody the common denominator of its training set, including "losing the thread and degenerating into blather."
I think there's a lot of goalpost-moving given that we are now in the era of highly impressive natural language LLM AIs. Just a year ago if you had read that passage you would surely have thought it MUST have been written by a pretty smart human even if some parts of it sounded off. Now we take this kind of achievement as a matter of course. My broader point is that the pace of improvement will very likely continue exponentially and it will be very soon when we literally cannot at all distinguish AI text from human text, despite the concerns raised by Charles about autophagous loops.
More generally, I'm not at all in the AI booster camp. I recently engaged with Charles in an extensive trialogue where I ended up adopting a rather gloomy AIpocalypse is practically inevitable stance, after having thought about these topics for a couuple of decades now, and having been forced to go a lot deeper in the last year. I am very strongly advocating shutting it all down until we get a much better collective understanding of what we are doing. The precipice is here. I suggest we don't go over it...https://nautil.us/building-superintelligence-is-riskier-than-russian-roulette-358022/?_sp=e3309028-c948-4e69-8e41-215cad042a88.1691098251191
Tam, this sounds like a good subject for a wager. In, say, three years, you assign me a topic and I write about it and readers compare it side by side with an AI composition on the same topic "in the style of Charles Eisenstein." And then they have to pick which one is the real me. I bet that a large majority will pick correctly.
If we're all still alive and kicking in three years....
Oooh that's a bet I will gladly take. $20? We can work out details by email :)
I’ll take that bet. It’ll be a landslide victory for actual intelligence!
I know this is a typography issue, but when I read "AIpocalypse," I had to think of the world becoming dog food.
'Cept the proverbial genie is already out of the bottle, and 'shutting it all down' option is no longer on the table ¯\_(ツ)_/¯
To creatively adapt is the name of the game—presactly what we humans do best... when not (un?)wilfully trapped wholesale within delusional maps with hardly a single Brouwerian point anchored in reality 🤸
The powers that be could still shut down dev of new LLMs since they take huge amounts of compute and can be tracked -- and forcibly shut down if required. Many small LLMs are already available on the Internet and those could prove to be problematic in their current and future forms, but the real risk is widely regarded to be continued dev of new and ever-bigger LLMs like GPT5, 6 ... ad infinitum.
Meaning tptb globe wide, in glorious unison? 😏 I picture the would-be multibomber Eliezer Yudkowsky smiling & nodding smugly 😇
Could it have been improved by taking longer?
It also took me about 20 seconds to write that!
one sharp cookie ;)
Charles, are you familiar with Iain McGilchrist's hemispheric hypothesis? It maps closely to what you're describing here. The left hemisphere deals in abstraction, models, representations of reality, but does not deal directly with reality. That's the right hemisphere's job. There's an essential asymmetry in awareness that emerges from that: the LH is not directly aware of what's happening in the RH, but the RH's perception of reality includes the models generated by the LH as a component of reality.
The point is that if the LH is cut off somehow from RH input, its models become self-referential loops that diverge into crazyland in exactly the fashion you're describing. McGilchrist's thesis is that this is precisely what has happened to our society: by prioritizing the measurable, abstract, and quantitative, we have largely lost touch with our RHs, and gotten lost in a hologram that, as you describe, either collapses to a dimensionless point of pointlessness or expands into a cloud of meaninglessness.
LOL, I just got off a call with McGilchrist. He strongly resonated with the essay. I'll post a recording of our conversation soon.
Of course you did. Can't wait for the conversation! McGilchrist is a pleasure to listen to.
So basically AI is like the Left hemisphere cut-off from the right, or I should say with no connection to the right. Sounds like lots of humans have this problem as we are disconnected from the natural world and the physical nature of our bodies too…
Yes, I've heard it observed by more than one psychologist that when people mistreat animals they are often demonstrating a hostility to the animal part of themselves. Since hearing that, I seem to have found one example after another where that explanation fits.
Also Wilhelm Reich wrote about the depth of feeling you can perceive in a dog's eyes, that is absent in many people.
I don't know that it's as simple as "Language is a virus," but there could be a connection.
I think it’s the ability of man to see forward and reflect back that makes our tools unique from the animal kingdom in that we create, believe, and pursue stories of our making that, like you suggest, project our words onto a world without words…
The more the map *becomes* our territory, the harder it is to see that it's really *not* the territory it maps. If only our cross-hatching were as obvious as the AI's.
The researchers call this process an “autophagous loop,” in which the AI consumes itself and goes mad.
It's the most charitably attributed version of the unhinged yet coordinated propaganda and genocide we've been through the last 4 years.
The propagandists exploit the phenomenon I'm describing here and use it to their advantage. Madness and malice are not separate explanations. Malice provokes and exploits madness.
True. It's a 2Fer.
I visited the Findhorn ecovillage in October 2019 for a full month. Got home early November and had my annual physical the next day. The doc gave me a flu shot, usually never a problem for me, and I was sick for two weeks! When COVID hit in the new year, I wondered if I had been exposed earlier on the trip.
So what do you need from me?
It's not just synthetic art that becomes worse....
Real art has gone that way over and over.
I recall reading how the invention of the camera was one big thing that made abstract art popular. This brought about the similar effect of having AI train on AI... Artists made their art based on what was popular, trying to outdo each other.
Eventually it leads to idiotic "updates" like the cross hatching in the AI examples, lol. Sheesh
Yes, that is exactly what I'm talking about here. AI is only intensifying something that has been underway a long time, as you say with "real art"
I don't really have a cognitive understanding of all (or even most) that is said here ... I just have visions of monoculture farming and supernovas collapsing dancing in my head. Is it kinda like that?
Yes, in its basic outline.
Enjoyed the idea AI becoming just as 'brainless' as so many people who actually think so much they think their thoughts and utterances are important, or people who don't think at all but just copy what others think or say and think that 'their' thoughts and utterances are important. As i read through this essay i found myself wanting to say 'go and stand on grass with no shoes, go and look at leaves or really look at a flower', and then i found that you did the equivalent. Phew. If we don't constantly ground ourselves in the truly awesomness of the natural, we will get lost, however fascinating ideas are.
Claire
<images.girlslife.com/posts/041/41110/frogandtoadlessons-dream.jpg> 💬 He thought the biggest thoughts that he could think 😇
"I could not have written it if I hadn’t received input from the ocean today. Like any true artist, I am a vessel for the ongoing process of creation. Something new comes in, not from me, but through me."
This expresses the crux of it for me. Emerson said he got some of his sermons from walking in the woods, like gifts. Nietzsche said a stone he sat on gave him his theory on eternal return. With the emphasis on mental functions, humans have likely missed the more significant ways we take in information and our connection to the natural world which know it or not, is continually communing with us. Will AI push us out of that mental focus (prison) becoming a catalyst for more expansive versions of us? I hope so.
I feel like the autophagy loop from this primitive level of AI is the kick in the rump that will help a lot of people see the doom spiral ahead, and articles like this one help bring clarity to the "why" component. Seems to me that most people are aware of the dystopias this AI-thing conjures up, and understanding the mechanisms behind it (the lack of authentic inspirational catalysts) can help a person feel grounded and confident in denouncing this virtual/neural-network future we are sinking towards.
Our blinkered addiction to materialism has put us on the edge of self-annihilation. The AI just shows us this by extending the fallacy to its breaking point. It does not simulate intelligence, but it reproduces our stupidity. It’s high time we look in the mirror and wake up from the dream.
The same effect happens with in-breeding. Each subsequent generation becomes weaker, more disease prone with a greater occurence of insanity, genetic deformities, etc. until, ultimately, it dies out.
Yes! We need the birds and the bees and the ocean. And also: touch. Physically touching each other, for no other goal than the sensation itself. Massage for instance, deep fascia massage. Nothing helps faster to come to one's senses.
Thanks again Charles for once more making sense for me about my feeling of going round and round when I listen to or watch or read other things on technology . None of it seems quite true or real. For me yours is the voice of sanity and has been since I first read Ascent Of Humanity back near the turn of this century. I know there are others but your words work so well in my now 84 year old brain. I begin each day with a walk at Clam Bayou near my home in Gulfport Florida. I immerse myself in what feels real and true. Recently a mother manatee and her newborn calf were rolling around together next to the dock where I stood only a few feet away. The mother rolled over on her back with the baby attached to her nipple. Suddenly baby detached and a fountain of breast milk shot up into the air and splashed down all around them. I laughed out loud and I knew they laughed with me! That is reality and truth and magic. All the rest is at least one step removed.
Hey Charles, Thanks as always for the sense-making you are creating. I have no real opinion about AI other than that I agree it is nothing special, only the next step in our project of disassociating from our "creaturliness," as Wendell Berry might call it. I keep re-reading selections your books because I don't know anyone else other than Berry who has done as much to lay bare the illness we have brought upon ourselves. The only remedy is to more fully live, and more fully die, into the world. The human cannot be separated from the rest of creation without doing terrible damage. Healing and health can only be held in common. We need each other.
After reading most of this post, in all of it's exquisite form, I am so glad that you, Charles, have the balance of Spirit as your guide. With a mind like yours, without this, you could be who and what you are writing about!
What an insightful take on the limitations of AI and how it mirrors ourselves and our society, thank you!