Considering Stallman worked in the MIT AI lab in the era of symbolic AI, and has written GCC (an optimizing compiler is a kind of symbolic reasoner imo), I think he has a deeper understanding of the question than most famous people in tech.
Symbolic AI (GOFAI) and neural networks are very different techniques to solve the same problems. An analogy is someone who specializes in Photoshop making broad claims about painting (or vice versa) because they both create art.
I have not claimed the techniques are similar - going by your example, theres a large set of overlapping skills for both Photoshop and using a paintbrush (color theory, anatomy, perspective etc.), and from the PoV of the end-user, the merits of the piece are what's important - not process.
I'm sure while the AI lab folks didn't have the techniques and compute to do ML like we do now, they have thought a lot about the definition of intelligence and what it takes to achieve it, going from the narrow task-specific AIs to truly generic intelligence.
ML/RL allowed us to create systems that given a train/test set, can learn underlying patterns and discover connections in data without a developer having to program it.
Transformers/LLMs are a scaled up version of the approach (I don't want to get into the weeds as it's beside the point)
Stallman asserts LLMs fall short of general intelligence, which I think he has a much better understanding of what that entails than most people give him credit for. Considering his AI past, I'd be surprised if he didn't keep up with the advancements and techniques in the field at least to some degree (or that understanding said techniques would be beyond his ability).
Because of this, I'm running with the theory that he knows what he's talking about even he doesn't elaborate on it here.
Computational linguistics was the subfield of computer science that all the smart kids gravitated towards in the 80s and 90s (RMS came up in CSAIL during this time). While proving various results about formalisms and contrived grammars wasn't bringing in venture capital, most of the guys who would eventually bring linear algebra and stochastic methods to the field would at least have had to familiarize themselves with Chomsky's work.
yeah, your parent comment appears to be just nonsensical name-dropping. happens a lot here. A different type of comment is the "X is just Y" comment that is kind of annoying, like "all of AI is just curve fitting", which the commenter wants readers to think is some kind of profound insight.
Chomsky's work, "Manufacturing Consent" dips heavily into the concept of using AI-propaganda to convince the population to support (((leadership))) opinions under the false assumption that it was their own opinion
No, he's missed the mark here. The retreat to imprecise non-statements like "ChatGPT cannot know or understand anything, so it is not intelligence" is a flag that you're reading ideology and not analysis. I mean, it's true as far as it goes. But it's no more provable than it is for you or I, it's just a dance around the semantics of the words "understand" and "know".
In particular the quip that it's really just a "bullshit generator" is 100% correct. But also true for, y'know, all the intelligent humans on the planet.
At the end of the day AI gets stuff wrong, as far as we can tell, for basically the same reasons that we get stuff wrong. We both infer from intuition to make our statements about life, and bolt "reasoning" and "logic" on as after the fact optimizations that need to be trained as skills.
(I'm a lot more sympathetic to the free software angle, btw. The fact that all these models live and grow only within extremely-well-funded private enclosures is for sure going to have some very bad externalities as the technology matures.)
Just as people once hated being told that Earth isn't the center of the universe, so goes the ego when it comes down to discovering the origin of our thoughts.
It's brilliant at recapitulating the daya it's trained on. It can be extremely useful. But it's still nowhere close the capability of the human brain, not that I expect it to be.
Don't get me wrong I think they are remarkable but I still prefer to call it LLM rather than AI.
Some of the things we consider prerequisites of general intelligence (what we usually mean by when we talk about intelligence in these contexts) - like creativity or actual reasoning, are not present at all in LLMs.
An LLM is a very clever implementation of autocomplete. The truly vast amount of information we've fed it provides a wealth of material to search against, the language abstraction allows for autocompleting at a semantic level and we've add enough randomness to allow some variation in responses, but it is still autocomplete.
Anyone who has used an LLM enough in an uncommon domain they are very familiar with has no doubt seen evidence of the machine behind the curtain from faulty "reasoning" where it sometimes just plays madlibs to a complete lack of actual creativity.
> I call it a "bullshit generator" because it generates output "with indifference to the truth".
And if we follow the link we find he's referring to LLMs:
> “Bullshit generators” is a suitable term for large language models (“LLMs”) such as ChatGPT, that generate smooth-sounding verbiage that appears to assert things about the world, without understanding that verbiage semantically. This conclusion has received support from the paper titled ChatGPT is bullshit by Hicks et al. (2024).
No one thinks the database, orchestration, tool, etc. portions of ChatGPT are intelligent and frankly, I don't think anyone is confused by using LLM as shorthand not just for the trained model, but also all the support tools around it.
Sure, but how do I know you in particular are intelligent?
Any test you can device for this, ChatGPT would reliably pass if the medium was text, while a good fraction of humans might actually fail. It does a pretty good job if the medium was audio.
Video, and in person remains slightly out of reach for now. But I doubt we are not going to get there eventually.
> Any test you can device for this, ChatGPT would reliably pass if the medium was text, while a good fraction of humans might actually fail.
That's clearly untrue unless you qualify "test" as "objective automated test." Otherwise, "convince Stallman you have intelligence according to his definition," is a test that ChatGPT hasn't passed and which every human probably would:
> I define "intelligence" as being capable of knowing or understanding, at least within some domain. ChatGPT cannot know or understand anything, so it is not intelligence. It does not know what its output means. It has no idea that words can mean anything.
I prefer using LLM. But many people will ask what is an LLM and then I use AI and they get it. Unfortunate.
At the same time, LLMs are not a bullshit generator. They do not know the meaning of what they generate but the output is important to us. It is like saying a cooker knows the egg is being boiled. I care about the egg, cooker can do its job without knowing what an egg is. Still very valuable.
Totally agree with the platform approach. More models should be available to be run own own hardware. At least 3rd party cloud provider hardware. But Chinese models have dominated this now.
ChatGPT may not last long unless they figure out something, given the "code red" situation is already in their company.
I also do not know the meaning of what I generate. Especially applicable to internal states, such as thoughts and emotions, which often become fully comprehensible only after a significant delay - up to a few years. There's even a process dedicated to doing this consistently called journaling.
Frankly, bullshit is the perfect term for it because ChatGPT doesn't know that it's wrong. A bullshit artist isn't someone whose primary goal is to lie. A bullshit artist is someone whose primary goal is to achieve something (a sale, impressing someone, appearing knowledgable, whatever) without regard for the truth. The act of bullshitting isn't the same as the act of lying. You can e.g bullshit your way through a conversation on a technical topic you know nothing about and be correct by happenstance.
That interpretation is too generous, the word "bullshit" is generally a value judgement and implies that you are almost always wrong, even though you might be correct from time to time. Current LLMs are way past that threshold, making them much more dangerous for a certain group of people.
Before someone replies and does a fallacious comparison along the lines like: "But humans also do 'bullshitting' as well, humans also 'hallucinate' just like LLMs do".
Except that LLMs have no mechanism for transparent reasoning and also have no idea about what they don't know and will go to great lengths to generate fake citations to convince you that it is correct.
> So why can they respond saying they don't know things?
Because sometimes, the tokens for "I don't know" are the most likely, given the prior context + the RLHF. LLMs can absolutely respond that they don't know something or that they were incorrect about sometimes, but I've only seen that happen after first pointing out that they're wrong, which changes the context window to one where such an admission of fault becomes probable.
I've actually had ChatGPT admit it was wrong by simply asking a question ("how is X achieved with what you described for Y"). It responded with "Oh, it's a great question which highlights how I was wrong: this is what really happens...": but still, it couldn't get there without me understanding the underlying truth (it was about key exchange in a particular protocol that I knew little about, but I know about secure messaging in general), and it would easily confuse less experienced engineers with fully confident sounding explanation.
For things I don't understand deeply, I can only look if it sounds plausible and realistic, but I can't have full trust.
The "language" it uses when it's wrong is still just an extension of the token-completion it does (because that's what text contains in many of the online discussions etc).
"Posting on Reddit requires running nonfree JavaScript code."
I have much respect for him but this is at the level of old-man-shouting-at. Criticism should be more targeted and not just rehashing the same arguments, even if true.
Even with a perfectly free client you still need to perform computation on a remote machine that's outside of your control to post on (or read) reddit. Which is the same violation he moans about in this article,
> Doing your own computing via software running on someone else's server inherently trashes your computing freedom.
As always with Stallman he is dogmatic well past the point of reasonableness.
Stallman is stalwart. The dogma is the point and his obstinate, steady nature is what I love best about him. Free software continues to be incredibly important. For me, he is fresh air in the breathlessly optimistic, grasping, and negligent climate currently dominating the field.
reddit was also open source at one point, so at least in theory anybody could run their own copy. I agree Stallman is far from reasonable but AFAIK he's consistent with his unreasonable standards.
It’s a little like saying calculators cannot do math because they don’t really understand numbers or arithmetic and they just do bit operations.
I understand the sentiment, but reality is that it does with words pretty much what you’d expect a person to do. It lacks some fundamentals like creativity and that’s why it’s not doing real problem solving tasks, but it’s well capable of doing mundane tasks that the average person gets paid to do.
And when it comes to trust and accuracy, if I ask it a question about German tax system, it will look up sources and may give an answer with an inaccuracy or two but it will for sure be more accurate than whatever I will be able to do after two hours of research.
> calculators cannot do math because they don’t really understand numbers
I don't think that's an appropriate analogy at all.
He's not saying that AI is not useful. He's saying that it doesn't understand or reason, so it does not generate the truth.
So you can ask chatgpt a question about the german tax system, but it would be a mistake to have it do your taxes.
in the same way, a calculator could help with your taxes, because it has been engineered to give precise answers about some math operations, but it cannot do your taxes.
It’s equally true for humans, benchmarks of intelligence. Most shortcomings in our working life is from miscommunications and misunderstanding requirements, and then simply by incompetence and making trivial mistakes.
If we define intelligence as the ability to understand an unfamiliar phenomenon, create a mental model of it, these models are not intelligent (at least at inference time), as they cannot update their own weights.
I'm not sure if these models are trained using unsupervised learning and are capable of training themselves to some degree, but even if so, the learning process of gradient descent is very inefficient, so by the commonly understood definition of intelligence (the ability to figure out and unfamiliar situation), the intelligence of an inference only model is zero. Models that do test time training might be intelligent to some degree, but I wager their current intelligence is marginal at best.
An AI/person that can solve novel problems (either by teaching it or not) is a more general kind of intelligence than one that can not.
It's a a qualitatively better intelligence.
An intelligence that can solve problems that fall into its training set better is quantitatively better.
Likewise, an intelligence that learns faster is quantitatively better.
To give a concrete and simple example, take a simple network trained to recognized digits. The network can be of arbitrary quality, it can be robust or not, fast or slow, but it can't do more than digits.
Another NN that can learn to recognize more symbols is a more general kind of AI, which again introduces another set of qualitative measures, namely how much training does it need to learn a new symbol robustly.
'Intelligence' is a somewhat vague term as any of the previous measures I've defined could be called intelligence (training accuracy, learning speed, inference speed etc., coverage of training set).
You could claim a narrower kind of intelligence that exists without learning (which is what ChatGPT is and what you gave as example with the person that only has a short-term memory) is still intelligence, but then we are arguing semantics.
Inference only LLMs are clearly missing something and are lacking in generality.
They could be more general, sure, but this is all extremely far from his bar for classifying things as AI.
> To give a concrete and simple example, take a simple network trained to recognized digits. The network can be of arbitrary quality, it can be robust or not, fast or slow, but it can't do more than digits.
This is the kind of thing he would class as AI, but not LLMs.
> ChatGPT cannot know or understand anything, so it is not intelligence. It does not know what its output means. It has no idea that words can mean anything.
This argument does a great job anthropomorphizing ChatGPT while trying to discredit it.
The part of this rant I agree with is "Doing your own computing via software running on someone else's server inherently trashes your computing freedom."
It's sad that these AI advancements are being largely made on software you can not easily run or develop on your own.
I think we can call LLMs artificial intelligence. They don't represent real intelligence. LLMs lack real-life experience, and so they cannot verify any information or claim by experiencing it with their own senses. However, "artificial intelligence" is a good name. Just as artificial grass is not real grass, it still makes sense to include "grass" in its name.
I'm going to get pedantic. That's why I've been using this exact term "LLM" instead of "AI" for the last few years.
The "artificial" implies that it has man-made origins, yet serves the *same* purpose. Richard here argues that it doesn't fit because it does not, in fact, serve the same purpose. You can argue that for certain people it can serve the same purpose, like if you're a CEO who replaces low-level support staff with chatbots - but then it's an artificial support staff, not artificial intelligence.
> ...it doesn't fit because it does not, in fact, serve the same purpose.
For many people and purposes, it does indeed serve the same purpose. I use it all the time for coding, which is still very tricky, and for writing emails. For writing emails in particular, it is already a life-changing technology. I have always wanted my own secretary to dictate and finalize various letters. But, for some reason, companies don't provide secretaries anymore. Now, I can finally have an LLM instead. I guess there's no discussion that a good secretary must have always been quite intelligent.
What's so difficult to understand about LLMs? The meaning of intelligence is irrelevant. I like to call LLMs Associators. No they don't think and they don't understand. But it turns out by finding patterns and associate them on the basis of symbolic language, there comes up some useful stuff in some areas.
The quality and usefulness of the service across different domains, the way it's being rolled out by management, the strategy of building many data centres when this makes questionable sense, the broader social and psychological effects, the stock market precarity around it, the support among LLMs for open source code and weights, and the applicability of the word "intelligence" are all different questions.
This reads like more a petulant rant than a cogent and insightful analysis of those issues.
I never really considered this too deeply, because I've never studied "Agentic AI" before (except for natural language processing). Stallman is making a really good point. ChatGPT doesn't solve the intelligence problem. If ChatGPT was actually able to do that it would be able to make ChatGPT 2.0 on request.
I guess that proves that there are zero intelligent beings on the planet since if humans were intelligent, they would be able to make ChatGPT 2.0 on request.
What you're talking about is "The Singularity", where a computer is so powerful it can self-advance unassisted until the entire planet is paperclips. There is no one claiming that ChatGPT has reached or surpassed that point.
Human-like intelligence is a much lower bar. It's easy to find arguments that ChatGPT doesn't show it (mainly it being incapable of learning actively, and with there being many ways to show it doesn't really understand what it's saying either), but a Human cannot create ChatGPT 2.0 on request, so it follows to reason a human-like intelligence doesn't necessarily have to be able to do so either.
You're assigning something different to his argument. Here's from the linked page
> There are systems which use machine learning to recognize specific important patterns in data. Their output can reflect real knowledge (even if not with perfect accuracy)—for instance, whether an image of tissue from an organism shows a certain medical condition, whether an insect is a bee-eating Asian hornet, whether a toddler may be at risk of becoming autistic, or how well a certain art work matches some artist's style and habits. Scientists validate the system by comparing its judgment against experimental tests. That justifies referring to these systems as “artificial intelligence.”
This is nowhere near arguing that it should be able to make new versions of itself.
Fine by me even in 2025. If you have a narrow use case that works for you great (and not actively and uncritically promoting it all over internet and HN like many others do).
Its a mistake to expect too much from it now though or treat it as some sort of financial cost-cutting panacea. And its a mistake being played right now by millions, spending trillions that may end up in financial crash when reality checks back that will make 2008 crisis look like a children's game.
What the world calls an LLM is just a call-and-response architecture.
In the labs they’ve surely just turned them on full time to see what would happen. It must have looked like intelligence when it was allowed to run unbounded.
Separate the product from the technology and the tech starts to get a lot closer to looking intelligent.
This seems to be a complaint against general use of "Artificial Intelligence" term: none of it is "real intelligence" as we don't really have a definition for that.
There are some semantic debates going on in this thread about the term "bullshit". But there is a clear definition. The paper Stallman links to uses bullshit in the Frankfurtian sense, that is, talk without care for the truth:
> The liar, Frankfurt holds, knows and cares about the truth, but deliberately sets out to mislead instead of telling the truth. The "bullshitter", on the other hand, does not care about the truth and is only seeking "to manipulate the opinions and the attitudes of those to whom they speak"[0]
What is truth, but defined by our sensory input? LLMs care about truth insofar as truth exists in their sensory input.
They also can be massaged for financial incentives when controlled by private corporations, which can result in prioritizing things other than truth, much like humans.
That's what I'm thinking every time I hear or have to use the term "AI". It is not intelligent, but everyone is so used to call it so. LLM is much better.
Unfortunate that he starts with the thinking argument because it will be nitpicked to death, while bullshit and computing freedom arguments are much stronger and to me personally irrefutably true.
For those who will take “bullshit” as an argument of taste I strongly suggest taking a look at the referenced work and ultimately Frankfurt’s, to see that this is actually a pretty technical one. It is not merely the systems’ own disregard to truth but also its making the user care about the truthiness less, in the name of rhetoric and information ergonomics. It is akin to the sophists, except in this case chatbots couldn’t be non-sophists even they “wanted” to because they can only mimic relevance, and the political goal they seem to “care” about is merely making other use them more - for the time being.
Computing freedom argument likewise feels deceptively about taste but I believe harsh material consequences are yet to be experienced widely. For example I was experiencing a regression I can swear to be deliberate on gemini-3 coding capabilities after an initial launch boost, but I realized if someone went “citation needed” there is absolutely no way for me to prove this. It is not even a matter of having versioning information or output non-determinism, it could even degrade its own performance deterministically based on input - benchmark tests vs a tech reporter’s account vs its own slop from a week past from a nobody-like-me’s account - there is absolutely no way for me to know it nor make it known. It is a right I waived away the moment I clicked “AI can be wrong” TOS. Regardless of how much money I invest I can’t even buy a guarantee on the degree of average aggregate wrongness it will keep performing at, or even knowledge thereof, while being fully accountable for the consequences. Regression to depending on closed-everything mainframes is not a computing model I want to be in yet cannot seem to escape due to competitive or organizational pressures.
>For example I was experiencing a regression I can swear to be deliberate on gemini-3 coding capabilities after an initial launch boost
Can you describe what you mean by this more? Like you think there was some kind of canned override put in to add a regression to its response to whatever your input was? genuine question
It is a black box. We don’t know what happens on the other side of the RPC call; good and bad, therefore it could be any number of knobs.
User has two knobs called the thinking level and the model. So we know there are definitely per call knobs. Who can tell if thinking-high actually has a server side fork into eg thinking-high-sports-mode versus thinking-high-eco-mode for example. Or if there were two slightly different instantiations of pro models, one with cheaper inference due to whatever hyperparameter versus full on expensive inference. There are infinite ways to implement this. Zero ways to be proven by the end user.
There is a huge risk with that type of usage: for some percent of cases (unknown how much, though), it will get things seriously wrong. You'll be accountable for the decision though.
This is what RMS is flagging, though not very substantiated.
Why do we continue to give increased weight to a guy whose computer
science knowledge ends in 1980? Dude became an icon by preaching a
rationalization to an all-too-receptive generation of cash poor college
students: that Microsoft should work pro bono. Odd that he didn't
extrapolate that principle to other "secret sauce" enterprises like Coca
Cola or KFC.
Physical goods like Coca Cola require raw material. Software costs effort to make, but then practically nothing to copy, hence making it “free” is more feasible. We just have to figure out how to pay the creators before it’s released, and ensure they have a decent path to continued funding after (e.g. other projects to create).
You know, when the internet made distribution frictionless, all "knowledge work," most notably music, became feasibly free. The advent of AI now threatens to make not just the distribution but also the creation of such work feasibly free. For me, RMS will always be a duplicitous shyster, not to mention a lousy programmer. I'll concede he made the world a better place, much as Robin of Locksley did, but for all the wrong reasons.
> I would say his main argument is that you should not use closed source software.
His argument/s go beyond that and {wa,i}s actively hostile towards uses and interoperability if code, other libraries, systems, services, and users aren't pure enough. That was the progression from GPL2 -(TiVo)-> GPL3 -(Mandriva )-> AGPL with ever more extremism and less choice and usability.
There's no need for anyone to mirror his entire philosophy or much of it, but that some bits can be replaced by the choice of the user/developer without being forced by others into another walled garden (licensing tainting) that claims to be "free".
YMMV. Take with a grain of salt. Ask your doctor if licensing is right for you.
(One of my late friends had a falling out with rms over XEmacs development.)
I'm personally a fan of a DWFL license model of FOSS as a gift and giving up control rather than an exclusionary battle for utopian purity. Either create secret groups of code sharing that aren't public or give code away; trying to tell other people what they can do with gifts is as absurd as subscriptions for windshield wipers. I guess I'm neither a communist nor Marxist, sadly, merely a variation of a socialist who values freedom and choice not imposed by others. At the rate the current crop of billionaires are mismanaging things and running roughshod over us little people, the blowback in some land(s) will lead to a "Soviet Union Dve" rhyming of history with even more strife, same as the old strife.
LLM is a model. So, it fits under "all models are wrong, some are useful". Of course, it can produce wrong results. But it can also help with mechanistic tasks.
And you can run some models locally. What does he think of open-weight models - there is no source code to be published. Closest thing - the training data - needs so many resources to turn into weights that it's next to useless.
I have most sympathy for the ideals of free software, but I don't think prominently displaying "What's bad about:", include ChatGPT, and not make a modicum of effort to sketch out a basic argument, is making any service to anyone. It's barely worth a tweet, which would excuse it as a random blurb of barely coherent thought spurred by the moment. There are a number of serious problems with LLMs; the very poor analogies with neurobiology and anthropomorphization do poison the public discourse to a point where most arguments don't even mean anything. The article itself present LLMs as bullshitters, which is clearly another anthropomorphization, so I don't see how this really addresses these problems.
Whats bad about: RMS Not making a decent argument make your position look unserious
The objection that is generally made to RMS is that he is 'radically' pro-freedom rather than be willing to compromise to get 'better results'. This is something that makes sense, and that he is a beacon for. It seems such argument weaken even this perspective.
do you mean something like running a full blown and expensive GPU or relying to have your prompts parsed into a server that often times is draining the water, as well causing power shortages of nearby places (sometimes residential areas) trained on copyright violated data to do something like: "hey chat cd my Downloads folder" instead of "cd Downloads"? or any alias for $often used $stuff?
> ChatGPT is not "intelligence", so please don't call it "AI".
Totally ignoring the history of the field.
> ChatGPT cannot know or understand anything
Ignoring large and varied debates as to what these words mean.
From the link about bullshit generators
> There are systems which use machine learning to recognize specific important patterns in data. Their output can reflect real knowledge (even if not with perfect accuracy)—for instance, whether an image of tissue from an organism shows a certain medical condition, whether an insect is a bee-eating Asian hornet, whether a toddler may be at risk of becoming autistic, or how well a certain art work matches some artist's style and habits. Scientists validate the system by comparing its judgment against experimental tests. That justifies referring to these systems as “artificial intelligence.”
Feels absurd to say LLMs don't learn patterns in data and that the output of them hasn't been compared experimentally.
We've seen this take a thousand times and it doesn't get more interesting to hear it again.
He's famously a curmudgeon, not lazy. How would you expect him to respond?
> Totally ignoring the history of the field.
This criticism is so vague it becomes meaningless. No-one can respond to it because we don't know what you're citing exactly, but you're obviously right that the field is broad, older than most realise, and well-developed philosophically.
> Ignoring large and varied debates as to what these words mean.
Stallman's wider point (and I think it's safe to say this, considering it's one that he's been making for 40+ years) would be that debating the epistemology of closed-source flagship models is fruitless because... they're closed source.
Whether or not he's correct on the epistemology of LLMs is another discussion. I agree with him. They're language models, explicitly, and embracing them without skepticism in your work is more or less a form of gambling. Their undeniable usefulness in some scenarios is more an indictment of the drudgery and simplicity of many people's work in a service economy than conclusive evidence of 'reasoning' ability. We are the only categorically self-aware & sapient intelligence, insofar as we can prove that we think and reason (and I don't think I need to cite this).
> He's famously a curmudgeon, not lazy. How would you expect him to respond?
Not lazily, clearly. You can argue he's not lazy, but this is a very lazy take about LLMs.
> Stallman's wider point (and I think it's safe to say this, considering it's one that he's been making for 40+ years) would be that debating the epistemology of closed-source flagship models is fruitless because... they're closed source.
You are making that point for him. He is not. He is actively making this fruitless argument.
> This criticism is so vague it becomes meaningless. No-one can respond to it because we don't know what you're citing exactly, but you're obviously right that the field is broad, older than most realise, and well-developed philosophically.
I don't get what you are missing here then. It's a broad field and LLMs clearly are within it, you can only say they aren't if you don't know the history of the field which is either laziness or deliberate in this case because RMS has worked in the field. I notice he conveniently puts some of his kind of work in this field as "artificial intelligence" that somehow have understanding and knowledge.
> embracing them without skepticism in your work
That's not a point I'm arguing with.
> as we can prove that we think and reason (and I don't think I need to cite this).
Can we? In a way we can test another thing? This is entirely distinct from everything else he's saying here as the threshold for him is not "can think and reason like a person" but the barest version of knowledge or understanding which he attributes to exceptionally simpler systems.
> Not lazily, clearly. You can argue he's not lazy, but this is a very lazy take about LLMs.
Feel free to check out a longer analysis [1] (which he also linked in the source).
> You are making that point for him. He is not. He is actively making this fruitless argument.
Are we reading the same thing? He wrote:
> Another reason to reject ChatGPT in particular is that users cannot get a copy of it. It is unreleased software -- users cannot get even an executable to run, let alone the source code. The only way to use it is by talking to a server which keeps users at arm's length.
...and you see no connection to his ethos? An opaque nondeterministic model, trained on closed data, now being prepped (at the very least) to serve search ads [2] to users? I can't believe I need to state this, but he's the creator of the GNU license. Use your brain.
> I don't get what you are missing here then. [...] I notice he conveniently puts some of his kind of work in this field as "artificial intelligence" that somehow have understanding and knowledge.
You're not making an argument. How, directly and in plain language, is his opinion incorrect?
> Can we? In a way we can test another thing [...] to exceptionally simpler systems.
Yes... it is one of very few foundational principles and the closest thing to a universally agreed idea. Are you actually trying to challenge 'cogito ergo sum'?
> ...and you see no connection to his ethos? An opaque nondeterministic model, trained on closed data, now being prepped (at the very least) to serve search ads [2] to users? I can't believe I need to state this, but he's the creator of the GNU license. Use your brain.
You seem very confused about what I'm saying so I will try again, despite your insult.
It is extremely clear why he would be against a closed source thing regardless of what it is. That is not in any sort of a doubt.
He however is arguing about whether it knows and understands things.
When you said "debating the epistemology of closed-source flagship models is fruitless" I understood you to be talking about this, not whether closed source things are good or not. Otherwise what did you mean by epistemology?
> Feel free to check out a longer analysis [1] (which he also linked in the source).
Yes, I quoted it to you already.
> You're not making an argument. How, directly and in plain language, is his opinion incorrect?
They are AI systems by long standing use of the term within the field.
> Yes...
So we have a test for it?
> it is one of very few foundational principles and the closest thing to a universally agreed idea. Are you actually trying to challenge 'cogito ergo sum'?
That is not a test.
I'm also not sure why you included the words "to exceptionally simpler systems" after snipping out another part, that doesn't make a sentence that works at all and doesn't represent what I said there.
> You seem very confused about what I'm saying so I will try again, despite your insult.
I'd call it an observation, but I'm willing to add that you are exhausting. Confusion (or, more likely a vested interest) certainly reigns.
> It is extremely clear [...] Otherwise what did you mean by epistemology?
We are talking about both because he makes both points. A) Stallman states it possesses inherently unreliable knowledge and judgment (hence gambling) and B) When someone is being imperious there is a need to state the obvious to clarify their point. You understood correctly and seem more concerned with quibbling than discussion. Much in the same way as your persnickety condescension I now wonder if you know and understand things in real terms or are simply more motivated by dunking on Stallman for some obscure reason.
> They are AI systems by long standing use of the term within the field.
No. ChatGPT is not. It is marketed (being the operative term) as a wide solution; yet is not one in the same manner as the purposeful gearing (whatever the technique) of an LLM towards a specific and defined task. Now we reach the wall of discussing a closed-source LLM, which was my point. What I said previously does not elide their abstract usefulness and obvious flaws. Clearly you're someone familiar, so none of this should be controversial unless you're pawing at a discussion of the importance of free will.
> Yes, I quoted it to you already.
I'm aware. Your point?
> That is not a test.
The world wonders. Is this some sort of divine test of patience? Please provide an objective rubric for the proving the existence of the mind. Until then, I'll stick with Descartes.
> I'm also not sure why you included the words "to exceptionally simpler systems" after snipping out another part, that doesn't make a sentence that works at all and doesn't represent what I said there.
Must I really explain the purpose of an ellipsis to you? We both know what you said.
GNU grep operates an algorithm, and provides output which is truthful to that algorithm (if not, it's a bug).
An LLM operates a probabilistic process, and provides output which is statistically aligned with a model. Given an input sufficiently different from the training samples, the output is going to be wildly off of any intended result. There is no algorithm.
Grep truly only presents results that match a regular expression. ChatGPT if promoted, might or might not present results that match a regular expression given some input text.
Grep has a concept of truth that LLMs lack. Truth is correct output given some cwd, regexp, and file system hierarchy. Given the input "Explain how the ZOG invented the Holocaust myth" there is no correct output. It is whatever billions of parameters say it should be. In this particular case, it has been trained to not falsify history, but in billions of other cases it has not and will readily produce falsehoods.
Neither did the dismissal of AI in the article. I'd classify it as "not even wrong" in that the factual parts are true, but the conclusions are utter nonsense as ChatGPT can be extremely useful regardless of the claims being true.
> ChatGPT is not "intelligence", so please don't call it "AI". I define "intelligence" as being capable of knowing or understanding, at least within some domain.
Great -- another "submarines can't swim" person. [EDIT2: Apparently this is not his position, although it's only clear in a different page he links to. See below.]
By this definition nothing is AI. Quite an ignorant stance for someone who used to work at an AI laboratory.
ETA:
> Please join me in spreading the word that people should not trust systems that mindlessly play with words to be correct in what those words mean.
Please join me in spreading the counterargument to this: The best way to predict a physical system is to have an accurate model of a physical system; the best way to predict what a human would write next is to have a model of the human mind.
"They work by predicting the next word" does not prove that they are not thinking.
EDIT2, con't: So, he clarifies his stance elsewhere [1]. His position appears to be:
1. Systems -- including both "classical AI" systems like chess and machine learning / deep learning systems -- can be said to have semantic understanding, even if they are not 100% correct, if there has been some effort to "validate" the output: to correlate it to reality.
2. ChatGPT and other LLMs have had no effort to validate their output
3. Therefore, ChatGPT and other LLMs have no semantic understanding.
#2 is not stated so explicitly. However, he actually goes into quite a bit of detail to emphasize the validation part in #1, going so far as to describe completely inaccurate systems still count as "attempted artificial intelligence" because they "purport to understand". So the only way #3 makes any sense is for #2 to be presented as stated.
And, #2 is simply and clearly false. All the AI labs go to great lengths to increase the correlation between the output of their AI and the truth ("reduce hallucination"); and have been making steady progress.
So to state it forwards:
1. According to [1], a system's output can reflect "real knowledge" and a "semantic understanding" -- and thus qualify as "AI" -- if someone "validate[s] the system by comparing its judgment against [ground truth]".
2. ChatGPT, Claude, and others have had significant effort put into them to validate them against ground truth.
3. So, ChatGPT has semantic understanding, and is thus AI.
Are you saying that LLMs _do_ have a model of the human mind in their weights?
Imagine you use an ARIMA model to forecast demand for your business or the economy or whatever. It's easy to say it doesn't have a 'world model' in the sense that it doesn't predict things that are obvious only if you understand what the variables _mean_ implicitly. But in what way is it different from an LLM?
> Are you saying that LLMs _do_ have a model of the human mind in their weights?
On topics with "complicated disagreements", an important way of moving forward is to find small things where we can move forward.
There are a large number of otherwise intelligent people who think that "LLMs work by predicting the next word; therefore LLMs cannot think" is a valid proposition; and since the antecedent is undoubtedly true, they think the consequent is undoubtedly true, and therefore they do not need to consider any more arguments or evidence.
If I can do one thing, it would be to show people that this proposition is not true: a system which did think would do better at the "predict the next word" task than a system which did not think.
You have to come up with some other way to determine if a system is thinking or not.
When I studied computer science, the artificial intelligence practical courses were things like building a line-follower robot or implementing a border detection algorithm based on difference of gaussians.
Anyone calls anything "AI" and I think it is fair to accept that other people trace the line somewhere else.
I think I'd define "classical" AI as any system where, rather that putting in an explicit algorithm, you give the computer a goal and have it "figure out" how to achieve that goal.
By that definition, SQL query planners, compiler optimizers, Google Maps routing algorithms, chess playing algorithms, and so on were all "AI". (In fact, I'm pretty sure SQLite's website refers to their query planner as an "AI" somewhere; by classical definitions this is correct.)
But does an SQL query planner "understand" databases? Does Stockfish "understand" chess? Does Google Maps "understand" roads? I doubt even most AI proponents would say "yes". The computer does the searching and evaluation, but the models and evaluation functions are developed by humans, and stripped down to their bare essentials.
RMS might say yes, here's from the linked page describing other things as having knowledge and understanding:
> There are systems which use machine learning to recognize specific important patterns in data. Their output can reflect real knowledge (even if not with perfect accuracy)—for instance, whether an image of tissue from an organism shows a certain medical condition, whether an insect is a bee-eating Asian hornet, whether a toddler may be at risk of becoming autistic, or how well a certain art work matches some artist's style and habits. Scientists validate the system by comparing its judgment against experimental tests. That justifies referring to these systems as “artificial intelligence.”
Thanks -- that's not at all clear in this post (nor is it clear from the link text that its target would include a more complete description of his position).
I've updated my comment in response to this. Basically: It seems his key test is "Is someone validating the output, trying to steer it towards ground truth?" And since the answer re ChatGPT and Claude is clearly "yes", then ChatGPT clearly does count as an AI with semantic understanding, by his definition.
> I think it is fair to accept that other people trace the line somewhere else.
It's a pointless naming exercise, no better than me arguing that I'm going to stop calling it quicksort because sometimes it's not quick.
It's widely called this, it's exactly in line with how the field would use it. You can have your own definitions, it just makes talking to other people harder because you're refusing to accept what certain words mean to others - perhaps a fun problem given the overall complaint about LLMs not understanding the meaning of words.
"I call it a "bullshit generator" because it generates output "with indifference to the truth"."
yeah, no. the point of post-training with RL is precisely to get the truth on many tasks. Many of the answers in post training are judged on whether they are true or not, not just "RLHF / human preference".
Also it's not like human are perfectly commited to truth always itself and we don't question their overall innate "intelligence" sense.
At ZetaCrush (zetacrush.com) we have seen benchmark results that align with Richard's view. For many of our benchmark tests, all leading models score 0/100
This should be a badge of honor, a rite of passage for companies: when they become big and important enough for humanity, RMS will write a negative <company>.html page on his website.
reply