Reminds me of crypto/Web-3.0 hype. Lots of bluster about changing economic systems, offering people freedom and wealth, only to mostly be scams, and coming with too serious inherent drawbacks/costs to solve many of the big problems it promises to solve.
In the end leaving the world changed, but not as meaningfully or positively as promised.
I’m watching Ken Burns Documentary on the Dust Bowl, and interesting that one of the causes of the dust bowl was a wheat hype cycle in western Oklahoma with all sorts of folks theorizing that because they were able to re-form the grassland to wheat land and grow wheat it would somehow cause more rains to come (to an area that is known for droughts) and it was thought the growth would go on forever. Turns out the grasses they replaced had roots like 6 feet deep that kept the soil in place and prevented events like the dust bowl during dry spells.
Basically the hype cycle is as American as Apple Pie.
the difference is the impact of crypto was always hypothetical, chatgpt can be used, explored, and if you are creative enough, levered in ways as the ultimate tool
You've done nothing but reuse the Sam Altman/Elon Musk playbook of making wild and extremely vague statements.
Maybe say something concrete? What's a positive real world impact of LLMs where they aren't hideously expensive and error prone to the point of near uselessness? Something that isn't just the equivalent of a crypto-bro saying that their system for semi-regulated speculation (totally not a rugpull!) will end the tyranny of the banks.
they speak in generalities because the models are profoundly general, a general learning system. below someone asked me to list the capabilities, its the wrong question to ask. its like asking what a baby can do
So to translate: You want concrete examples of capabilities for something billions are being spent on? What a Silly question! (hand waving about completely speculative future abilities "when they grow up")
The woo is laughable. A cryptobro could have pulled the same nonsense out of their ass about web 3.0
But you need to verify everything unless it’s self evident. The number of times CoPilot (Sonnett 4) still hallucinates Browser APIs is astonishing. Imaging trying to learn something that can’t be checked easily, like Egyptian archeology or something.
You have to verify everything from human developers too. They hallucinate APIs when they try to write code from memory. So we have:
- documentation
- design reviews
- type systems
- code review
- unit tests
- continuous integration
- integration testing
- Q&A process
- etc.
It turns out when include all these processes, teams of error-prone human developers can produce complex working software. Mostly -- sometimes there are bugs. Kind of a lot actually. But we get things done.
Is it not the same with AI? With the right processes you can get consistent results from inconsistent tools.
Taking the example of egyptian archeology, if you're reading the work of someone who is well regarded as an expert in the field, you can trust their word a lot more than you can trust the word of an AI, even if the AI is provided the text you're reading.
This is a pretty massive difference between the two, and your narrative is part of why AI is proving to be so harmful for education in general. Delusional dreamers and greedy CEOs talking about AI being able to do "PhD level work" have potentially ruined a significant chunk of the next generation into thinking they are genuinely learning from asking AI "a few questions" and taking the answers at face value instead of struggling through the material to build true understanding.
There needs to be a reasonable chance of correctness. At least the local toddlers around here don’t randomly provide a solution to a problem that would take me hours to find but only minutes to validate.
>I'll take a potential solution I can validate over no idea whatsoever of my own any day.
If you have to validate what the LLM says, I assume you'd do that by researching primary sources and works by other experts. At that point, the LLM did nothing except charge you for a few tokens before you went down the usual research path. I could see LLMs being good for providing an outline of what you'd need to research, which is definitely helpful but not in a singularity way.
> If you have to validate what the LLM says, I assume you'd do that by researching primary sources and works by other experts.
For research, yes, and the utility there is a bit more limited. They’re still great at digesting and contextualizing dozens or hundreds of sources in a few minutes which would take me hours.
But what I mean by “easily testable” is usually writing code. If I already have good failing tests, verification is indeed very very cheap. (Essentially boils down to checking if the LLM hacked around the test cases or even deleted some.)
> At that point, the LLM did nothing […]
I’d pay actual money for a junior dev or research assistant capable of reading, summarizing, and coming up with proofs of concept at any hour of the day without getting bored at the level of current LLMs, but I’ve got the feeling $20/month wouldn’t be appealing to most candidates.
All of the information available from an LLM (and probably more) is available in books or published on the internet. They can go to a library and a read a book. They can be fairly certain books written by subject matter experts aren’t just made up.
Sure, I just gave the Browser API example as evidence that the 'hallucination' problem is not gone.
OP said it's like "talking to a professor" and you can use it to learn college level stuff. This is where I disagree. I did not double check my professors or text books usually.
I just vibecoded a photo gallery 100% from scratch - Frontend, backend, infrastructure, hosting and domain, from 0 to launch, in a couple of hours the other night.
It would have taken me a whole day, easily, to do on my own.
I have seen AI improve the quality and velocity of my wife's policy analysis dramatically.
She doesn't like using Claude, but she accepts the necessity of doing so, and it reduces 3-month projects to 2-week projects. Claude is an excellent debating partner.
Crypto? Blockchain? No-one sceptical could ever see the point of either, unless and until their transaction costs were less than that of cash. That... has not happened, to put it mildly.
that was not the sentiment people had in 2017. they were almost certain every major visa card provider and payment gateway on the planet will scrap fiat cash and adopt crypto just like how they are all thinking about every major software company adopting AI. Dont forget hindsight bias
Except that nothing of crypto/web3 ever touched my day or day life - ‘blockchain’ is now shorthand for ‘scam’ - whereas LLM-generated content is now an everyday element of both personal and professional projects, and while we maybe already be seeing diminishing returns, even the current state of advancement has already changed how digital content is searched and created forever.
The hype is real, but there’s actual practical affordable understandable day-to-day use for the tech - unlike crypto, unlike blockchain, unlike web3.
In the end leaving the world changed, but not as meaningfully or positively as promised.