This sci-fi thing goes as far back as the 1983 movie WarGames, where they wanted to pull the plug on a rogue computer, but there was a reason you couldn’t do that:
McKittrick: General, the machine has locked us out. It's sending random numbers to the silos.
Pat Healy: Codes. To launch the missiles.
General Beringer: Just unplug the goddamn thing! Jesus Christ!
McKittrick: That won't work, General. It would interpret a shutdown as the destruction of NORAD. The computers in the silos would carry out their last instructions. They'd launch.
Further than that, even - this trope appears in Colossus: The Forbin Project, released in 1970, where the rogue computer is buried underground with its own nuclear reactor, so it can't be powered off.
In real life it won’t be that the computer prevents you from turning it off. It’ll be that the computer is guarded by cultists who think its god, and unstoppable market forces that require it to keep running.
When AI ends up running everything essential to survival and society, it’ll be preposterous to even suggest pulling the plug just because it does something bad.
Can you imagine the chaos of completely turning off GPS or Gmail today? Now imagine pulling the plug on something in the near future that controls all electric power distribution, banking communications, and Internet routing.
This is the case with capitalism today. I don't like where he took the philosophy, but Nick Land did have an insight that all the worst things we believe about AI (e.g. paperclip optimizing etc) are capitalism in a nutshell.
Just listen to what these CEOs say on the topic and they basically admit something terrible is being built, but that the most important things is that they are the ones to do it first.
(I think something along these lines was actually in the Terminator 3 movie, the one where Skynet goes live for the first time).
Agreed though, no relation to the actual post.