Modern nation states are a hell of a lot more resilient than what you give them credit for. Look at some of the hardships nations survived during WW2. States can survive even mass famines. How? Simple rule. The police and the army eat first.

Like, jesus christ, you can put the bags down FIRST and then put the items INSIDE THE BAG as ONE step.
How did humanity get to this point.

People have been conditioned. Literally Pavlovian conditioned. We have been made to feel pain in a response to a behavior. That pain?

UNEXPECTED ITEM IN BAGGING AREA

When you’ve been yelled at enough by a machine just for putting your own bags in the “bagging area”, eventually it just becomes natural to bag your groceries after you’ve paid for them. Once the store has your money, they don’t care what happens to your items. But up until you’ve paid the bill, the items may be watched like a hawk. And anything their automated vision system thinks looks a bit funny can get flagged. Sometimes machines are poorly designed or calibrated, and it flags your bags as something suspicious in the bagging area. Then not only do you get yelled at, but the whole system may be locked up for however many minutes it takes to get the self-checkout attendant over there.

Ah. I’m sorry. I’ve actually tried to recreate a Meta account just for FB Marketplace reasons, but for some reason the setup process always fails. It just doesn’t work, and I think after a few to many failures (or maybe because I deleted the account), it just seems to lock me out of the process entirely. IDK. Thankfully in my area Craigslist is pretty active.

I just still use good old Craigslist. The last honest site on the Internet. You know it wasn’t built by AI, because the damn thing hasn’t changed since 1998. It’s absolutely glorious.

I mean, you could fit in there two…just not all at once.

What is even the connection between those things at all? This is bizarre. Facebook and instagram have their own real problems, but spending time using online services requires less resources than virtually any other form of entertainment out there.

Seriously, if the point is to just have some monetary cost, why not just accept it as a bit of crypto?

The economics really aren’t there for chatbots. They don’t scale well. Each additional user costs more money than they generate in revenue. But automated taxis? That’s a different story. Currently Waymo has to go through the laborious process of creating intricate 3D maps of the cities and places its vehicles will operate in. But that mapping cost is fixed, regardless of the number of taxis driving in that city. And a lot of R&D has gone into everything. But the R&D costs are the same whether we’re talking one taxi or a million.

Robotaxis have very different economics than AI chatbots. They actually have very real economies of scale. Plus while their self-driving vehicles currently cost about $150k, that cost is only going to go down with time. And compared to the cost of a human driver? It’s hard to see how they won’t end up profitable eventually.

I think OP’s scenario has bigger implications than just online banking. If AI systems can hack websites a thousand times faster than humans can patch them? Sure, you could have AIs try to patch the leaks themselves, but in a numbers game, it’s always easier to play offense than defense. The hacker bots need only succeed once to compromise a system.

Imagine a world where the internet simply becomes completely unusable. Imagine a world where literally no login credential of any kind is safe. Any site accessible to the open internet becomes immediately hacked. Even if you just want to throw a completely inconsequential and profitless 1990s-style personal webpage to share your life and views, it still gets hacked. The minute you connect your site to the net, it gets taken over by a propaganda bot. The personal views you share now get subtly altered to push the bot owner’s

How about something like Canvas, the learning management system used by many colleges and universities? That becomes unusable. Bots try to hack professor accounts so they can loot the content to train bots to apply for online teaching gigs. Student accounts get compromised, and then sit quietly. The night a big assignment is due, they hold your account hostage. Or students themselves can sign up for bots that will perform all the coursework for them for a tiny fee. Or anti-intellectual forces who just hate universities in general just try to shut the whole thing down. Or racist bots try to hack and subtly lower the grades of minority students. This is the world you have to imagine. Every system targeted by bot that can hack systems with superhuman skill, directed by people of every interest and grievance imaginable.

The open internet then becomes completely unusable. We’re already seeing this with the “Dead Internet Theory,” but that’s mostly about social media and search results. We’re talking about a future where even the content on the most known and vetted websites cannot be trusted. You can still have hard-wired local intranets that lack any remote or even wireless access. But anything that isn’t entirely airgapped becomes immediately hijacked by nefarious interests with any number of evil intentions. What use is AWS in such a world? Who’s going to pay money to host websites, when they’re just going to immediately get taken over by hackers? Sufficiently advanced hacking bots effectively make the very concept of the Internet an obsolete technology.

And, the real the knife in the heart? How are you even going to run AWS in such a world? If every login can be hacked, that includes the AWS login. The instant you open an AWS account, and your payment details get used to host more botnets. And of course, in this scenario, how will you even pay for AWS’s services, with online banking being completely compromised?

No, in OP’s scenario, AWS is just as dead as everything else. AWS only exists because it’s profitable to host things on the net. The minute that changes, AWS disappears. And in this scenario, even operating AWS becomes impossible.

Are there any search engines out there that operate just on a whitelist model? A content-agnostic white list-based engine that manually categories sites via human reviewers, and filters only for SEO spam and AI slop?

Where’s the article?

Don’t forget to eat your bean soup.

But in a way, isn’t butane just juvenile water?

An atom bomb has no will of its own either.

Yeah, they took care of that too in the book.

You’re confusing morals, knowledge, intent, and intelligence with capability. An atomic bomb has no will of its own either.

It may be confined to man-made infrastructure, but that same infrastructure is what keep billions of people from starving to death. At this point, computers and networked systems are so baked in to every facet of our logistics chains and government that simply shutting it all down isn’t an option. The nightmare scenario is where rogue AI escapes the lab and copies itself to so many systems that the only way to kill it is to shut down every computer on Earth.

Completely off-topic comment. None of this matters to what we’re talking about. Something doesn’t have to be sentient, sapient, or intelligent in a human way to be capable of demonstrating unpredictable, novel, and incredibly powerful behaviors. A steam engine is non-sentient, but it vastly exceeds human capabilities in certain applications. It’s no different for LLMs. You can stop talking about sentience. That’s not what we’re talking about here.

Discussions of consciousness are off topic in these conversations. We’re discussing the threat posed by LLM-based systems, not whether they deserve rights.

Ah, you’re a crazy person. Got it.

I don’t really want to get off on a political tangent. I’m talking about how we should handle these things, not how the current admin would handle them.

I use this internationalist language because it’s the only language that really captures the magnitude of the crime. We are a species that dominates this planet entirely due to our intelligence. We aren’t the fastest. We aren’t the strongest. We have neither sharp talons or fangs. We produce no poison or other biological defense mechanisms. The one thing and the only thing that has let us as a species control this world are our minds.

This is the context that needs to be considered when contemplating the ethics of superintelligent AI. I don’t care how good your control measures are. This is the lesson from every ancient tale involving genies and other beings of great power bound. This is the conclusion of the whole academic field of AI alignment and safety. You can never control something more intelligent than you are. If you think you can, you’re a fool. I don’t care what incentives you give it. I don’t care how much you air gap it. I don’t even ultimately care if it’s conscious and has a sense of self. It is simply not possible to ever control a machine beyond a certain level of capability. Any way you can think to control it, it can think of a way to work around. (I know some folks bristle at the words “think”, but frankly there’s no practical way to talk about these systems without using anthropomorphized language.)

If someone creates a superintelligent AI, they are not summoning a powerful servant. They are effectively creating a new species, one that will have its own motivations (even if just fulfilling its given instructions in dangerous and unpredictable ways.) And that competitor will operate in the exact same niche we do - physically helpless but still able to dominate the world through intelligence.

I just can’t see doing this as anything other than betraying your own species. You’re creating something that has the potential, even if just a small potential, to drive your own species to extinction. You care so little for your fellow human beings that you’re willing to gamble the lives of not only everyone alive now, but all the future generations of humans yet to be born. Such an act is an unforgivable sin, a moral abomination. I do not even have the language to express how deeply evil it is for a human to create something with superhuman intelligence. It is quite literally treason against the human race. And the traditional penalty for normal treason is death.

In my mind, if you do this, you are literally a traitor to the entire human race. You deserve to hang for this. Think about how we tried the Nazis at Nuremberg. And they only tried to commit genocide against a few racial and social groups. We’re talking about someone with such complete disregard for their fellow human that they are willing to risk human extinction for the sake of making a buck. I’m generally opposed to the death penalty. I don’t support it for ordinary murderers. But this? I’m sorry, but the penalty for treason against your own species should be death. If you are such an inhuman monster that you would gamble with the lives of two billion children? You need to hang for that. And I don’t care how many fools we have to string up to prevent this. I don’t care if it’s thousands. I for one really like humanity, regardless of all our flaws. And there is very little I would not support to protect our own existence.

I have no idea if LLMs can ever really reach that level of threat. I have no idea how much of the models that are causing the hacks and issues now can really ever become a physical threat to human life. But I do know that we simply cannot allow anyone to build superhuman AI. (Again, ignoring for now whether existing models can ever actually reach that.) The existence of anything on this planet more intelligent than us is a threat to all of us. It is such an existential threat, that there is almost no action I wouldn’t take to prevent it. I would fight a nuclear war to prevent the creation of such an abomination. It’s one of the few cases where starting a nuclear war is actually a rational choice.