Quote:

---

Would your rationale for this pause have applied to basically any nascent technology — the printing press, radio, airplanes, the Internet? “We don’t yet know the implications, but there’s an excellent chance terrible people will misuse this, ergo the only responsible choice is to pause until we’re confident that they won’t”?

---

Quote:

---

Why six months? Why not six weeks or six years?

---

Quote:

---

When, by your lights, would we ever know that it was safe to resume scaling AI—or at least that the risks of pausing exceeded the risks of scaling? Why won’t the precautionary principle continue for apply forever?

---

Quote:

---

Were you, until approximately last week, ridiculing GPT as unimpressive, a stochastic parrot, lacking common sense, piffle, a scam, etc. — before turning around and declaring that it could be existentially dangerous?

---

Quote:

---

How can you have it both ways?

---

Quote:

---

If the problem, in your view, is that GPT-4 is too stupid, then shouldn’t GPT-5 be smarter and therefore safer? Thus, shouldn’t we keep scaling AI as quickly as we can … for safety reasons? If, on the other hand, the problem is that GPT-4 is too smart, then why can’t you bring yourself to say so?

---

Quote:

---

Originally posted by 6IXSTRINGJACK:

There is possibly a race of beings on another planet in our near infinite universe that is capable of being responsible shepherds of AI technology. Human beings are not capable of that.

Not even close.

---

Quote:

---

Originally posted by SIGNYM:
The problem with AI is that, unlike nuclear power/weapons (another potentially world-ending technology) we don't have a prayer of understanding what it's doing, or why. In fact, people are dicking around with it in the hopes that it can "understand" or "see" things that mere mortals can't.

---

EXACTLY. And by "we", let's not fool ourselves into thinking that we're just talking about you and I, the other dummies on this site, and most of the dummies we know in our lives. The people who are working on it don't know what it will be capable of either.

I'm always laughing about the fact that most of the people in my life walk around with smartphones today, and these are the same people that showed zero interest in computers while it was my most passionate hobby growing up. The only time any of them even touched a computer was if their job required them to, and they hated every minute of them. So I look at them as the average car driver. Sure, they know how to drive it. Maybe they even know how to change a tire or if they're really adventurous they know how to change their own oil. Other than that, they have no idea what's going on under the hood. And there was one point where I would have said that if they did have that knowledge they wouldn't walk around with a smartphone, but that day has passed. It's a drug and they're addicted. It wouldn't even matter anymore.

So no. I don't have a smart phone. I don't have anything close to the latest technology in my house in any category. My TVs are all hand me downs, my computers are all 10 year or older technology. I sacrifice a faster internet speed by rejecting the cable company's offer for a free Company Modem and use a much older one that at least will make them work to get into my network instead of giving them direct access to my computers that I signed away to them in the fine print.

I expect the average person to be dumb as well as ignorant. But at least when it came to nuclear weapons, the people working on it knew what the consequences would be. This is much more like tampering with bio-weapons, except it's probably 100 times worse. When you're refining a virus to be more deadly, it's still acting without intelligence even if on some basic level it does have a directive for self-preservation (and even that is arguable since many viruses will outright kill a host without thinking about it until mutations evolve that can co-exist with the host).

Quote:

---

Letting the genie out of the bottle is an apt metaphor. Just don't let it reside on the internet (in the "wild") where it might presumably copy itself onto every server in the world (or, worse, exist in transit everywhere) or control its own power supply.

---

This should be an obvious rule, but human beings are in charge so even if they made this law, I doubt it would matter. Somebody is going to leak it onto the internet just like somebody getting a copy of a new Hollywood movie a week before it hits theaters leaks it out on the internet.

And honestly, I'm not even arguing any of this on the issue of the "End of the Human Race" or anything so dramatic.

What I'm worried about is what will happen to everyone between now and the end.

I like not having to work because I set myself up right. But I'm lucky I've lived long enough to enjoy it because I almost destroyed myself with that freedom. Not only don't I think early retirement is good for most people on a "spiritual" level, but what's going to happen when all those meaningless and thankless McJobs that are going to people who can't get anything better disappear to AI? It's already happening right now. What are people going to do when they can't work and provide for themselves or their family because they're outdated?



I used to think this was funny and I had it hung on a wall in my apartment in the mid aughts:



I don't find this stuff amusing anymore.

--------------------------------------------------

Growing up in a Republic was nice... Shame we couldn't keep it.

Quote:

---

Originally posted by JAYNEZTOWN:
In the history of panic calling for a pause, this usually means they already screwed up

genie is out of the bottle

---

Quote:

---

Originally plagiarized by SECOND

So, one idea that people have had—this is actually Yudkowsky’s term—is “Coherent Extrapolated Volition.” This basically means that you’d tell the AI: “I’ve given you all this training data about human morality in the year 2022. Now simulate the humans being in a discussion seminar for 10,000 years, trying to refine all of their moral intuitions, and whatever you predict they’d end up with, those should be your values right now.”

There have already been some demonstrations of this principle: with GPT, for example, you can just feed in a lot of raw data from a neural net and say, “explain to me what this is doing.” One of GPT’s big advantages over humans is its unlimited patience for tedium, so it can just go through all of the data and give you useful hypotheses about what’s going on.

---

Quote:

---

Originally posted by SIGNYM:
Also, there are innate social instincts in most people. (SECOND seems to lack them, which might be why he aligns with AI.)

---

Quote:

---

Originally posted by 6IXSTRINGJACK:
Scott needs to shut the fuck up and sit down.

---

Quote:

---

Originally posted by second:

Quote:

---

Originally posted by 6IXSTRINGJACK:
Scott needs to shut the fuck up and sit down.

---

. . . no matter what AI safety proposal anyone comes up with, Eliezer has ready a completely general counterargument. Namely: “yes, but the AI will be smarter than that.” In other words, no matter what you try to do to make AI safer—interpretability, backdoors, sandboxing, you name it—the AI will have already foreseen it, and will have devised a countermeasure that your primate brain can’t even conceive of because it’s that much smarter than you.

I confess that, after seeing enough examples of this “fully general counterargument,” at some point I’m like, “OK, what game are we even playing anymore?” If this is just a general refutation to any safety measure, then I suppose that yes, by hypothesis, we’re screwed. Yes, in a world where this counterargument is valid, we might as well give up and try to enjoy the time we have left.

But you could also say: for that very reason, it seems more useful to make the methodological assumption that we’re not in that world! If we were, then what could we do, right? So we might as well focus on the possible futures where AI emerges a little more gradually, where we have time to see how it’s going, learn from experience, improve our understanding, correct as we go—in other words, the things that have always been the prerequisites to scientific progress, and that have luckily always obtained, even if philosophically we never really had any right to expect them. We might as well focus on the worlds where, for example, before we get an AI that successfully plots to kill all humans in a matter of seconds, we’ll probably first get an AI that tries to kill all humans but is really inept at it. Now fortunately, I personally also regard the latter scenarios as the more plausible ones anyway. But even if you didn’t—again, methodologically, it seems to me that it’d still make sense to focus on them.

https://scottaaronson.blog/?p=6823

The Joss Whedon script for Serenity, where Wash lives, is Serenity-190pages.pdf at
https://www.mediafire.com/folder/1uwh75oa407q8/Firefly

---