Roko's Basilisk

digitalscream

Anybody come across this?

http://rationalwiki.org/wiki/Roko's_basilisk

It appears to be one of those rare thought experiments which has caused very real consequences for people who've considered it. As I understand it, it takes a number of unrelated concepts (the singularity, the concept of self and a few others) and extrapolates a logical conclusion in which the personal outcome is negatively affected by the mere knowledge of the thought experiment itself.

Some people have destroyed their own finances and suffered psychological harm just by thinking about it.

All because a forum's leadership didn't handle it properly. Wow.

Richardj

It hasn't suddenly just become April has it?

More sci-fi weirdness for the weak willed sheep.  IMHO of course

JayGee

[Sigh...]

Old news here.

Transhumanists are nutters. If that's the next evolutionary step I'll stick with the Mk1 Homo Sapiens...

Hootsmon

penicillin or get it lanced

monquixote

Just had a bit of a read. 

Seems a bit silly to me.

digitalscream

monquixote said:

Just had a bit of a read. 

Seems a bit silly to me.

Incredibly so...however, the way the forum admins chose to deal with it shows they don't really understand their community. It's obvious that some of them would get so obsessed with an idea that it'd cause some serious problems.

monquixote

Yes as they say it's probably a good example of the Streisand effect. 

I'm baffled that someone could be intelligent enough to understand the concept whilst simultaneously not grasping that it's a concept holier than the pope's fishnet tights.

digitalscream

monquixote said:

...whilst simultaneously not grasping that it's a concept holier than the pope's fishnet tights.

Indeed. It would, however, make an excellent Doctor Who plot.

PolarityMan

This is not a straightforward "serve the AI or you will go to hell" — the AI and the person punished have no causal interaction, and the punished individual may have died decades or centuries earlier. Instead, the AI would punish asimulation of the person, which it would construct by deduction from first principles. In LessWrong's Timeless Decision Theory (TDT),[3] punishment of a copy or simulation of oneself is taken to be punishment of your own actual self, not just someone else very like you. You are supposed to take this as incentive to avoid punishment and help fund the AI. The persuasive force of the AI punishing a simulation of you is not (merely) that you might be the simulation — it is that you are supposed to feel an insult to the future simulation as an insult to your own self now.

Wouldn't a suitably advanced AI realise that this is pointless..even putting aside if it was possible?

digitalscream

PolarityMan said:

Wouldn't a suitably advanced AI realise that this is pointless..even putting aside if it was possible?

It's a context thing - one of the central agreements on LessWrong is about acausal bargaining. It's basically where two agents (without any causal or direct link between them) learn that they can each get what they want if they give the other what they want, without actually interacting with each other. That then forms the basis of TDT.

The other part of it is another central agreement - that the person is equivalent to a computer program (and its data). Based on that equivalence, the computer program can be run anywhere capable of interpreting its code, therefore so can the person. For some reason, they ignore the fact that instances of programs can do different things depending on where they're run, but that's one of the many inconsistencies with the theory.

If you assume those things to be true, then the AI's position is less absurd. Those things, however, are not absolutely true (particularly the latter), so the AI's position remains ridiculous.

thermionic

digitalscream said:

All because a forum's leadership didn't handle it properly. Wow.

So... do you think Mark D Phillips is a replicant sent by an AI from the future to punish us with hacked nuts and Tele bridge pickups at the neck?

Richardj

Guys, GUYS, you are discussing this in DEPTH.  Everyone now leave, have a proper drink and get normal. Please. Now.

Bogwhoppit

I used Roko's Basilisk on my sore bits, and it burnt like hell

PolarityMan

150,000 people die every day. Some of those deaths are truly unavoidable, but most are the result of inadequate knowledge of cognitive biases, advanced futurism, and quantum mechanics.(**)

This guy from lessWrong is seriously almost offensively insane.

PolarityMan

1) '-D(u/h)' optimizes for human understanding

I do wonder if this whole site is some kind of elaborate troll...you know like my sig.

holnrew

It's illogical.