A Potted History of Modern Rationalism

Like most rationalists, I do not identify as a rationalist. 

Nevertheless I notice that that movement has grown to the point where there are people who do identify as rationalists but do not remember where it came from.

And indeed there are persons attacking the movement for being unduly convinced of the power of human reason. This would indeed be good attack on rationalism, the 16th century philosophical movement which was opposed to empiricism.

It is not such a good attack on the modern movement which perhaps unfortunately shares its name.

So I had occasion to write a short history of modern rationalism as I remember it appeared to me as it was developing:

--------------------------------------------------------

Our holy text, "The Sequences" was originally a series of blogposts on Robin Hanson's blog "Overcoming Bias", which he shared with the blessed Eliezer Yudkowsky.

Their joint project was to find ways of overcoming the recently discovered 'cognitive biases' that suggest that humans don't usually reason correctly, compared to an idealized rational agent (which is kind of the standard model person in economics). Those cognitive biases were discussed in Daniel Kahneman's popular book "Thinking Fast and Slow", an excellent read.

Robin's shtick is pretty much "X isn't about Y", and he recently published an excellent book "The Elephant in the Brain", which is almost entirely devoted to the idea: 

Human economic and political actions (education, health, etc) make very little sense compared to their declared reasons, what the hell is going on?

It's a very good read.

Eliezer was much more interested in what an ideal reasoner would look like, because he wanted to build one so that it could save the world from all the other horrible existential risks that are pretty obviously going to wipe us all out quite soon.

That project got delayed when he realized, or possibly was informed by Nick Bostrom (of Superintelligence), that a rational agent would, by default, kill everyone and destroy everything of value in the universe.

So Eliezer's new plan is to save the world by working out how to design a powerful agent that will try to act as a benevolent god, rather than an all-destroying one. Or at least to try to convince all the people around the world currently working on artificial intelligence to at least be aware of the problem before destroying the world.

And Robin's still interested in economics and human reasoning.

[ This paragraph is disputed, see comments:

The whole 'Effective Altruism' movement was originally at least partly an evil plan by Eliezer to convince people to give him money for his scheme to optimize or at least not destroy the world (they are pretty much the same thing to a powerful agent), which has now ballooned out to be a separate force largely under the control of people who have other goals, such as eliminating suffering.

That's my personal memory, but it seems like that's wrong. I'm confused]

 

Scott Alexander/Scott Siskind/Yvain (πολυτρόπως) wrote all sorts of excellently readable articles on Less Wrong about how to think correctly if all you've got is a human brain, before deciding that it would be better to have his own blog where he could talk about political issues. (Less Wrong was against discussion of political issues because they're (probably for evolutionary reasons) incredibly hard to think about, and the idea was to practise thinking in less inflammatory areas.)

Anyway, I think it's reasonable to claim that the whole rationalist movement was founded on comparing broken human reasoning to what real reasoning might look like!

-----------------------------------------------------

I should point out that I don't actually know any of these people, don't even live on the same continent, am not privy to their thoughts and plans, and have made absolutely no contribution myself, but I have always been very interested in their published writings. I wonder if I should expand this outsider's view into a proper essay and seek input from the people I'm slandering.

Yudkowsky on Physics

An interlocutor recently spoke disdainfully of Eliezer Yudkowsky's writings about Quantum Physics.

The relevant excerpt is:

like the infamous quantum mechanics sequence, that seemed to fail to change anyone's mind with relevant domain knowledge

And I replied:

It's funny that you should mention the quantum mechanics sequence. That was actually my introduction to his thinking.

I did a degree in pure and applied maths, and although I was definitely on the "pure" side, QM was one of the things I'd gone to university to understand, so I took a couple of courses on it.

There's no question that I could do the maths, and I could solve all the examples and exam questions, and my teachers seemed very pleased with how well I took to it.

But it made absolutely no sense to me as a theory of how the universe worked. I don't mean that it was counter-intuitive. Lots of things in probability and physics are counter-intuitive. I just mean that I really didn't understand how to set up the problems, or what it meant to live in a universe which ran on QM. 

Whenever someone took a physical problem and represented it as a piece of quantum mechanical maths, the set-up for the problem just seemed completely ad-hoc. And any attempt to think about how such a universe might support living beings was hopeless. It really seemed to me that observers, and conscious observers at that, were part of the fundamental physics. And that they had to act from outside the physics.

My teachers were very much of the 'shut up and calculate' school. We know it doesn't really make sense, they said, but people have been beating their brains out about this for something like seventy years and no-one's really come up with a way of looking at it that makes intuitive sense. But on the other hand, the calculations do give us a way of predicting the results of lots of otherwise incomprehensible experiments. Your best bet is to push on with doing the calculations and maybe by the time you get to Quantum Field Theory it will have started to feel more normal.

I wouldn't accept that. But I hadn't got anything better, so I eventually just gave up on the subject.

I stayed curious though, and in fact later on I had several conversations with people doing actual PhDs in QM, where it became obvious that they just didn't get it. They had really fundamental misunderstandings of what was happening in simple set-ups like the double-slit experiment.

Which didn't stop them making worthwhile contributions, of course. Euler didn't get a lot of the stuff he worked on, he makes some quite silly errors with complex numbers at times, because he didn't understand what they were. But his intuition and technical skill allowed him to get results that ended up being actually proved very much later. This is quite normal in mathematics, and is probably the real difference between pure and applied maths.

One day someone told me that they'd read an explanation of QM that seemed sane, and that I should take a look at it. They linked to EY's essay on Less Wrong.

And finally I got it. I don't claim to understand how the quantum universe works, or what consciousness is, or anything like that, any more that I would claim that about a classical universe.

But it doesn't seem mystical anymore. I can at least understand the model itself, if not how it relates to physical reality.

The mathematical model that looks like a wave equation represents a 'magic reality fluid' that sloshes around over the space of all the possible configurations of the universe.

There's no collapse, no mysterious observer effects, no physical systems behaving differently depending on who's measuring what and when they're looking at the measurements.

It's all just nice and sane and deterministic and predictable and follows laws.

And I absolutely didn't get that until I read EY's quantum sequence, which is really no more than a particularly well-written explanation of the many-worlds point of view written by an autodidact who'd managed to puzzle out for himself something that I'd been well and professionally taught by renowned experts at a famous university.

A little later I decided to have a go at Umesh Vazirani's Quantum Computation course on Coursera, and it made perfect sense. I could just do it. There wasn't anything funny or baffling going on. I don't think that would have been true if I hadn't read EY's QM essays.

So I don't know if you'd count that as 'changing the mind of someone with relevant domain knowledge', but it certainly seemed important to me.

And I decided that I liked this man and his thinking and his style of writing, and that's why I read the rest of The Sequences.

Young Earth Creation Science for Bayesians

I have recently been really impressed by some of the people struggling to reconcile the Word of God with the evidence all around us, which does not seem terribly supportive on its face. 

(An excellent example: https://creation.com).

They strike me as some weird mirror of us. I really don't think they're being dishonest.

What they're saying is: "Given that we know that the Bible is true, what's the simplest explanation of the evidence that we see that makes sense?". 

I can't refute their arguments, they know more about the relevant disciplines than I do.

But I'm fairly sure that if I did come up with a good objection, they'd respond: "That's a really good point, we accept it, let's work out what sort of explanation does work to both explain the evidence and stay compatible with the Bible."

They're doing excellent work even from our point of view in pointing out holes in our best theories, and challenging us to make our arguments watertight.

That's exactly what I'd be doing if I had a really really high prior on the Bible being true.

Where do they get that prior from? Well, priors are kind of a problem for us as well.