Simpson's Paradox

Suppose that you went to the doctor, and he told you that you had cancer.

And he said that there were two possible drugs, placebifen and nogudox, which he could give you.

And luckily there had just been a big survey done, where they tracked the progress of lots of cancer sufferers, taking both drugs, and a higher fraction of people taking nogudox recovered.

Which of the two would you ask him to prescribe, do you think?

A while later, you're at the pharmacist, collecting your nogudox, and the chemist tells you that he has just seen a large survey published in the science supplement of the Wisbech Advertiser, where for males like yourself, placebifen proved to be the better drug.

And because this is the twenty first century, and the Advertiser has an online edition, you go googling, and actually it is worse than that.

In the WA survey, placebifen was indeed slightly better for males with your type of cancer, but it was also slightly better for females.

So now you are a bit confused. If you are a man, placebifen works better, and if you are a woman, placebifen works better, but if you are a person, then you should go for nogudox.

What on earth do you think is going on?

As fine a place as the internet is, it's probably time to talk to your doctor. He looks at both surveys, and looks a bit grave. It turns out that the Advertiser has published the same survey as he read in the Lancet, but they put the conclusions in the other order. (Obviously no-one ever troubles to read the bits in the middle. In fact many journals, to save on typesetting expenses, simply replace everything between abstract and conclusion with

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam lectus. Sed sit amet ipsum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante hendrerit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean ut gravida lorem. Ut turpis felis, pulvinar a semper sed, adipiscing id dolor. Pellentesque auctor nisi id magna consequat sagittis. Curabitur dapibus enim sit amet elit pharetra tincidunt feugiat nisl imperdiet. Ut convallis libero in urna ultrices accumsan. Donec sed odio eros. Donec viverra mi quis quam pulvinar at malesuada arcu rhoncus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In rutrum accumsan ultricies. Mauris vitae nisi at sem facilisis semper ac in est.

Here are the conclusions of the survey:

In the Lancet:

            survival rate
placebifen  26%
nogudox     28%


In the Advertiser:

for males only

            survival rate
placebifen  20%
nogudox     15%

for females only

            survival rate
placebifen  40%
nogudox     35%

Pretty clearly, placebifen has a 5% advantage as a cancer cure, whether you're male or female.

Just as clearly, nogudox has a 2% advantage as a cancer cure if you're either.

If you're anything like me, and you haven't seen this sort of thing before, then you're staring in disbelief at these numbers, thinking 'But maths just can't work that way'.

That goes to show just how bad our intuitions are on matters of statistics and probability. I've seen this several times before, and I just constructed this example, and there is still a little voice shouting 'It cannot be'.

Here are the  actual numbers from the survey:

males given placebifen:   lived 40  died 160
males given nogudox:      lived 15  died 85
females given placebifen: lived 40  died 60
females given nogudox:    lived 70  died 130

Check the percentages yourself.