Friday, April 03, 2009

Good science is not good science if it is unethical.

I do not any longer think there is that much of a distinction between good science and bad science.

For sure “crank” science is as much the bane of this generation as it has ever been, in terms of giving unsupportable validity to distasteful concepts, however following on my recent blog about the distortions of science in Nazi Germany I really think that what is equally important to good science, is not it’s accuracy but the spirit in which it is carried out.

Where most autism science fails is at the outset, before even the construction of the experimental paradigm, is in the attitude, because before everything comes the formulation of the hypothesis, that imaginative leap from what is currently known or unknown, toward the question one believes one can answer through the scientific method.

Nobody comes to this without bias. My particular bias, which you could argue gets in the way of objectivity, is my inability to read the serried ranks of journals, without an increasing sense of anger, with regard to the contempt that most of this shows to the humanity of my people.

I cannot but see the FMRI scanner in the same intrusive light as the discredited anthropometrists calliper.

So much of the paper chase is driven not by a desire to know anything useful, but by the pressure to publish, and to publish something new and original, even to the neglect of examining the wider context and grounding for that.

It would be disengenous (Godwins law again) to compare most scientists currently climbing the academic ladder with the racist tainted science of pre WWW2 Germany, however I cannot help wondering, if the metaphorical gloves were taken off, what they might do. The prospects for human veniality are bleak, as the experiments of Zimbardo and Milgram (in themselves dubiously ethical) have shown.

It seems to me, in journal after journal all we see is a number of cliques arguing between themselves, that “I think this and my scans prove it,” never mind someone elses set of scans show something else and “prove” something else.

In terms of good science vs bad science this may well be because of poor experimental design, and bias being multiplied through the analytical tools chosen, just as something that is not there can come out of a photoshopped picture simply through chosing one particular filter algorithm over another.

(Does anyone remember the famous Nessie pictures, where reputable experts in there own fields Edgerton, strobe photography, Rines , Sonar produced evidence that Nessie existed, only to be debunked some years later when one of the pictures turned out to be largely an artefact of the enhancement of a rotting tree stump )

One should beware of simply taking anything for granted just because it has a University seal behind it and has passed peer review, because one needs context, context is everything.

The context with autism, is that everything so far has proceeded from it’s cultural construct. The scientists all come in with a bias, that autism exists, and it is this, that or the other. Nobody has ever reframed any of the questions ab initio.

So is my bias against their ethics any worse than there bias against the dignity of the “subjects” they research and there refusal to countenance that they may be mistaken, especially given the mutually insupportable and contradictory nature of so many of their findings.


Joseph said...

I think the biggest problem in published science, besides outright fraud, is that we don't know to what extent negative results are not published, because they are not that interesting, or because the results were not what the researchers expected. There could be a lot of statistical noise in the literature, and this is not easy to measure.

I think most decent scientists realize that you shouldn't call one unreplicated study 'science.' I'd say you need a substantial body of work by many different teams of researchers, with papers of many different levels of quality, to be able to call something 'science.'

When it comes to the question of what scientists should study, what they should be interested in, what their biases are, and so on, I don't believe there's much that can be done to control that, although it's done via funding to an extent.

laurentius rex said...

I am not accusing anybody of fraud, just of accidental and incidental cognitive bias, seeing what you think ought to be there and unconciously tweaking.

Of course replicability is not the hallmark of success either, I am reminded of the old chestnut that comes out every time in any research teaching, that a reliable watch can tell the wrong time, unfailingly.

Science is not method, it is the repository of knowledge and if that knowledge is incorrect, well it is incorrect no matter how it was derived.

The problem with German Science under Hitler (and I have taken the book back so I can't find the exact reference) is what it aspired to be. An aryan science, in which all the sciences were syncretised into the pursuit of the one goal.

Now I am a syncretist myself, being a polymath I cannot help but be. The biggest problems in science do come from the distance at which the various branches of research have come from the roots, so that with the increasing specialisation one really is stuck in a rut, one cannot get out of.

What has happened is that there are so many of these ruts, and they are so deep that no-one can see over them.

I had a discussion with Declan Murphy which was enlightening, apparantly any old brain scan will not do, researchers prefer images gained on the same equipment, calibrated the same way. Whilst I can see the value of that for consistency it is like the reliable watch question isn't it.

We have to ask for every study that makes a new claim, what else could be creating that effect, that the researcher has not considered.

There is just so much selection bias, from who one includes as a "subject" or a "control" to just what instruments one uses and whether someone else would get the same results if one removed those biases. They rarely do.

Estee Klar said...

"Science is not method, it is the repository of knowledge and if that knowledge is incorrect, well it is incorrect no matter how it was derived."

I enjoyed this post. What is knowledge if not simply a sum of perspectives built upon previous ones?

Dinah said...

I enjoyed this post too, important issues, and salutary reminder re accidental and incidental cognitive bias