Friday, August 15, 2008

Critical thinking is not a bad thing

I talked a little about criticism in my last blog, and was indeed critical of a certain genre of writing about autism.

I intend to turn aside from the sociological rumblings around autism and back to what seems to be everyone's favourite topic in the blogosphere, which is the science of autism.

I will be presenting a paper very soon, which is rather critical of that science, and I expect the that those devotees who see everything in science as either black or white will be muttering to themselves "There goes would be Dr Larry again, with his sour grapes, knocking everything down but not putting anything in it's place" I had similar comments on my school reports when I refused to take the standard theories of the day at face value simply because they came with the authority of the teacher.

Well for once I shall defer a little to authority, to a more experienced researcher than myself who has doubtless read more papers than I know existed.

Incidentally it is an article of his that gave me a lot of pause for thought when writing my recent paper.

His name is Dermot Bowler, and he has not long ago had a book called:

"Autistic Spectrum Disorders, Psychological Theory and Research"

Published by Wiley in 2007

Now whilst his literary style would not gain him an A* pass in todays A level exams, I should be the last person to hold that against him :)

There are a number of chapters there where he reviews the current theories and the studies out without interposing his own position too much.

I would just like to quote a couple of pertinent passages from the final chapter of the book where he talks about the somewhat erratic nature of all the research so far:

"Failure to replicate findings can be explained in a number of ways. Either the phenomenon does not exist and the initial, positive finding was a random event. Or perhaps there were differences in the samples used; one study may have tested higher-functioning or older individuals while another used those with lower IQ. Many studies have small sample sizes and as a consequence have insufficient statistical power to reveal between group differences. Procedural differences may also yield different outcomes......"
There speaks someone who has read many studies critically in the way they should be read, and he even includes his own work in that summary.

Further on he says at the very end of the book:

"The speculations in the last paragraph bring us back to the question of what it must be like to be autistic (See Frith and Happe 1999). Ultimately, the psychology of ASD must provide an answer to this question. But it must also provide an explanation. Readers who have made it this far (as well as those who have skipped straight to here) may be expecting such descriptions and explanations as well as a punchy take-home message. Without wishing to be unkind to colleagues in the field (or perhaps wishing to be unkind to those who often misinterpret their ideas), the message is probably that we are sometimes too quick to generate quick snappy messages and that we are often too uncritical of the work we ourselves do. Autism spectrum disorder is now known to be a set of conditions that should not be reduced to a simple dichotomy of presence and absence. When present the conditions are multidimensional and complex, and although they share the common characteristics of social impairment and repetitive behaviours (at least from the perspective of a typically developed person) they often exhibit additional features that are not necessarily defining features of the spectrum. Such complexity requires a more subtle explanation than a simple reduction to an absent theory of mind, a failure of affective appreciation, diminished sense of self of fragmented perception. The complexity of ASD requires us to take a more distanced view and to go beyond simply trying to find new ways of describing the fact that people with ASD are autistic Science is about the reduction of complexity to simpler sets of entities and processes that interact in ways that are controlled descriptions of the behavioural manifestations of ASD. The challenge that faces us now is to step outside our own narrow conception of the issue and to work out how they fit together and why."
It is very pertinent for me to be considering that too, at a time when I am currently trying to attract funding to my own research in the hope that I will in turn not replicate the faults of all too many small scale studies.

7 comments:

Socrates said...

Nevertheless, good solid psychological research has been done; i.e. Baron-Cohen's work with the Strange Stories study. Surely a model of good practice for psychological studies of autism?

Larry Arnold PhD FRSA said...

I don't actually rate SBC that highly as a researcher he has too much of an obsessional streak to be objective.

The point I am trying to make is that every researcher ought to be self critical and in the best Popperian tradition consider the null hypothesis first. I don't know where that purity has gone, and all I know is that I am in the same danger of going native myself and assuming too much to begin with.

However there is another danger of being so self critical that one never gets anything done because one cannot achieve that degree of perfection that would really satisfy one.

We go back then to folk like Weber, and Quine and make the most of our lot.

I have to be extremely critical of SBC and what he has previously made of TOM, there was a lot of empirical research at IMFAR which was suggesting very strongly (bearing in mind Bowlers caveats I have quoted) that the results are all artifacts of the experiments and once you find a non linguistic and ostensibly neutral way of looking at the phenomenon it diminishes in your sight.

Again the problem is of creating instruments to measure what they measure, they will always measure what they measure and in so doing create the entity that they measure whether it has any real coherent existance outside of that measurement or not.

TOM is a self defining problem, it can only be expressed in terms of what you describe it as or conceptualise it might be, it has not reality beyond the discourse that created it.

DJ Kirkby said...

I wish you success in obtaining funding for your research proposal.

Larry Arnold PhD FRSA said...

Hmm seems to me on critical reflection that calling one of the leading researchers "obsessional" is perhaps not the most polite way to go about it, and Simon if you are reading this I do wish you would pursue an alternative line to your extreme male theory and consider the possibility of social construction of the diagnosis somewhat in the incidence of males vs females.

As for ToM I think that one is gradually breaking down now as it seeks out an oversimplistic explanation which falls down when one considers just what it takes to set up an unbiased experiment, and how dependent it is on the notion of what "theory" of mind might be in the first place. It seems to me that neuroscience is somewhat like physics, in that the closer we look, the more intricate the system appears and the greater number of puzzles emerge. That is why I like Dermot Bowler, he seems to accept complexity and suggest that what eventually emerges as the observable traits of autism emerge from the complex interaction of systems as a whole.

CS McClellan/Catana said...

What strikes me most forcibly these days about reports of scientific studies (and not just those with relevance to ASD), is that they overgeneralize from too little evidence. The oxytoxin study, for example, may have relevance to social anxiety, but it is completely irrelevant to anyone on the spectrum who does not have social anxiety. Yet it is being discussed almost exclusively in the context of autism.

And then, as you note in reference to TOM, there is the question of correct diagnosis. It's ridiculous to speak of cures for conditions that haven't been accurately defined or diagnosed. An online Asperger's test that I took some time back came up with an amusing analysis: I probably have a social phobia. This was the outcome of responses that indicated a preference for being by myself. Does such a test reflect attributes such as introversion or a cost/benefit analysis of social relations? Considered preference vs inability?

RAJ said...

SBC's theories appear to be nonspecific. TOM deficits are also observed in mentally retarded, schizophrenic and even Alzheimer's patients.

The extreme male theory could also be applied to leprosy since there is a high (3:1) male female ratio reported. Many diseases have a high male female ratio, I suppose you could say that maleness itself is a biological risk for disease.

His Autism Qoutient Test, tests personality types, not a profound neurodevelopmental disorder.

Progress in understanding autism, however it is defined, has stalled since the mid-1970's.

We have entered a new era where new 'unified' theories of autism are published seemingly every month.

For example, Sabat and Wigler wrote their new 'Unified Theory of Autism' paper based on finding de novo copy number variations in a sample of a few hundred cases. They conjectured that there are only two types of autism, CNV's inherited from the parents which account for 10% of all cases and de novo CNV's which account for 90%of the cases.

They found CNV's in different chromosomes in each case but did not find more than one CNV in any case.

Their theory would dispel the polygenic hypothesis as a myth and since in 90% of the cases the parents were unaffected, the unified theory would also expose the Broad Autism Phenotype as another myth dispelled.

Larry Arnold PhD FRSA said...

Dispel Mischpell I dunno, but I have a lot to dispute with SBC as I have historically had to dispute with thee Bobby me old pal

I have never much been convinced by your arguments but then you cannot help it, genetics happenstance and the great wen notwithstanding :)

I like the counterpoint between Borat's cousin and the Aussie exile from Brum, neither of them will ever have unique truth the great sadness being that you shall have even less :(

I am actually prepared to stand up amongst those who think they are my academic peers being those who tend to denigrate my social peers.

You think you are radical, you think you are making a point, consider what I gamble on this every day, consider what I do not have to spend which I have spent and consider that I do no look to the main chance but what I want to reveal as the truth so far as the empiricist scientific tradition allows me to.