Evidence isn’t enough

Why do some people continue to believe that climate change is not a threat despite the overwhelming scientific evidence that it is?

One theory is that the people who reject the evidence for climate change are doing so because they are uninformed. The idea is that if they were only a bit more educated on science they would suddenly realise the error of their ways. This is called the knowledge deficit model.

Researchers have sought to investigate how accurate this model is. They tested the hypothesis that a lack of knowledge about climate change predicted thinking it was less of a risk. They tested subjects scientific literacy and compared that with views on climate change.

If their hypothesis were true we would expect to see that risk perception would rise with more knowledge. It would look a bit like this.

source here

In fact they found this. 

source here

There was no correlation between scientific literacy (as measured by a test) and belief that climate change was a risk. Some of those who believed climate change was a big problem knew a lot about science and conversely some who knew little about science thought the risk was great. In short, the knowledge deficit model was a bust.

However, when the researcher compared political affiliation with belief in climate change they found the correlation they were looking for. It seems that being left wing was a much bigger indicator of believing climate change was a risk and vice versa for being right wing and this was regardless of scientific literacy.

The researchers propose that when a subject, like climate change, becomes politically partisan, facts will do very little to alter opinions. David McRaney has dubbed this “tribal psychology“. When something becomes a matter of identity for your in-group, then the evidence ceases to matter very much.

Take something as simple as wearing face masks during a pandemic. A year ago it would have been hard to image people physically attacking each other over an issue like this, but here we are. McRany has an excellent Podcast on this subject. He notes that masks may now be a badge of group loyalty, like putting pronouns in your bio. These findings are especially interesting when thinking about research and teacher beliefs.

In education most people are already left wing, yet the battle between trad and prog teacher “tribes” rages on with almost daily outrages during which members can prove their group loyalty. Today, it’s language use in schools, before that it was exclusion and before that it was something else. Topics change, but the dance remains the same.

It’s very “natural” for people to pick a side and to stick to that side regardless. As McRaney notes, our evolutionary history as social animals means that group membership could have meant the literal difference between life and death. It’s also a very frustrating part of human psychology leading to irrationality, partisanship, class divisions, religious intolerance, and so on.

A large body of psychological research shows how when we form in and out groups we start to view those two sides very differently. Our side is hard done to, sensible, upstanding and decent. The other side is getting away with murder, idiotic, immoral and craven. If we make a mistake, it was an accident, if they do it was calculated. worst of all, McRaney notes that people would rather be wrong than out of good standing with their “tribe”.

If people’s identities are wrapped up in a certain sets of beliefs and if those beliefs form an in-group/out-group dynamic it is likely that everything will be viewed through that lens. it is also likely that, in this situation, no amount of evidence will change minds. A good example of this is phonics research, the results of which have been resisted for decades.

So here are some suggestions to try to avoid our natural tendencies towards “tribal psychology”. This advice is aimed mostly at myself, but if you find it useful, then great. I suggest these in the full knowledge I will probably be unlikely to follow them very well.

  • Try not to become too invested in any one side 

This is very difficult, but the more we see the other side as the enemy, the easier it is to start seeing everything through a partisan lens. Next, we may stop looking at the evidence all together and just side with our ‘tribe’.

  • Try to find things you can agree with  

Whenever possible and no matter how small it is, try to find points of agreement between you and people in your out group.

  • Try to stay agnostic where possible 

We would like the research to say “X” but it doesn’t exactly say “X”. Well, the best thing we can do is just not have an opinion. “I don’t know” is a valid opinion.

  • Be dubious of research that supports your world view

Paper finds thing I want to be true is wrong *scoff* “what was the sample size?” “Let me check the methodology section!” “This author is right winger!”

Paper finds thing I want to be true is true! RETWEET without reading! “OK the sample size isn’t ideal but it’s a promising piece of research!” “who cares what the author’s politics are?!”

  • Make appeals to people on their terms

Learning styles advocates never seem to get particularly fazed by the charge that the practice lacks evidence. They do, however, get upset by the idea that it pigeon holes students. My guess is learning styles advocates like the theory because it represents individualism and student centric learning. Questioning that claim resonates more than pointing out the lack of evidence.

Facts aren’t enough

Research tell us that the knowledge deficit model is flawed. If we can’t get people to believe in the threat of climate change despite the evidence, there is little hope of convincing them about anything else. A research based approach has to consider not only the facts that can be gleaned from research, but people’s feelings too. Bashing teachers over the head with research is unlikely to change minds but it may cause them to resent you. It’s then fairly easy for teaching experts to dismiss research as “irrelevant for teachers” and find a receptive audience.

Non evidence based teaching tips

Here are a random collection of things which I think are really important but which are not really evidence based.

5. Deal with behavioural issues as soon as possible

If you’re like me, then having a student on their phone or basically doing something distracting will put you off your stride. I think most of the time, with adult learners there’s no malice, they are just thinking about their own problems.

If something is bothering you in class then deal with it as soon as possible. If a student is bothering you, then it’s likely they are bothering other students in the class. They suffer and you suffer. I usually ask to see anyone who is doing something they shouldn’t after the class. I don’t think this has to be a difficult conversation, just “don’t do X”. Setting clear rules on day 1 is a big help.

4. use coloured paper with wide margins and light ink when making cards

It might seem a bit “TEFL” for an EAP teacher but it really saves time. If you are producing cut-ups then pre-make a document which has side margins wide enough that you don’t have cut them at all. Make the spaces between the sentences as large as possible because there is nothing worse than chopping a few words off one set. Finally, print each set on different coloured paper to make them easy to sort.

3. Everything usually takes longer than you think

I have a compulsive fear of running out of materials before the end of the lesson and I know that I am not alone. I did have a tendency to over plan for a long time and end up not getting through the material. I’m better at this now, and there are some easy ways to control the time in the lesson. These usually relate to how you give the answers (just write them all up and let the students check themselves or nominate people and have others confirm the answers). You can also, – shock horror – do activities twice if you find you have too much time. It might seem lazy, but the students will probably benefit from the practice.

(I only actually ran out of material twice in 20 years, which isn’t bad.)

2. Learn student names

I’m not great with names but I make sure that I always do this in the first class. It’s really cringy by the third lesson to be pointing and saying “you” and I think it makes the classes less effective. If I can’t learn them because there are too many students, or because I will only see the group a few times then I give them a name plate of some kind.

  1. Do the damn worksheet

This is a cardinal rule for me and yet the one that I observe the least, much to my detriment. Do the damn worksheet yourself. It’s only once you are in class that you realise that Q3 isn’t actually possible in passive, or that you can’t explain something about the grammar point you are supposed to be focusing on, or you don’t actually know why “pain” is countable in “what a pain”, or how to explain the word “innovative”, or why “try” takes infinitives and gerunds while”trying” only takes infinitives, and on and on for 20 years. It is only when you try to do the worksheet that you realise the planned activity is not physically possible or that it would take 2 hours to complete. Do the damn worksheet!

What are your basic teaching tips?

Retraction in ELT

I am currently reading the new book, Science Fictions by Stuart Ritchie. It deals with meta research (research about research) and outlines all the ways in which science is currently going wrong. In one section dealing with retraction, Ritchie notes that “1.97 per cent of scientists admit to faking their data at least once” and suggests that that number is probably an underestimate as people are unwilling to admit to things like this even when asked in anonymous surveys.

This number means that for every 50 papers published in ELT one is likely to contain faked data. Some of these cases have come to light in biology and psychology and this led me to wonder if there were many retractions in ELT. So I asked twitter.

Marc Jones instantly found two (here and here) both of which were retracted due to plagiarism. One was plagiarism of another scholars work and the other was self plagiarism (submitting the same paper for double credit). I also found one from The journal of Computer Assisted Language Learning retracted for (self) plagiarism. So far no fraud.

Stuart Ritchie pointed me to the retraction watch searchable database. I tried searching by journal and found a RELC paper published by Ivan Chong which was withdrawn for “significant data errors“. There was also The journal, System which had the misfortune to publish one article twicein the same issue!

The journals Applied Linguistics, ELTJ and TESOL Quarterly have apparently had 0 retractions to date. The database is not complete though and I was also sent this piece which was retracted from the prestigious “language learning” journal. It’s not that clear what went on here but it seems like one author noticed errors in the data and requested a retraction.

As an interesting aside, Richie notes that people continue to cite retracted articles even after they have been retracted. I was curious about this so I used google scholar to check citations of the 2003 Language Learning article. There were hits from 2018 and even from 2019. I don’t know the date of the retraction but I feel pretty confident it was well before these dates.

Out of thousands of papers in ELT I could find only two that were withdrawn due to data issues. So either ELT is a paragon of virtue or we haven’t got very good at sniffing out fraud yet.

But how easy is it to get a journal to retract a paper? While researching an article on learning styles, I came across a couple of very curious articles. The first was Hyland 1993, the second was Hyland 1994. While reading the 1994 article I got a strange sense of Deja Vu:

In essence, learning style research suggests that people make sense of the world in different ways and these ways are partly created by cultural experiences (Hyland 1994)

Learning style research therefore suggests that people make sense of the world in different ways, more importantly however, these ways are partly created by cultural experiences (Hyland 1993)

and

Eight Japanese universities participated in the survey with 265 undergraduate students responding. The questionnaire was also administered to 140 Japanese students at various English proficiency levels at a tertiary college in New Zealand to determine whether overseas study influenced modality and group preferences. (Hyland 1993)

Eight Japanese universities participated in the survey with 265 undergraduates responding. The questionnaire was also administered to 140 Japanese students at different English proficiency levels at a tertiary college in New Zealand. (Hyland 1994)

and

Essentially the concept expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikes, but studies have addressed an enormously wide range of factors. (1993)

Learning style research expresses the simple idea that each learner has a clear and coherent set of learning likes and dislikes, but people differ in their learning styles in a number of ways and studies have addressed a huge range of factors. (1994)

Most of the later (1994) article is a verbatim copy of the earlier one with minor phrasing adjustments such as those shown above. This type of thing is usually known as either self-plagiarism (see the examples above) or, on a smaller scale, text recycling and is considered unacceptable in academic publishing. Many journals have rules against it, such as JALT itself:

Papers sent to JALT journal should not have been previously published, nor should they be under consideration for publication elsewhere.

I thought this was a bit strange and so I contacted JALT, the 1994 publisher, to make them aware of the issue. They told me that they take such matters very seriously and would investigate. After an investigation they informed me that this was all just an honest mistake, a bit of a mix up. The paper had been submitted to two journals by accident and and as a previous editor had dealt with the matter, it would be wrong to retract the article now.

How one accidentally submits the same article to two journals is, I must confess, a mystery to me. More mysterious was JALT’s reasoning. Regardless of what a previous editor had decided, a repeat publication is still in the literature with no indication that it is a repeat.

I wrote back suggesting that since I was unaware that it had all been resolved, as was the current editor and presumably future readers, it might be worth retracting the article, or at least adding a note to explain what had happened. They told me they were very grateful for my suggestion but no, they weren’t going to do anything. And so both articles remain in the literature.

It is also odd that Hyland himself, a incredibly respected editor and prolific author would not want the article to be retracted. In fact, until recently he continued to list both papers among his publications (his new blog, however, only lists papers from 2003). Not retracting the paper may be less embarrassing in the short term but it means that there is always the chance for some annoying blogger to bring attention at some point, to what was a mix up .

In truth, I’m not ‘that’ surprised that nothing happened here. Brendan O’Connor a student at the University of Leicester discovered that a well respected psychologist was “recycling” parts of papers into new publications to an alarming degree. Although O’Connor has documented this to an impressive degree, some journals were reluctant to do anything at all when confronted with these findings.

As whistleblowers, data sleuths and anyone else who’s contacted a scientific journal or university with allegations of impropriety will tell you, getting even a demonstrably fraudulent paper retracted is a glacial process – and that’s if you aren’t simply ignored or fobbed off by the authorities in the first place.

Science Fictions

Well quite.