In defence of: The Test

13 Mar

There seems to be a lot of anti-testing sentiment prevalent in the teaching world at the moment.  There’s a particular degree of vitriol that seems to be reserved for standardised testing, but which has tapped a general anti-educational zeitgeist and spilled over, flooded even, into ELT.  In this piece I’m hoping to look at where some of these attitudes to testing might come from and think about what might be the best way forward.

Tests are tools.  Much like a hammer or a wrench, they can be extremely effective when used for their intended purpose.  More frequently, they get used in a bodge job – that is they get used to do a job they weren’t designed for but which kind of works, the end result is the same and only someone taking a close look would know the difference.  It’s perhaps their use as a blunt instrument which offends so many people – learners get coshed in the back of the head by a testing regimen that seems to exist only to perpetuate itself and leaves both teachers and students feeling slightly stunned, queasy and when it all goes wrong?  Mugged.

Tests can:

  1. tell you what class the student should start out in (placement).
  2. help you figure out problem areas the learners need more help with (diagnostic).
  3. tell you whether the learners are ready to go for a high-stakes exam (predictive / diagnostic / mock / aptitude).
  4. tell you whether the learners have, in fact, learnt anything (achievement / summative)
  5. introduce an element of competition to the class (informal / competition)
  6. tell you what the learners can actually do with the language (proficiency)

And I’m sure there are many other uses besides these.

Possibly the problem that comes up most commonly is use of tests for inappropriate purposes.  I haven’t turned up any research that examines the proportion of inappropriately applied tests – let me know if you know of any!

This is not the fault of teachers, many teachers are asked to design tests without any formal training in the principles of testing and assessment – or even any informal training.  There is possibly a vague idea of what is to be tested, but not necessarily knowledge of why this needs to be tested or what the best way to go about testing it might be.  For example, I might ask my learners to write an email to their friend telling them about their holiday.  Fine, but how will I assess this?  Am I looking for production of specific language forms, is this a test of past / narrative tenses and lexis associated with holidays?  Or do I want to see discourse patterns and structure with letter writing conventions?  Or both?

There might then be an issue when tests designed for one purpose are employed for another.  If I want to find out whether my learners have retained anything from the last three months, I might design a grammar and vocabulary test  based on specific language items covered in the class.  But this does not tell me how good at English my learners are – it tells me whether they have learnt the items I am asking them to recall.  Memory does not equal ability.  There is nothing wrong with using a test designed to ascertain retention – unless it is used to determine ability.   I suspect that many language schools suffer problems with learners being moved up into inappropriate levels because the tests used to determine whether they should progress do not demonstrate “language”, merely “language like behaviour” (see Cook & Newson 2007, p53), and mastery of the latter is deemed to be sufficient.  When learners then attempt a high stakes proficiency test (for example FCE et al), there can be problems as their knowledge does not always match their ability.

Reagrdless of whether the test employed is fit for purpose, the other big issue surrounds results and their interpretation.  We have, perhaps sadly so, ended up in a results driven environment where success is everything and where nobody knows how to interpret failure appropriately.  If all my learners get 100% on their reading test, does it tell me that all my learners are brilliant at reading?  Does it tell my colleagues what an amazing teacher I am?  Or does it tell me that the reading test was too easy?  If one of my learners fails, does it tell my school director I’m a bad teacher or does it tell me what the learner needs to focus on in order to improve?

Actually, that last question is a pertinent one and speaks to the heart of many of those currently railing against the system in the anti-test camp.  When I say, it tells me what the learner needs to improve – this does of course depend entirely on the test.  If it is a good one, it will tell me whether the learner lacks vocabulary in a certain topic area, or lacks the ability to discern subtext / speaker intent in a listening task.  If the test is poorly designed, it does neither.  At this point we enter the idea of teaching to the test.

If you ask many teachers for their views on teaching to the test, many would answer “Well, I don’t like doing it, but there isn’t really a choice.”  I beg to differ.  For a start, I think that if it is possible for a learner to improve their mark ONLY by doing practice tests, then the test is fundamentally flawed and is not testing whatever it was intended to test – it is testing test taking technique.  Thus the problem with preparing learners for high-stakes tests is that the test then tends to determine the curriculum.  Wall and Horak (2006, p119) found that for most institutions teaching TOEFL – “the test is the curriculum” and one teacher is quoted as saying “I teach taking a test, not a language skill.” (ibid, p68).

Some test providers actually factor “teaching to the test” into their considerations and attempt to design their tests so as to improve the standard of teaching.  Studies that look at whether or not this happens are referred to either as “washback” or “impact” studies – both UCLES (e.g. Cheng 2005) and ETS (e.g.Wall & Horák 2006 & 2011) take this into account in their test design.  The theory here is that by designing a test that it is not possible to improve your scores on merely by repetitive practice, you engender a situation where the teaching must focus on improving language skills.  Whether these efforts work or not are debatable, not least because many teachers and learners simply don’t believe that this is the case and revert to test practice as the principle teaching and learning methodology.  I should highlight a difference here between test familiarisation and test practice.  In my view the former is a necessary process to help learners understand what types of tasks are given in an exam and how best to approach them.  In the latter, any language development is often coincidental.

Leaving the issue of test practice aside, there are knock on effects down the system when a high stakes test is used or introduced, namely in ascertaining learner readiness.  This is especially true where teachers and schools are assessed or judged on the results.  So an additional testing regime is laid over the top of the classes to make sure that the learners are ready.  Additional skill assessments, grammar and vocabulary tests, practice testing in class, mock exams – in retrospect you can understand the earlier comment “I teach taking a test” – in all the testing, I wonder if there’s actually any classroom time left for language development work?

It is clear that the system, if not completely broken, is not functioning as it should.  It has, perhaps been “bodged” to the extent that even an untrained eye can spot the flaws from a mile away and that most participants in the process have lost their faith in the security and honesty of the system.  It is not longer an achievement, it is an ordeal.  This however, is not the fault of the test.  The test is trying it’s hardest to do the job we ask of it – it’s not the hammer’s fault it doesn’t function as a screwdriver.

A personal note:

I don’t have any quick and easy solutions – I’m not sure anybody does particularly…  I do however have some half formed thoughts and vague ideas about ways of moving forward from where testing is now (at least in a language school context), but precisely because they’re not fully thought through, I’m not going to share them just yet.  I am however very keen to hear any and all ideas that readers might have!  How do you approach testing in your context?  Is there too much teaching to the test?  What alternatives are you aware of?  Do please comment on this one!

Further Reading & References:

Cheng, L.  (2005)  “Changing language teaching through language testing“, UCLES: Cambridge

Cook, V. and Newson, M.  (2007)  “Chomsky’s Universal Grammar: An Introduction”Blackwell: Oxford.

Wall, D. and Horák, T.  (2006)  “The Impact of Changes in the TOEFL® Examination on Teaching and Learning in Central and Eastern Europe: Phase 1, The Baseline Study.”  ETS

Wall, D. and Horák, T.  (2011)  “The Impact of Changes in the TOEFL® Exam on Teaching in a Sample of Countries in Europe: Phase 3, The Role of the Coursebook. Phase 4, Describing Change.”  ETS.

Advertisements

3 Responses to “In defence of: The Test”

Trackbacks/Pingbacks

  1. In defence of: The Test | ESOL exams | Scoop.it - Thursday 22 March 2012

    […] background-color:#222222; background-repeat : repeat; } teflgeek.net – Today, 12:43 […]

  2. In defence of the standardised test | efl-resource.com - Tuesday 27 March 2012

    […] Thomas David Petrie distinguishes between good and inappropriate uses of standardised tests, and argues for their use in specific situations.Share this post:Bookmark on DeliciousDigg this postRecommend on FacebookGoogle Buzz-up this […]

  3. Ten Things Worth Further Investigation (#03) « teflgeek - Wednesday 20 June 2012

    […] Not so long ago I wrote “In defence of: The Test” – Annie Paul has written a similar piece for TIME Magazine: “In defence of […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: