ESL EFL Writing Tests

The Pragmatic Tests

Dictation

Though dictation is not accepted as an efficient means of language testing by Lado (1961), Harris (1969), Anderson (1953), Somaratne (1957), J B Heaton (1975) and Davies (1977), according to Oller (1979) it meets the pragmatic naturalness criteria and is an efficient tool to measure language use.

Anderson (1953), considers dictation as a very indirect and inadequate test of auditory comprehension, while Somaratne (1957) and Ingram (1977) consider it as a test of spelling only.  According to Lado (1961), dictation does not test word order or vocabulary since they are given by the teacher and it measures very little of language.  J B Heaton (1975) also thinks dictation is not effective in providing a means of assessing any one skill because it measures too many language features.  Davies (1977) also suggests that dictation is too imprecise in diagnostic information.

But Oller (1979) while admitting the limitations of dictation, considers it to be a reliable means to test languages proficiency, if it is administered and scored properly.  According to him, any proposed testing procedure, which is to qualify as pragmatic language processing task, must meet two naturalness criteria, viz.

i.             It must require the processing of temporal sequences of elements in the language constrained by the normal meaningful relationships of such elements in discourse, and

ii.             It must require the performer of the task to relate the sequences of elements to extralinguistic context via pragmatic mappings.  In short, pragmatic tasks require time constrained processing of the meaning coded in discourse.

According to Oller, dictation meets both the criteria of pragmatic tests, if the sequences of words or phrases to be dictated are selected from normal prose, dialogue or some other natural form of discourse, and if the presentation of the material challenges the short-term memory of the learners.

“....    a simple traditional dictation meets the naturalness requirements for pragmatic language tests.  First, such a task requires the processing of temporally constrained sequences of material in the language and second, the task of dividing up the stream of speech and writing down what is heard requires understanding the meaning of the material - i.e., relating the linguistic context (which in a sense is given) to the extralinguistic context (which must be informed).”       (Oller, 1979; 39)

Types of dictation

There are different types of dictations. Some of them are as follows:

$               Standard dictation – The complete passage is read out and learners are asked to write down what they hear.

$               Partial dictation – Part of the passage is given in writing.  Learners hear the complete passage and try to fill in the missing parts.  This technique combines dictation with cloze.

$               Dictation with competing noise – The dictation passage is read out on tape. Some noise is superimposed on the taped version of the passage.  The noise may be such as we find in real life situations.  For example, talking to somebody in a factory while the machines operating are making a lot of noise.

$               Dictation – composition – Before starting to write, learners listen to a passage two or three times.  Then they try to write from memory the passage as they heard.

$               Elicited imitation – Learners listen to a passage two or three times.  Then they try to recount orally the passage as they listened to.

ii.         Selection of material & administration procedure 

Once we decide the purpose of testing and the procedure which is most suitable to out learners, we can select the material.  The material should be appropriate to the level of the learners.  Once the material is selected the difficulty level can be decided.  There are a number of ways of influencing the difficulty level of the task even after the difficulty level of the material has been set by the selection process.

Factors influencing the task difficulty are as follows:

The conceptual difficulty of the word sequences themselves (other factors being held constant

The overall speed of presentation

The length of sequences of material that are presented between pauses

The signal to noise ratio - i.e. the amount of noise added to the material

The number of times the text is presente

The dialect and the enunciation of the speaker and the dialect the hearer is most familiar with

A miscellany of other factors.

Since the purpose of the test is decidedly not to assess the speed with which examinees can write, the pauses must be long enough to ensure that the task is not turned into a speed writing context.  A rule of thumb suggested by Oller is for the examiner to subvocalize the spelling of each sequence of verbal material twice during the pause while the learners are writing it.

Scoring Procedure

Standard dictation is usually scored by allowing one point for every word in the text.  There are two methods of scoring.

Error – counting method – Here the number of errors are counted and the sum is subtracted from the total number of words in the text.

Correct words – in – sequence method – Here the number of correct words in sequence are counted and assigned one mark each.  The total is given out of the total number of words in the original text.

These two methods are not perfectly equivalent if intrusions are counted as errors.  For example, if two sentences are written as follows:

              ‘This is a Ramanbhai.  His is a good doctor’.

As per the error counting method ‘His’ will be counted as an error.  It should have been ‘He is’.  So the learner will get two less than the total number of words - i.e. six out of eight.

As per the correct words-sequence method, if the errors of intrusion are counted, then the article ‘a’ in ‘This is a Ramanbhai’ will also be counted as an error.  So the learner will get three less than the total number of words -i.e. five out of eight.

In word-for-word scoring errors counted  could be of the following types:

Errors of deletions – some words are left out by the learners

Errors of distortions – of form or sequence – The wrong form of the word is written by the learner – ‘gets’ instead of ‘get’ – or  the word order is not proper

Errors of intrusion – Some words are added by the learner in the written passage, though it was not read out as part of the original passage.

Spelling errors can be distinguished as a special category.  Only those spelling mistakes, which distort the form of meaning of the word, can be counted as errors.

For example, if ‘tree’ is the word given in the passage, and if the learner writes ‘three’ or ‘try’ it will be counted as an error.  If the learner writes ’tre’ or ‘tiri’ it will not be counted as an error.

Like dictation, cloze procedures are also considered as pragmatic tests because they require temporally constrained sequential processing and pragmatic mapping of linguistic elements on extralinguistic contexts.