ESL EFL Writing Research

Studies up to the 60’s

The majority of the studies made prior to 1940 were concerned with problems of reliability in essay tests.  Huddleston (1921) presents a comprehensive and well  - organized review of the literature on the reliability of essay tests covering a period from 1900 to 1940.  This review points out the lack of any intensive and well-organised attack upon the problem of reliability up to that time.  J M Rico was one of the forerunners of the scientific movement in education, which during the early decades of the 20th century produced substantial accumulation of research studies on the statistical properties of all kinds.

Mainly these studies were of three kinds:

a.            Those related to the construction, use and scoring of objective tests;

b.            Those related to the construction, use and scoring of essay tests;

c.             Those contrasting the merits of objective and essay tests in measurement evaluation, placements and prediction.

These comparative studies of subjective and essay tests were meant for finding out which of the two types of tests is easy to score, more objective, and more valid in predicting learners achievement level.  The main problem of essay tests have the lack of reader reliability – the extent to which different individuals can agree on the score which should be assigned to a particular question.

Some of the findings of significant studies are as follows:

$        The experience or maturity of the reader can bring about variation in scores.  Skilled readers tend to mark on more abstract qualities such as ‘style’ and thought content.  So unskilled readers are more reliable than skilled readers.    (Vernon &Millican 1954)

$        Highly trained readers are more reliable than untrained ones.                  (Diederich Paul 1957)

$        Long responses are more reliable than short ones.                              (Vernon & Millican 1954)

$       The more precise and detailed the marking criteria, the higher the reliability.    (Sharp Lawrence 1925)

Page Sixteen

The greater the structuring of the question for the examinee, the higher the reliability.                   (Swinford   1956)

$               If there are optional questions from which the candidate has to select one, some topics are more reliable than others.          (Seinford, Francis & Vernon 1956)

$               Marks of papers drawn from homogeneous population are much less reliable than marks of those from heterogeneous population.       (Vernon, 1956)

$               Markers well acquainted with the work of the student are more reliable than those who do not know the students.                                       (Vernon, 1956)

If the research studies up to the 60’s were concerned with the reliability of essay tests, the later studies were concerned with the mechanical aspects of the written work which affect marking.