Testing the Writing Skills
12.1 THE WRITING SKILLS
The writing skills are complex and sometimes difficult to teach, requiring mastery not only of grammatical and rhetorical devices but also of conceptual and judgemental elements. The following analysis attempts to group the many and varied skills necessary for writing good prose into five general components or main areas.
– language use: the ability to write correct and appropriate sentences;
– mechanical skills: the ability to use correctly those conventions peculiar to the written language – e.g. punctuation, spelling;
– treatment of content: the ability to think creatively and develop thoughts, excluding all irrelevant information;
– stylistic skills: the ability to manipulate sentences and paragraphs, and use language effectively;
– judgment skills: the ability to write in an appropriate manner for a particular purpose with a particular audience in mind, together with an ability to select, organize and order relevant information.
The actual writing conventions which it is necessary for the student to master relate chiefly (at the elementary stages) to punctuation and spelling. However, in punctuation there are many areas in which personal judgments are required, and tests of punctuation must guard against being too rigid by recognizing that several answers may be correct. Unfortunately, tests of punctuation and spelling have often tended to inhibit writing and creativity.
The various kinds of register include colloquialism, slang, jargon, archaic words, legal language, standard English, business English, the language used by educated writers of English, etc. the purpose of writing will also help to establish a particular register: for example, is the student writing to entertain, inform, or explain?
A piece of continuous writing may take the form of a narrative, description survey, record, discussion, or argument. In addition to the subject and the format, the actual audience (e.g. an examiner, a teacher, a student, a friend) will also determine which of the various registers is to be used. Consequently, the use of appropriate register in writing implies an awareness not only of a writing goal but also of a particular audience.
12.2 TESTING POSITION WRITING
An attempt should be made to determine the types of writing tasks with which the students are confronted every day. Such tasks will probably be associated with the writing requirements imposed by the either subjects being studied at school if the medium of instruction is English. Short articles, instructions and accounts of experiments will probably form the main body of writing. If the medium of instruction is not English, the written work done in the classroom. In both cases, the students may be required to keep a diary, produce a magazine and to write both formal and informal letters. The concern of students following a profession or in business will be chiefly with report-writing and letter-writing, while at college or university level they will usually be required to write (technical) reports and papers.
One large public examining body explicitly states the kinds of writing tasks its examinations test and the standards of writing expected in the performance of these tasks:
A successful candidate will have passed an examination designed to test ability to produce a selection of the following types of writing:
Basic Level: Letter; postcard; Diary entry; forms
Intermediate Level: As Basic Level, plus Guide; sets of instructions
Advanced Level: AS Intermediate Level, plus Newspaper report; Notes
The candidate’s performance will have met the following minimum criteria:
Basic level: No confusing errors of grammar or vocabulary; a piece of writing legible and readily intelligible; able to produce simple unsophisticated sentences.
Intermediate Level: Accurate grammar, vocabulary and spelling, though possibly with some mistakes which do not destroy communication; handwriting generally legible; expression clear and appropriate, using a fair range of language; able to link themes and points coherently.
Advanced Level: Extremely high standards of grammar, vocabulary and spelling; easily legible handwriting; no obvious limitations or range of language candidate is able to use accurately and appropriately; ability to produce organized, coherent writing, displaying considerable sophistication.
In the construction of class tests, it is important for the test writer to find out how composition is tested in the first language. Although the emphasis in the teaching and testing of the skills in English as a foreign/second language will of necessity be quite different to the development of the skills in the first language a comparison of the abilities required and methods used is very helpful. It is clearly ludicrous, for instance, to expect in a foreign language those skills which the students do not possess in their own language.
In the composition test the students should be presented with a clearly defined problem which motivates them to write. The writing task should be such that it ensures they have something to say and a purpose for saying it. They should also have an audience in mind when they write. How often in real-life situations do people begin to write when they have nothing to write, no purpose in writing and no audience in mind? Thus, whenever possible, meaningful situations should be given in composition tests. For example, a brief description of a real-life situation might be given when requiring the students to write a letter:
Your pen-friend is going to visit your country for a few weeks with her two brothers. Your house is big enough for her to stay with you but there is not enough room for her brothers. There are two hotels near your house but they are very expensive. The third hotel is cheaper, but it is at least five miles away. Write a letter to your pen-friend, explaining the situation.
Composition titles which give the students no guidance as to what is expected of them should be avoided. Examples of poor titles which fail to direct the students ideas are A pleasant evening, My best friend, Look before you leap, A food film which I have recently seen.
Tasks requiring the students to act the part of another person are often avoided as it is felt they are less realistic and communicative. However, this is usually far from being the ease. It is useful to provide the students not only with details about a specific situation but also with details about the particular person they are supposed to be and the people about (or to) whom they are writing, compare, for example, the two following tasks:
(a) Write a letter, telling a friend about any interesting school excursion on which you have been.
(b) You have just been on a school excursion to a nearby seaside town. However, you were not taken to the beach and you had no free time at all to wander round the town. you are very keen on swimming and you also enjoy going to the cinema. Your teacher often tells you that you visited the law courts, an art gallery and a big museum. It was all very boring apart from one room in the museum containing old-fashioned armour and scenes of battles. You found this room far more interesting than you thought it would be but you didn’t talk to your friends or teacher about it. In fact, you were so interested in that you left a small camera there. Your teacher told you off because you have a reputation for forgetting things. Only your cousin seems to understand you. Write a letter to him, telling him about the excursion.
Although the former task is one which students may conceivably have to perform in real life, the latter task will result in for more realistic and natural letters from the students simply because the specific details make the task more meaningful and purposeful. The detailed description of both the situation and the person involved helps the students to suspend their disbelief and gives the task an immediacy and realism which are essential for its successful completion.
12.3 SETTING THE POSITION
In addition to providing the necessary stimulus and information required for writing, a good topic for a composition determines the register and style to be used in the writing task by presenting the students with a specific situation and context in which to write. Since it is easier to compare different performances when the writing task is determined more exactly, it is possible to obtain a greater degree of reliability in the scoring of compositions based on specific situations. Furthermore, such composition tests have an excellent backwash effect on the teaching and learning preparatory to the examination.
The following are provided as examples of situational compositions intended to be used in tests of writing:
Type 1
Imagine that this is your diary showing some of your activities on certain days. first, fill in your activities for those days which have been left blank. Then, using the information in the diary, write a letter to a friend telling him or her how you are spending your time. write about 100 words. The address is not necessary.
1 Monday
|
Study!
|
2 Tuesday
|
Study!
|
3 Wednesday
|
Final Exams
|
4 Thursday
|
|
5 Friday
|
|
6 Saturday
|
Shopping, Driving lesson 2 pm.
|
7 Sunday
|
Only two more weeks to wait!
|
Type 2
(Question 1)
While you are away from home, some American friends are coming to stay in your house. You are leaving before they are due to arrive, so you decide to leave them some notes to help them with all the things they will need to know while staying in the house. Your friends have never been to your country before so there is quire a lot of advice you need to pass on. Write your message on the notelet pad sheet below.
(Question 2)
While your American friends are staying in your house, they write to say that they are enjoying themselves so much that they would like to spend two weeks visiting some other parts of the country. They would like your advice about what to go and see and where to stay. Write to your friends giving the best possible, advice you can from your own knowledge and experience, with whatever special hints and warning may be necessary. Make sure your friends know who they can write to for further information of an ’official’ kind to help them to plan the best possible holiday.
Write your letter in the space below. It should be between 150 and 200 words in length.
(A blank space follows.)
Type 3
Read the following letter carefully.
176 Wood Lane
London NW2
15TH May
Dear Mr John,
I wish to complain about the noise which has come from your home late every night this week. While I realize that you must practice your trumpet some time, I feel you ought to do it at a more suitable time. Ten o’clock in the evening is rather late to start playing. Even if you could play well, the noise would still be unbearable at that time.
I hope that in future you will be a little more considerate of the feelings of others.
Yours sincerely,
W. Robinson
Now write a reply to this letter. You do not play the trumpet but on two or three occasions recently you have played some trumpet music on your record player. You did not play the record very loudly – certainly not as loudly as Mr Robinson’s television. You want to tell him this but you do not want to become enemies so you must be reasonably polite in your letter.
Care must be taken in the construction of letter-writing tasks to limit the amount of information to which the student must reply. If this is not done scoring can become extremely difficult.
Type 4 A dialogue can be very useful in providing a basis for composition work. In such a writing task, students must demonstrate their ability to change a text from one register to another, as in the following example:
Read the following conversation carefully.
Mr Black: What was the weather like while you were camping?
Linda: Not too bad. It rained the last couple of days, but mostly it was fine. We weren’t able to visit the George Waterfalls on the next to the last day, but …
Mr Black: What a pity!
Linda: Well, a part from that we did everything we wanted to – walking, climbing and just sitting in the sun. We even managed a visit to Hook Cave.
Mr Black: How on earth did you get that far?
Linda: We cycled. Oh …and we went to the beach quite a few times.
Mr Black: Did you take your bikes with you?
Linda: No, we borrowed some from a place in the village.
Mr Black: Whereabouts were you?
Linda: Oh, in a lovely valley – lots of woods and about twenty miles from the sea. just north of Hilson.
Mr Black: I remember one time when I went camping. We forgot to take a tin-opener!
Linda: That’s nothing. A goat came into our tent in the middle of the night – it ate all the food we had with us!
Mr Black: Well, you seem to have had a good time.
Now write an account of Linda’s holiday, using the conversation above as a guide. Imagine other things which happened to her during the camping holiday.
Type 5 Tables containing information are also useful for situational composition since they can generally be read by the students without much difficulty. Moreover, as only a short written text is used, the students are thus not encouraged to reproduce part of the rubric for use in their composition.
Imagine that a local newspaper has asked you to write an article of approximately 250 words about the information in the following table. Write down the conclusions you draw from the figures about the various ways in which people spent their holidays in 1968 as compared with 1988. attempt to explain the reasons for these differences.
|
1968
|
1988
|
Traveling abroad
Going to seaside
Camping
Visiting friends/relatives in another town
Going to another town (but not to visit friends/relatives)
Staying at home
|
4
38
8
11
16
23
|
17
31
31
10
3
8
|
TOTAL
|
100
|
100
|
12.4 GRADING THE COMPOSITION
Because of the inherent unreliability in composition marking, it is essential to compile a banding system – or, at least, a brief description of the various grades of achievement expected to be attained by the class. The following are two examples of descriptions of levels of performance used by a well-known examining body in Britain: table (a) for intermediate-level learners and table (b) for more advanced-level learners.
As with the scoring or oral production, banding systems devised for a particular group of students at a particular level are far preferable to scales drawn up for proficiency tests administered on a national or an international basis.
Table (a)
18 – 20 Excellent Natural English, minimal errors, complete realization of the task set.
|
16 – 17 Very good Good vocabulary and structure, above the simple sentence
level. Errors non-basic.
|
12 – 15 Good Simple but accurate realization of task. Sufficient
naturalness, not many errors.
|
8 – 11 Pass Reasonably correct if awkward or Natural treatment of
subject with some serious errors.
|
5 – 7 Weak Vocabulary and grammar inadequate for the task set.
|
0 – 4 Very poor Incoherent. Errors showing lack of basic knowledge of
English.
|
Table (b)
18 – 20 Excellent Error-free, substantial and varied material, resourceful and controlled in language and expression.
|
16 – 17 Very good Good realization of task, ambitious and natural in style.
|
12 – 15 Good Sufficient assurance and freedom from basic error to maintain theme.
|
8 – 11 Pass Clear relation of task, reasonably correct and natural.
|
5 – 7 Weak Near to pass level in general scope, but with either
numerous errors or too elementary or translated in style.
|
0 – 4 Very poor Basic errors, narrowness of vocabulary.
|
Composition may be scored according to one of two methods: the impression method or the analytic method. Note, however, that the former method does not involve the use of a rating scale to any large extent.
The impression method of marking entails one or more markers awarding a single mark (= multiple marking), based on the total impression of the composition to appeal to a certain reader but not to another, it is largely a matter of luck whether or not a single examiner likes a particular script. As has been demonstrated, the examiner’s mark is a highly subjective one based on a fallible judgment, affected by fatigue, carelessness, prejudice, etc. However, if assessment is based on several (fallible) judgments the net result is far more reliable than a mark based on a single judgment.
Impression marks must be based on impression only, and the whole object is defeated if examiners start to reconsider marks and analyze compositions. Most examiners find it more enjoyable than any other method of scoring compositions.
Since most teachers have little opportunity to enlist the services of two or three colleagues in marking class compositions, the analytic method is recommended for such purposes. This method depends on a marking scheme which has been carefully drawn up by the examiner or body of examiners. It consists of an attempt to separate the various features of a composition for scoring purposes. Such a procedure is ideally suited to the classroom situation; because certain features have been graded separately students are able to see how their particular grade has been obtained, the following is reproduced simply as one example of such an analytic scheme; in this particular ease duplicate (blank) copies of this scheme were stenciled by the teacher and attached to the end of each composition.
Grammar
|
5
|
4
|
3
|
2
|
1
|
||
|
|
X
|
|
|
|||
Vocabulary
|
|
|
|
X
|
|
||
Mechanics
|
|
X
|
|
|
|
||
Fluency
|
|
|
|
X
|
|
||
Relevance
|
|
|
X
|
|
|
Note that Mechanics refers to punctuation and spelling: Fluency to style and ease of communication; and Relevance to the content in relation to the task demanded to the student. A 5-point scale has been used.
If the analytic method of scoring is employed, it is essential that flexibility is maintained. At the various levels it may become necessary to change either the divisions themselves or the weighting given to them. At the elementary level, for example, the tester may be far more interested in grammar and vocabulary than fluency, thus deciding to omit Fluency. At the intermediate level, the tester may be particularly interested in relevance and may, therefore, decide to award a maximum of 10 marks for this feature while awarding only 5 marks for each of the others. At the more advanced level, the tester may wish to include separate divisions for organization and register and to include mechanics and fluency in one division.
A third method of scoring compositions is the mechanical accuracy or error-count method. Although this is the most objective of all methods of scoring. It is the least valid and is not recommended. The procedure consists of counting the errors made by each testee and deducing the number from a given total: for example, a testee may lose up to 10 marks for grammatical errors, 5 marks for misuse of words, 5 for misspellings, etc. Since no decision can be reached about the relative importance of most errors, the whole scheme is actually highly subjective. For example, should errors of tense be regarded as more important than certain misspellings or the wrong use of words? Furthermore, as a result of intuition and experience, it is fairly common for an examiner to feel that a composition is worth several marks more or less than the score he or she has awarded and to alter the assessment accordingly.
12.5 TREATMENT OF WRITTEN ERRORS
An important distinction is now made between global and local errors. Those errors which cause only minor trouble and confusion in a particular clause or sentence without hindering the reader’s comprehension of the sentence are categorized as local errors (e.g. misuse of articles, omission of prepositions, lack of agreement between subject and verb, incorrect position of adverbs, etc.: ‘I arrived Leeds.’). global errors are usually those errors which involve the overall structure of a sentence and result in misunderstanding or even failure to understand the message which is being conveyed (e.g. the misuse of connectives: ‘Although the train arrived late, we missed the last bus to the city centre’; the omission, misuse and unnecessary insertion of relative pronouns: ’you should try to be as healthy as the girl arrived on the bicycle a short time ago’; etc.). this useful distinction, which provides criteria for determining the communicative importance of errors, has been further developed recently so that it can be more readily applied to the marking of free writing.
In most normal writing situations, however, we can only assess what a student writes and not what he or she wants to write. For this reason, pictures and diagrams can play a very useful part in testing writing, since they enable the examiner to tell immediately what a student wishes to write. Pictures were recently used by researchers in an experiment to show how L learners (i.e. less fluent learners) used avoidance strategies or reduction strategies, avoiding an actual topic. L learners (and possibly more fluent L learners) tended to use paraphrase strategies or achievement strategies. Fluent performance seems to be characterized by the use of fewer communication strategies of both kinds.
12.6 OBJECTIVE TESTS
Mechanics & punctuation
Type 1 The following type of punctuation item is very popular and is generally used to cover a wide range of punctuation marks. It is not truly objective, and the scoring of such an exercise would take considerable time since punctuation is to a large degree subjective and one particular use of a punctuation mark well determine the correctness of the punctuation mark following it.
In the following passage there is no punctuation. Write out the passage, putting in all the punctuation and capital letters.
Lend me your pen please peter asked
i took my pen out of my pocket
Be careful i said
ill give it back to you in a moment he promised
Dont worry i said you can keep it as long as you want
It is advisable, however, to maintain some degree of control over the task which the testees are expected to perform. One method of doing this is by substituting lines or circles for those punctuation marks which are being tested, thus also facilitating scoring.
Type 2
Put the correct punctuation mark in each square bracket [ ].
[ ] What do you want, [ ] I asked Henry[ ]
[ ] May I use your telephone? [ ] he asked.
[ ] Certainly [ ] [ ] I said. [ ] When you [ ] ve finished [ ] please let me know [ ] [ ]
[ ] I shall only be a moment [ ] [ ] Henry answered.
[ ] Has John Lee invited you to his party [ ] [ ] I asked.
[ ] No, he hasn [ ] t yet [ ] [ ] Henry replied.
[ ] [ ] He [ ]s invited Paul [ ] David [ ] Tony and Mary [ ] [ ] I continued.
[ ] He [ ]s probably forgotten about me [ ] [ ] Henry laughed.
[ ] How strange [ ] [ ] I answered, [ ] I’m sure he wants you to go to this party.[ ]
Type 3 A greater degree of objectivity can be obtained by using the multiple-choice technique, e.g.
Put the circle round the letter (A,B,C, or D) of the correctly punctuate sentence:
- Tom asked me if I was going to the meeting?
- Tom asked me, if I was going to the meeting.
- Tom asked me, if I was going to the meeting?
- Tom asked me if I was going to the meeting.
Spelling
Type 1: Dictation
Dictation of long prose passages is still regarded as an essential method of testing spelling. However, dictation measures a complex range of integrated skills and should not be regarded as constituting simply a test of spelling. The dictation of single words, nevertheless, can prove a fairly reliable test of spelling. Several such tests consists of up to fifty words and use similar procedures to the following:
(i) Each word is dictated once by the tester;
(ii) The word is then repeated in a context; and finally,
(iii) The word is repeated on its own.
Type 2: Multiple-choice items
Another fairly widespread method of testing spelling is through the use of multiple-choice items usually containing five options, four of which are spelt correctly. The students are required to select the word is incorrectly spelt, e.g.
1. A. thief B. belief C. seize D. ceiling E. deceive
2. A. happening B. offering C. occurring D. beginning E. benefiting
3. A. illegal B. generally S. summary D. beggar E. necessary
4. A. interrupt B. support C. answering D. occasional E. command
In some tests only four words are given as options, the fifth option being No mistakes or All correct, e.g.
A. exhibition B. punctually C. pleasure D. obayed E. All correct
12.7 CONTROLLED WRITING
Type 1 The students are given a short reading extract and then required to write a similar paragraph, using the notes they have been given, e.g.
Although dogs are only animals, they are very useful and help people a lot. For example, certain dogs help farmers to look after their sheep. Some dogs are used for hunting and others help to rescue people. Even now police officers use dogs when they are looking for thieves and criminals. People also teach dogs to race, and dog racing is a sport which many people like. All dogs like eating meat very much and like bones best of all.
Although – horses – animals – useful – a lot, for example, – horses – people – cattle -. Some horses – hunting – pull things. In the pest – soldiers – horses – fighting against the enemy, people – horses – horse racing – sport – like. All horses – hay – oats.
Type 2 The following item type is set in a few widely-used examinations and can prove very useful in controlling writing once students are familiar with the conventions observed in the item. Even the following rubric and item may cause difficulty if a student has not previously been given practice in completing such items. Oblique strokes are used in the fragmented sentences chiefly in order to reinforce the impression that the sentences have been given in note form.
Use the following notes to write complete sentences. Pay careful attention to the verbs underlined and insert all missing words. The oblique lines{/} are used to divide the notes into sections. Words may or may not be missing in each of these sections. Read the example carefully before you start.
Example: Parachute jump from aeroplanes and balloons/be very popular sport/many parts of world.
Parachute jumping from aeroplanes and balloons is a very popular sport in many parts of the world.
Greatest height/from which parachute jump ever make/be over 31,000 metres./1960 doctor in United States Air Force/jump from basket of balloons /and fall nearly 26,000 meters/before open parachute/Fall last 4 minute and 38 seconds,/and body reach/speed 980 kilometers hour./He land safely in field/13 minutes and 45 second/after he jump./On step of basket of balloon/be words,/’This be highest step in world.’/When ask/if he like jump again/from such height/doctor shake head.
Type 3(a) In some tests of composition, especially at the elementary and intermediate levels, sentences and clauses are provided at first in order to help the students to start writing. They are then required to finish the incomplete sentences in any appropriate way.
Read these sentences. Finish each one and then complete the story in your own words.
One day Hannah and Becky got up early to go …………………………….
They caught a bus to the large department store where …………………
‘Look, that’s Pete Shaw over there,’ Becky cried.
‘Let’s ……………………………………..
they shouted to Pete but …………………..
‘Why doesn’t he look at us/’ Hannah asked. He’s behaving as if …………………
(b) This item is similar to the previous one, but here the testees are required to write appropriate sentences rather than clauses. The following example shows how the item type can be used at the upper-intermediate levels.
1. Students who do not know a lot of English can take several steps to prepare for their study in British university. For example,
………………………………………………………………………………………
……………………………………………………………………………………..
2. Recent research shows that public opinion is divided on the subject of spending money on defense. About 40 per cent of the country believes we should increase such spending. On the other hand.
……………………………………………………………………………………..
……………………………………………………………………………………..
A well-selected series of such items can include sentences eliciting an ability to use exemplification, contrast, addition, cause, result, purpose, conclusion and summary. Consequently, students can be tested on their ability to use whatever specific functions and notions the test writer wishes.
Categories: