Ways of Testing a Translation & Testing and Evaluation in the Translation Classroom

Ways of Testing a Translation & Testing and Evaluation in the Translation Classroom

By Tayebeh Sadat Hosseini,
Iran, Isfahan University, English Department,
M.A. Student of English Translation

Get the List of 4,700+ Translation Agencies Now!
No Recurring Membership Fees!

Table of Content

1. Abstract
2. Ways of Testing a Translation
3. Using the Testing Results
4. Testing and Evaluation in the Translation Classroom
5. Key Terms
6. Types of Assessment
7. Test Items
8. Assessment and Grading / Marking
9. Case Studies of Tests for Translation Courses
10. Conclusion
11. References

Abstract

In this article, first different ways of testing a translation is studied. As it will be mentioned, there are five ways to test a translation which consist of comparison with the source language, back – translation, comprehension checks, naturalness and readability testing, and consistency checks. After that, testing and evaluation in the translation classroom are analyzed. At the beginning, the meaning of some key terms in testing is presented. Then, types of assessment are evaluated. After that, different types of test items are considered. Next, there exist assessment and grading or marking of the students’ translation in the classroom; and two kinds of assessments, that is norm – referenced and criteria – referenced, are examined. Finally, some information is given about testing and evaluation in an academic atmosphere as well as case studies of tests for translation courses.

Ways of testing a translation

According to Larson, There exist five ways to test a translation:

1. Comparison with the source text

2. Back – translation into the source language

3. Comprehension checks

4. Naturalness and readability testing

5. Consistency checks

Comparison with the source language

One of the main purposes of the comparison is to check for equivalence of information content. The comparison is actually a self – check; that is, it is done by the translator. Of course, it could be done by someone else who knows both languages well and knows translation principles. After checking to be sure that all of the information is there, the translator will make another comparison of source language and receptor language texts, looking for any problems.

Back – translation

A second way to check a translation is by having someone else, who is bilingual in the source and receptor languages, make a back – translation of the translated text into the source language. This person takes the translation and writes out the meaning he gets from it back into source language. He should do it without having read the source text used by the translator. This back translation will let the translator know what is being communicated to this person. In translating, one uses natural and clear forms; in back – translating, literal forms are used in order to show up the structure of the translation being back – translated.

Comprehension tests

Good comprehension testing is the key to a good translation. The purpose of this test is to see whether or not the translation is understood correctly by speakers of the language who have not seen the translation previously.

It is designed to find out what the translation is communicating to the audience for whom it is intended. This type of test involves having people retell the content of the translation and answer questions about it.

Comprehension testing is done with persons who are fluent speakers of the receptor language. These people should be ordinary people from various classes of the society. Testing should be done with young people, middle aged, and older people. It should be done with the more highly educated and with the newly literate, if the translation is intended for all.

The respondent is asked to retell or give a summary of the material read.

The tester should be careful to choose a section which is a unit and which is not so long that it would be hard to remember the content.

The second step in comprehension testing is asking questions about the translated text. The questions should be prepared beforehand, not made up on the spot. This gives the tester time to think through what he expects the respondent to understand and to decide exactly what he wants to check.

There are several kinds of questions, each with a different purpose.

Questions may be asked to give information about the discourse style, or about the theme of the text, or they may be questions which have to do with details; Style questions are related to the genre and the style of the translation. Questions may also be asked which are designed to evaluate the translation of the theme. These questions focus on the high points of the story or argument. The purpose of these theme questions is to determine if the main points of meaning are clear in the translation. The theme questions are used to begin the discussion.

Detail questions are questions about words, phrases, and other matters which the tester does not want to ask while he is concentrating on the main points of the text. Detail questions often lead to complicated discussions and would cause the respondent to lose track of main points, if asked while discussing genre or theme. The answers to most detailed questions should be clearly found in the text unless one is checking to see if implicit information is retrievable.

Naturalness tests

The purpose of naturalness tests is to see if the form of the translation is natural and the style appropriate. This testing is done by reviewers.

Reviewers are people who are willing to spend time reading through the translation making comments and suggestions. However, most reviewers simply read the translation looking for ways to improve the naturalness and style.

Reviewers need to know enough about translation principles to understand what is meant by an idiomatic translation. They can probably best be trained by having a consultant or translator work through a number of texts with them. All reviewers should be looking for ways to improve, the clarity, naturalness, flow of the discourse, and the emotive impact on the readers.

The process used by the reviewer is first to read through the whole section of the translation at one time. This is important for checking the flow of the translation and the overall meaning of the text. He should write notes either in the margin or on a separate paper to give to the translator.

After the reviewer has checked for clarity and naturalness, he may also check for accuracy, if he knows the source language well. He will compare the translation with the source text looking for omissions, additions, or any changes of meaning. Once again the reviewer should make careful notes for the translator.

Readability tests

The translator and tester may do readability tests. These tests are done by asking someone to read a part of the translation aloud. It should be a complete section; that is, a unit. As they read, the tester will notice any places where the reader hesitates. Also, if he stops and re –reads the sentence, this should be noted as it indicates some problem in readability.

Readability tests do not need to be done in formal sessions only. At any time that someone is reading the translation, the translator, testers, and reviewers who are listening should be aware of any difficulties in reading. A text is readable because it is good writing, that is, it has a pleasing style, a good rhythm, and moves along at an acceptable pace. It should be kept in mind that what is readable for one audience may not be readable for another. A highly educated audience will easily read rather complex sentence structure. This is why it is important that the readability tests be done with persons who will be the users of the translation.

Readability may also be affected by formatting matters. The size of type, punctuation, spelling, size of margins and space between lines may all affect the readability tests.

Consistency checks

As the translation comes near to completion, it is very important that consistency checks of various kinds be made. Some of these have to do with the content of the translation and others have to do with the technical details of presentation. All of those who are testing the translation should be alert for reading problems related to formatting as well as content.

The source text will have had certain key terms which were identified and for which lexical equivalents were found. If the document being translated is a long one, or done over a long period of time, it is possible that the translator has been inconsistent in the use of lexical equivalents for some key terms. At the end of the translation project, a check should be done of such terms. This will be especially true in technical, political, or religious documents.

In the final review, the formatting of the text and of any supplementary material like footnotes, glossary, and index or table of contents, should also be checked for formatting style.

Using the testing results

After all of the tests have been carried out, the results will need to be evaluated and recommended changes accepted or rejected or modified in some way.

After the initial draft is completed, it will be very helpful to the translator if he himself does some readability checks and comprehension checks with various people. The reworking of the initial draft results in the second draft. This draft is then tested by a careful comparison with the source text. A back – translation is prepared which the translator will use for a self – check and for working with a consultant. Comprehension checks, naturalness checks and readability checks are also made.

A third draft, the revision draft is then made by the translator incorporating the information into the draft. Once the revised draft is completed, some consistency checking may need to be done again. Additional readability testing may be done. It may even be wise to do some more comprehension testing or reviewing, especially on parts of the translation on which there was disagreement among the members of the team. The final draft will there was disagreement among the member of the team. The final draft will then need to be checked very carefully for consistency in technical matters and proofread a number of times. If a number of people can read through it completely, this will give the best check of the final draft.

Testing and Evaluation in the Translation Classroom

According to Dr. Carol Ann Goff – kfouri, one particular problematic area is that of marking translations and making decisions on student competence. One of the most challenging terms for professional educators is 'test '. Even experienced instructors may not always feel at ease with putting a grade or a mark on a student’s final paper. Instructors and curriculum designers today seem to be convinced that a more learner – centered, creative and flexible teaching system motivates students. Instructors who emphasize a communicative type of testing may promote a more efficient learning environment. Instructors of translation need to become competent in test writing, but they must keep in, mind that there is no perfect test and no foolproof grading or marking system.

Key Terms

Measurement is a Process that attempts to obtain a quantitative representation of the degree to which a student shows competence in a particular skill or area of knowledge. In order to measure, instructors must have an instrument. The instrument an instructor uses to measure a student’s competence has traditionally been the test. A test (oral or written) is made up of items.

Evaluation is also a process; it is the systematic process of determining the extent to which students reach the educational objectives set by the institution or standard – setting body that issues their diploma. Evaluation is part of a decision – making process; and it depends on the reliability of the test instrument.

Reliability refers to the test’s consistency. If the same test were administered a second time under equivalent conditions, the same results should occur. A test of technical translation ability may render more reliable results than a literary translation test.

Validity reflects whether the test measures what it was supposed to measure. For example, if students are asked to write an essay in a language class on the latest methods of imputing data into a database, and those students are not knowledgeable on that particular subject, that test will not be a valid judge of their language abilities.

Types of Assessment

Translation students will take a number of tests during their time in university.

A placement test is generally the first test a student translator will sit for at university. The purpose of the placement test is to classify the level of incoming candidates to a translation or any other skill – based program. According to the results, the department may have to implement remedial or intensive courses. Placement tests are a practical way to assess the evolution in incoming students talents from one year to the next.

Candidates to a translation or any other skill – based program. According to the results, the department may have to implement remedial or intensive courses. Placement tests are a practical way to assess the evolution in incoming students talents from one year to the next.

Diagnostic tests are tests designed to pick out student problems before it is too late in the year or the semester to do so. A diagnostic test is given so as to facilitate the student’s learning, to encourage students to correct areas of weakness. Some progress tests may also serve a diagnostic function.

Progress tests are the most frequent tests instructors give. The objective of a progress test is to determine if the students have mastered material that has already been taught. Progress tests are most often “open book “in translation classes; and students have access to notes, databases, dictionaries, etc. Quizzes, graded homework, short projects, weekly or bi – weekly tests are all types of progress tests.

Achievement tests are meant to determine if the student has met the course objectives. If students were placed in the correct course level, benefited from the results of diagnostic tests and progress tests, the achievement test should reaffirm their acquisition of skills necessary to advance to a further level of study. Their results should be examined closely so as to evaluate the program’s strengths and weaknesses.

There are two further traditional types of tests: formative and summative.

Formative assessment takes place during the instruction period and is designed to guide instructors to object their teaching, if need be. Progress tests also fall into this category, as do diagnostic tests. Feedback from formative assessment must be communicated to the student as soon as possible. Students react more positively to formative assessment if the results are analyzed by the instructor and the teaching style or class content is altered if need be. This is called the washback effect. Formative assessment is the ongoing process instructors and students use to gauge the success of the syllabus and to prepare for the second type of assessment, the summative.

Summative assessment contrasts with formative assessment first of all by its purpose. The purpose of summative assessment is to attribute value, and for that reason it is often more quantitative than the qualitative formative assessment. It also occurs at the midpoint and/or end of instruction so as to determine the extent to which syllabus objectives have been met. Achievement tests, final exams, oral or written, and research projects are examples of summative assessment. Grades or marks from summative assessment often provide a basis for passing a student or for repeating a class.

Process assessment is a relatively new assessment technique that is more formative than summative. It works most efficiently with long – term projects and is particularly applicable to higher – level translation studies. An instructor sets process assessment in place by first setting benchmarks the student must attain.

Portfolio assessment is also a relatively new technique to aid students in tracking their progress. Not only do the students track their own level but also the instructor is able to judge the student’s work in reference to past assignments. A portfolio is a file that students compile throughout the semester or course and in which they choose the work they have done and want to be marked for a final grade. Instructors can determine the minimum number of assignments per week, or each two weeks, to be included in the portfolio.

Test Items

Translation instructors need not depend only on a text as a basic test item In order to assess in a formative or summative manner, instructors have a wide range of item formats to choose from. The basic types of item format are objective and subjective. In a simple format objective test, the items may be supply, true – false or alternative response, or matching. Multiple – choice and interpretive items are more complex forms of objective tests. Essay tests and their derivatives form the basis of subjective exams.

Supply or free – response items

Unstructured short answer and fill in items are the main types of free response test questions. They are used primarily in informal testing. The great advantage to these items is their ease of preparation and correction.

The Two – Alternative Items

More commonly known as yes / no, true / false, such items measure how well students know facts and definitions, and if they can distinguish between fact and opinion.

Multiple Choice Items

Multiple choice items can be used to measure a variety of learning objectives such as vocabulary acquisition, analysis, application of principles, cause and effect association for the ability to interpret data. Actually, multiple – choice tests have more advantages than disadvantages. First, an instructor can build an item bank and alter particularly effective questions and use them more than once. Also, multiple – choice statements offer the instructor one means of being creative in the testing of translation skills.

If you want to experiment with constructing multiple – choice questions, the following guidelines are valuable:

1.   The stem of the question should be meaningful in itself.

2.   State the stem of the question in positive terms.

3.   Write all alternate answers in parallel form.

Dictation and Dicto – comp

Dictation is quite useful in a translation classroom to test the receptive skills of listening and recognition and use of terminology. After students have documented a text to be translated or read parallel texts, they can benefit from dictation taken from one or more of the texts. Students of interpreting skills benefit from dictations because the instructor can vary the speed of delivery, and can ask colleagues to deliver a dictation so students become accustomed to various accents.

As a means of creative dictation, you may use a small portion of the text students are to translate and dictate one section as you see it in the target language. Marking dictation is very straightforward. Inform students in advance of the criteria you are using. Beginning with 10 points or 20 if the text is longer, take off one mark for every error.

Language instructors have been using Dicto – comp as means as a means to test student ability to remember main ideas of a text in chronological or logical order and as a test of comprehension. Translation and interpreting instructors can use dicto – comp in both the L1 and the L2 of the student. It can be used after the students have prepared documentation for their translation but have not yet written the translation. The following is a simple methodology to try dicto – comp.

1. Read the text to the students several times. Students listen with pens down.

2. Then ask the students to write what they remember in a logical order staying as close to the original as possible. To correct the dicto – comp, provide students with the 5 , 10 , or 15 main ideas in the order of the original.

What is practical about this test is that translation students are initiated into the idea of translation units and can then move on to consecutive interpreting with greater ease. This type of test is particularly appropriate for instructors of consecutive interpreting.

Assessment and Grading / Marking

When instructors mark exams they usually do so based on one of two traditional options available. Norm – referencing assessment judges one student’s performance based on the rest of the students in their group. The group is the norm. Students will be informed if they fall in the top or bottom third of the class, for example. In competitive testing situations, a norm – referencing assessment is used; and the candidates are competing against each other.

Criteria – referenced assessment involves evaluating whether the student can perform a task or not; instructors are not concerned with the comparison among students. In translation classes, criteria – referenced tests are more frequent. Students are judged on how well they alone can perform a task. For example, can they complete a technical translation within a fixed time period? In theory, all of the students may be able to do so.

Ipsative referencing compares a student’s present performance with a pervious one. Generally considered effective in special needs education and performance coaching, it may be beneficial in translation classes as it enables students to judge how much they have progressed within a fixed period of time.

Instructor Assessment

There have been many suggestions made as to how to mark a translation. Certainly the type of translation whether technical or literary plays a crucial role in the type of correction you choose. The corrector also plays an important role. Some emphasize certain criteria above others. Students in a classroom must be informed of the criteria you are judging.

Self – Assessment

Translation students are adults who have chosen to pursue a career in language services. The majority knows that competition is quite stiff and in order to succeed they must be superior to others. Asking students to assess their own progress is one way of initiating them to see their work objectively.

Peer Assessment

Students are effective revisers and evaluators of each other’s work. They are even more effective when they help decide on the criteria for the assignment undertaken. In fact peer assessment is an extremely useful learning experience.

Testing and Evaluation in an Academic Atmosphere

Remember that testing the class is as much a reflection of teaching as it is of the students’ knowledge. A test may evaluate the effectiveness of the instruction. Teaching should be in a way that prepares students to apply what they have learned in any situation, test or normal class work.

As in the case of many university courses presently, if you are teaching with a team of teachers in what is called a "multi – section" course and are called upon to write a common exam for your students as well as the other instructors’ students, remember the following:

1. Contribute items that have not been covered on your own class quizzes, this is not a fair evaluation of your students in comparison to the others.

2. Consult with the other instructors in advance as to what is to be covered on the exam.

3. Set up a common grading scale as well as the common exam.

4. Meet and exchange papers to make sure grading is consistent. For example, ask that all your colleagues bring three papers for discussion: the highest, the average and the lowest grades. Exchange the papers and discuss objectively.

5. You may even experiment with exchanging entire class sets of papers for truly objective grading.

Case Studies of Tests for Translation Courses

It is tempting to give a text and simply request that it be translated. If the objective in testing is to evaluate the overall ability of the student then this is an appropriate method. However, instructors may wish to test specific skills.

The hardest part of writing a test is deciding how much material can be tested within a certain time frame. When you carry out activities in class, gauge the amount of time your class needs to complete the work.

One way to test basic knowledge on a theme is to give students terminology in the source and their equivalences in the disorder. Students are then allowed 4 minutes, or more (or less) depending on the length of the list to find the correct match.

In order to test the student’s ability to apply the terminology, you may give the students sentences that must be translated within a certain time limit.

For a higher – level course, provide two translations of the same text, or part of one, and ask students to evaluate the strengths and weaknesses of each.

Testing will probably never be the high point of a teaching experience, but we can try to make our tests as creative as possible so that students learn both from their time in our classes and our testing sessions.

Conclusion

There are five ways to test a translation:

1. Comparison with the source language

2. Back – translation into source language

3. Comprehension tests

4. Naturalness and readability tests

5. Consistency checks

For testing and evaluation in the translation classroom, knowing the meaning of measurement, evaluation, reliability, and validity is of great importance. There exist different types of assessment that translation students will take during their time in university: a placement test, diagnostic tests, progress tests, and achievement tests. There are two further traditional types of tests: formative and summative.

In the section of test items, there are supply or free – response items, the two alternative items, multiple choice items, dictation and dicto – comp. Assessment and grading are based on norm – referenced assessment and criteria – referenced assessment. Norm – referenced assessment judges one student’s performance based on the rest of the students in their group. It shows how the candidates are competing against each other. Criteria – referenced assessment involves evaluating whether the student can perform a test or not; so the instructors are not concerned with the comparison among students. Also the instructor assessment, self – assessment, and peer assessment are practical and useful in translation classroom. Moreover, testing and evaluation in an academic atmosphere, and case studies of tests for translation courses are two matters which should not be neglected in the translation classroom.

References

1. Larson, Mildred L. ( 1984 ) . Meaning – based translation. University press of America.

2. Kunnan, Antony John: (2000). Studies in language testing. California State University, Los Angeles.

3. Farhady, H., Jafarpoor, A., Birjandi, p., (1995) Testing language skills from theory to practice. Tehran: SAMT.

4. Heaton, J. B. (1990). Classroom testing. Longman, New York.

5. Google site.

6. Baker, M. (ED.). (1998). Encyclopedia of translation studies. London: Routledge.

7. House, J. (197/1977). A model for translation quality assessment. Tubingen: Gunter Narr.

8. Venuti, L. (2000). The translation studies reader. London & New York: Routledge.

9.Widdowson, H.G. (1973). Teaching language as communication.London: Oxford University Press.

10. Jaaskelainen, R., (2005). Translation studies: What are they? Retrieved November

11. 2006 from http://www.hum.expertise.workshop.In Ordoudary, M. (2007).M.A. thesis.

Published - September 2008