On the Test Methods and Translation Criteria Used to Assess Iranian Students’ Translations in Translation Courses

By Abdolmehdi Riazi, Ph. D.
and
Zahra Davoodi, M.A. in translation studies,

Department of Foreign Languages & Linguistics,
Shiraz University, Shiraz, Iran

Get the List of 4,500+ Translation Agencies Now!
No Recurring Membership Fees!

Abstract

One of the main reasons for students’ dissatisfaction with their exam scores in translation programs is related to the way their translations are evaluated. Although translation courses have been taught for several years in Iranian universities (both Islamic Azad universities and State universities), almost no organized study has been done on the type of tests and the criteria used to assess students' translations. The present study set out to survey the way translation instructors evaluate students' translations. Participants were solicited for their test method preferences and the criteria they used to assess students' translations. To achieve the objectives of the study, a 22-item questionnaire was administered to 36 male and female translation instructors who have taught translation courses in 9 different Iranian universities. Results of the data analysis indicated that there is agreement among translation instructors in assessing students’ translations. Almost all of the instructors preferred to use essay-type questions in translation exams. They also indicated that they paid attention to the grammatical points, word equivalence, genre, register, and shifts to assess students’ translations. The findings of this study may be useful to the students majoring in translation as well as translation instructors. Students will certainly improve their translations if they are aware of the criteria used to evaluate their translation. Translation instructors who do not have much experience in teaching or testing translation courses can also use the findings of the study to improve their tasks. 

Key terms of the study

Classroom translation, assessment of student translation, translation test methods, translation assessment criteria

1. Introduction

Translator Training is a roughly new course which has been taught in almost 133 universities in Iran. Students should take about 135 credits for a B.A degree out of which the number of translation courses is 36 two credit, and 1 four credit courses (76 credits). Students pass the courses through four years (eight semesters).

Assessing students’ translations in translation courses is one of the most difficult tasks facing translation instructors. The main difficulty in students’ translation assessment is its subjective nature. So a translation that is appropriate in one context or by one instructor may be unacceptable in other circumstances.

The fact is that most of the translation instructors teaching at universities in Iran hold a degree in English literature, TEFL, or linguistics and rarely in translation studies. So they do not have enough academic and educational background in translation and this has caused many problems for them and for students. They may be successful in teaching in translation courses, but certainly most of them have problems with assessing students’ translations and this important task turns into personal preferences.

Most of the translation instructors have serious problems with the assessment of students’ translations at universities. Some instructors may be professional enough in teaching and testing, but may not be in the field of translation. On the other hand, those who might be some professional in translation, who lack the necessary knowledge and skill in the evaluation and assessment of classroom translation this important task may be affected by different factors including the instructor’s subject knowledge, language command, reading comprehension, translation experience and attentiveness and so forth that together could be referred to as instructor’s traits.

The result is that students are often not satisfied with their grades, and complain about the way their translations have been evaluated. While instructors are usually familiar with the theories of translation and have enough knowledge in teaching they might not have proper criteria for assessing students’ translations, or if they do, students are not informed of these criteria before the exam. So they do not know what criteria and factors are important to their instructor, and which criteria they should pay more attention to in their translation and at the time of exam.

Not knowing the criteria used by the instructors is mostly the main reason of students’ dissatisfaction with their grades. This, of course, is done by means of classroom translation exercises in which instructors identify specific problems in students’ translations such as translation loss, the importance of cultural context, and so on.

This study intended to find out how translation instructors prepare translation tests especially for midterm and final exams and what criteria they use in the assessment of students’ translations. There is the hope that a study on translation test preparation and assessment will translation instructors and other stake holders in translation curriculum by identifying test methods and assessment criteria currently being used in Iranian universities. Being aware of the assessment criteria in translation, students will understand what the characteristics of a good translation are to their instructors. The findings will be helpful to translation instructors, especially amateur instructors. Once the instructors understand the proper methods and test items that are more useful in making translation tests, they will be able to prepare more effective translation tests. This is while the finding of the study will hopefully contribute to the body of the knowledge of the translation studies.

The study, therefore, sought answers to the following questions.

1. What are the various test items translation instructors use to assess students' translations?

2. What are the major criteria translation instructors use to assess students' translations?

3. Are the criteria used by the instructors teacher-made or theory-based?

4. Is there any consensus between translation instructors and translation specialists about the translation evaluation criteria?

5. Is there any consensus among instructors in using the criteria?

2. Review of the previous studies

The review on the previous studies that have been done in the field of translation assessment shows that to translation researchers and translation theorists, the followings are the most important criteria and test methods to evaluate a translation:

Almost all of the translation researchers and theorists believe that a translator should render the same message of the source text into an acceptable form of the target language. So a translation must be accurate, natural and clear especially to the target language readers.

Since no two languages are the same systematically and culturally, some shifts such as; additions, omissions, and substitutions are necessary in some occasions in translation, but not in every occasion. So the theory of the shifts in translation is one which is strongly confirmed by many people (see for example, Catford 1965 cited in Leonardi, 2000).

The applied style and grammar, by the author, should be under consideration. A translator must try to use the same grammar and style in the target text. But if there is a wide difference between two languages, s/he could change them into an acceptable form in the target language.

Since each word has different meanings in different situations (functions), the translator should try to choose the best meaning for the words. To do that, a translation should be done in the context. Doing translation out of context, the translator will miss the chance to find the correct meaning of the words (see for example, House 1997).

As every country has its own language and culture, to translate idioms, metaphors, proverbs, etc. accurately, paying attention to the cultural effects and differences is very important. The translator should render the nearest meaning of them base on the culture of both source and target languages. However if the differences are wide, to make the translated text sensible to the readers, s/he can translate them according the target language culture. 

Register and genre are important criteria that should be concerned in translation, because they strongly stress the communicative role of the text. It is also the translator’s duty to recognize the text type before beginning his/her translation (see for example, Nieminen, 2004).

The studies also show that the following test types are workable to assess students’ translations:

- Open-ended questions (free-response items)

- Multiple-Choice

- True/False statements

- Matching

- Fill in the blank

3. Method

This study was a descriptive and survey study in which translation instructors’ methods and ideas about the assessment of students’ translations in universities was investigated.

3.1. Participants

Overall two groups participated in this study. Both groups were the instructors that have taught translation courses in different universities in Iran.

Five Translation instructors participated in the first phase of the study, and completed the first questionnaire. The participants were selected from four universities in Iran; Azad Islamic universities in Kazeroon, Shiraz, Khorasgan, and Yazd. This group of participants was asked to participate in the study because they were experienced (between 10 to 20 years) in teaching and testing in translation courses in the universities. Their answers to questions were used to prepare the second questionnaire. To elicit accurate and objective responses from the participants, the following points were taken into consideration: 1) both the significance and the purpose of the study were explained to them, and 2) the participants answered the questionnaire items anonymously, so they felt at ease in answering the questions objectively.

In the second phase of the study, 36 translation instructors completed the second questionnaire. The instructors were from nine universities in Iran. 56% of them held an MA degree, 11% were Ph. D students, and 33% of them held a Ph. D. degree. 69% (25 instructors) of them were male and 31% (11 instructors) of them were female. They had between 2 years to 35 years experience in teaching in university. They were asked to participate in the study because they have taught translation courses for at least two semesters.

3.2. Instruments

Two questionnaires were used. The first questionnaire, which was an open-ended questionnaire, was designed on the basis of reviewing several studies on translation issues, and interviewing translation students informally in universities about the tests and criteria for classroom translation assessment. The second one, which was structured, was constructed on the basis of the participants' answers to the open-ended questionnaire (see Appendices A and B).

The first questionnaire contained two parts. The first part contained 9 questions about the instructors’ principles in preparing and scoring translation tests. The second part included four questions regarding the instructors’ ideas and knowledge in the assessment of the students’ translations, and the appropriate criteria that could be applied in this regard. At the end of the questionnaire, the instructors were asked to mention their general comments regarding evaluating students’ translations. Completing the questionnaire, the instructors introduced the criteria and factors that may be observed in translation assessment, and different testing methods which they may apply in preparing classroom translation exams.

In the second questionnaire, 36 Translation instructors marked the frequency of their agreement with the selected items. Overall there were 22 items in this questionnaire categorized under three parts. The first part contained 6 factors instructors may consider in evaluating students’ translations. In this part, the instructors marked the frequency of the factors they applied when grading students’ translations. In the second part, the frequency of the things students are required or allowed to do in their translation exams was solicited. The last part included 8 items was about translation instructors’ attitudes toward different aspects of students’ translations. Participants’ answers to questionnaire items were codified and entered into SPSS program for statistical analyses.

4. Results

Parts A and B of the second questionnaire contained 14 items and focused on the second research question. To have a better picture, the first two scales (always and usually) and the second two scales (often and sometimes) for each item were are merged. Table 1 presents the frequency and percentage of the items in parts A and B of the questionnaire.

Table 1: Descriptive statistics for parts A and B of the questionnaire (14 items)

Part C contained 8 items and focused on the research question one. To have a better picture, the first two scales (strongly agree and agree) and the second two scales (strongly disagree and disagree) of the items were merged. Table 2 presents the results of this part.

Table 2: Descriptive statistics of part C of the questionnaire (8 items)

To find out if the differences between participants' responses were statistically significant or not, the Chi-square test of significance was run on the data. Table 3 presents the results of the Chi-square and as can be seen, items 6, 9 and 13 did not show a significant difference at a 0.05 level and were, therefore, excluded from the findings.

Table 3: The Chi-square results for the frequency of the items

5. Discussion

This section includes three parts; in the first part, the issues to which instructors paid attention in preparing translation tests are discussed. The second part discusses about the test methods that were used by the instructors to prepare translation exams. The evaluation criteria which were important to instructors in assessing students’ translations are discussed in the last part. The following points were extracted from the panels’ suggestions:

5.1. The issues related to translation tests

Majority of translation researchers and theorists believe that assessment of translation is a subjective task and that is really difficult to assess translations objectively (see for example, Williams, 2001). That is while most of the participants of this study believed in the possibility of objective assessment of students’ translations. So there was a disagreement between the translation specialists and Iranian translation instructors in this regard. The instructors’ belief may be attributed to the fact that they suggest typical equivalents and translations for words and texts in the class and they expect students to emulate the same translations at the time of exam. This might be what leads them to believe that students' translations must be assessed objectively.

As there are different criteria in evaluating students’ translations, do the students have the right to be informed of the criteria that are important to their instructors before exam? Almost all the translation instructors agreed that the instructors should inform students of their evaluation criteria. They stated that they explained the criteria to their students before exams to help them get better marks in translation exams.

5.2. Test methods used to assess students’ translations

Participants of the study identified several test methods including Multiple-Choice, Completion test, Essay questions, True/False statements, and Cloze test they used in preparing translation exams. Every instructor may consider one or some of the test methods in translation tests. Which one do they prefer to use to assess students’ translations in Iranian universities? All of the mentioned test types were more or less acceptable methods to the translation instructors, but to assess students’ translations, almost all of them preferred to use Essay type questions in translation tests. To this end, the instructors stated that they usually assign some short texts of different types and ask students to translate them in translation exams.

Contextualization of the translation test items is an issue which is confirmed by many translation researchers. Even if the semantic equivalent of an expression in the source language text is quoted in a dictionary or glossary, it is not enough, and it does not guarantee a successful translation. So there is no absolutely correct translation outside the context, and the translation should be done in a context (see Pinto, 2001 House, 2001). There was an agreement between the translation specialists and Iranian translation instructors in this regard. Almost all of the instructors in Iranian universities, believed in the contextualization of the translation test items, i.e., providing enough context for the students to translate selected texts.

5.3. Translation assessment criteria

Although almost all of the translation evaluation criteria stated in the questionnaire were acceptable to the instructors, the following criteria were more important to them.

1. Almost all of the translation instructors believed that students must try to render the same message of the source language into the target language in their translations. Accuracy in rendering the exact message from source language into target language in translation has been one of the important factors attended to by translation theorists and specialists (see, for example, Newmark, 1988).

2. Attending to the grammatical points and word equivalents chosen by the students in their translations was the other important factors mentioned by all the instructors in evaluating students' translations. They expected students to choose the word equivalents and the grammatical points which were close to ones in the source text and acceptable in the target language. This is based on the fact that the translator’s goal should be to reproduce in the receptor language a text which communicates the same message as the source language but using the natural grammatical and lexical choices in the receptor language (see, for example, Larson, 1984).

3. Since each language has its own division of the lexicon into classes such as nouns, verbs, adjectives, etc., it is not uncommon to translate a noun in one language into a verb in another language or vice-versa (see Larson, 1984). Applying appropriate shifts in translation was another important issue to almost all of the translation instructors. The important point to instructors was that students should not apply the shifts every where in their translations, but they should recognize the situation and use the shifts where it is necessary (i.e. where there is no proper equivalent part of speech for the word in the target language).

4. Most of the instructors stated that register and genre of the texts were important things which students ought to consider in their translations. These criteria were not considered as important as the three criteria mentioned above, but they were important to instructors. Attending to the register and genre of the texts is important because in some situations, mistranslating a word or having an ungrammatical element in the translation does not necessarily ruin the whole translation, but a failure to recognize either register, or genre will affect the whole text (see Nieminen, 2004).

5. Observing the cultures of the source language or the target language in translation is a controversial issue. There is no agreement among researchers in this area. Some believe that rendering the exact message of the source text to the target text, a translator must pay attention to the source language culture and translate the text exactly according to it. To this group, being loyal to author’s culture is more important than that of reader’s culture. On the other hand, some researchers believe that because the cultures are different, a translator must translate according to the target language culture even if some changes occur. All of the translation instructors indicated that they subscribed to the second view. The instructors stated that students must pay attention to the differences between source and target language cultures in their translations and translate the text according to the target language culture to make a translated text clear to the readers.

6. Conclusion

Fortunately there is an agreement among translation instructors in assessing students’ translations. Although most of the instructors were not familiar with the translation theories, they applied the theories to assess students’ translations. So we can say the criteria that are considered to assess students’ translations are teacher-made, because most of the instructors chose them based on their experience in this field, and not based on their knowledge. The followings are the most important test method and translation criteria used by instructors to assess students’ translations.

To prepare a translation test to assess students’ translations, the instructor should try to use essay questions in the context. Choosing this test type the students may get better results in their translations qualitatively and quantitatively.

What the students must take care in their translations as the most important factor is render the same message of the source text into an acceptable form in the target text.

The students must consider grammatical points, world equivalents, and cultural differences between the source language and target language.

Applying the necessary shifts (additions, deletions, and substitutions) is acceptable in translation in part of the students.

The students must also take care of register, genre, and style in the source text.

References 

House, J. (1997). Translation quality assessment: A model revisited. Tubingen: Gunter Narr.

House, J. (2001). Translation quality assessment: Linguistic description versus social evaluation. Meta, XLV1, 2, 243-257.

Larson, M. L. (1984). Meaning-based translation: A guide to crass-language equivalence. New York & London: University Press of America.

Leonardi, V. (2000, October). Equivalence in translation: Between myth and reality. Online Translation Journal, 4, 4, 1-14. Retrieved 27/2/2005 from the World Wide Web: http://accurapid.com/journal/14 equiv.htm

Miremadi, A. (2003). Theories of translation and interpretation (8^th ed.). Tehran: The Center for Studying and Compiling University Books in Humanities (SAMT).

Newmark, P. (1988). Approaches to translation. London: Prentice Hall International (UK) Ltd.

Nieminen, T. (2004). The value of register, text type and genre for translation and translation assessment. Retrieved 20/8/2006 from http://www.uta.fi/

Pinto, M. (2001). Quality factors in documentary translation. Meta, XLV1, 2, 288-300.

Williams, M. (2001). The application of argumentation theory to translation quality assessment. Meta, XLV1, 2, 326- 344.

Appendix A: Questionnaire No. 1

Part A: Please answer the questions about assessing translation tests.

1. What are the various types of tests you use to assess students’ translations? Please name.

2. What are the major criteria that you use to assess students' translations? Please describe.

3. How did you select the criteria?

4. What sources have you used to select your criteria and what do you think about them?

5. Do you inform your students about the criteria before exams? Why?

6. Do you prefer to be objective or subjective through the assessment of students’ translations? Please explain.

7. Are students allowed to use dictionary during their exam? Why?

8. Do you respect students' tastes and performances in their translations, especially for equivalence finding?

9. Some instructors compare students' translations with their own and the more similarity between the two translations they find, the better mark they assign. What is your idea about it?

10.   In which situations will students get negative or positive point for additions and deletions in their translations?

11.  Do you prefer seen or unseen translation test? Please explain.

Part B: Please answer the following questions about the theories of translation.

12.   Do you follow an especial theory of translation? If yes; what is it and why do you follow it?

13.   What is your idea about beauty of translation? What does it mean for you?

14.   What types of changes (word equivalent, grammar, style, etc.) in translation are acceptable to you? Please give some examples.

15.   Would you please provide us with your general comments regarding evaluating students’ translations?

Appendix B: Questionnaire No. 2

Part A: The following are factors instructors consider in evaluating students’ translations. Please mark the frequency of the factors as they apply to you when grading students’ translations.

1. I consider Grammar as a criterion in my evaluation.

Always  Usually Often Sometimes Never

2. I look for appropriate use of Word Equivalents as a criterion in my evaluation.

Always  Usually Often Sometimes Never

3. I attend to Style as a criterion in my evaluation.

Always  Usually Often Sometimes Never

4. I check Accuracy in rendering the exact message from the source language in my evaluation.

Always Usually Often Sometimes Never

5. I consider Beauty as a criterion that should be kept in my evaluation.

Always  Usually Often Sometimes Never

6. The more similarity between students’ translations and my own translation, the better mark students will obtain.

Always  Usually Often Sometimes Never

Part B: The following items ask about the things students are required or allowed to do in their translations.

7. Students must take care of the Registers in their translations.

Always  Usually Often Sometimes Never

8. Students must take care of the Genres through the translation.

Always  Usually Often Sometimes Never

9. Students must translate the text exactly according to the SL culture even though it may not make sense in the TL.

Always  Usually Often Sometimes Never

10. Students must observe the differences between TL & SL cultures and translate the text according to the TL culture.

Always  Usually Often Sometimes Never

11. Students have the chance to invent equivalent for the words that have no proper equivalent in the TL.

Always  Usually Often Sometimes Never

12. Additions and Omissions on the part of students are acceptable in translation.

Always  Usually Often Sometimes Never

13. Catford’s* Shifts are acceptable in translation on the part of the students.

Always  Usually Often Sometimes Never

14. I think objective assessment of translation is possible.

Always  Usually Often Sometimes Never

Part C: The following are instructors’ attitudes toward different aspects of students’ translations. Please select the choice you use in preparing translation tests.

15. Students should be informed of my evaluation criteria before exam.

Strongly agree Agree Undecided Disagree Strongly disagree

16. Students are allowed to use dictionary in the exam.

Strongly agree Agree Undecided Disagree Strongly disagree

17. I think Multiple-Choice is a proper test type to assess students’ translations.

Strongly agree Agree Undecided Disagree Strongly disagree

18. I think Essay type is a proper test type to assess students’ translations.

Strongly agree Agree Undecided Disagree Strongly disagree

19. I think Completion Test is a proper test type to assess students’ translations.

Strongly agree Agree Undecided Disagree Strongly disagree

20. I think Cloze Test is a proper test type to assess students’ translations.

Strongly agree Agree Undecided Disagree Strongly disagree

21. I think T/F statement is a proper test type to assess students’ translations.

Strongly agree Agree Undecided Disagree Strongly disagree

22. I think the test items should be contextualized (put in a context).

Strongly agree Agree Undecided Disagree Strongly disagree

*According to Catford (1956), four kinds of Shifts are possible in translation; structure-shifts, class-shifts, unit-shifts, and intra-system shifts.

Read more articles - Free!

E-mail this article to your colleague!

Need more translation jobs? Click here!

Translation agencies are welcome to register here - Free!

Freelance translators are welcome to register here - Free!

Subscribe to TranslationDirectory.com newsletter - Free!

Take part in TranslationDirectory.com poll - your voice counts!

Menu

Free Newsletter