A Roadmap to Quality Translations, Part 1

A Roadmap to Quality Translations, Part 1

By Luigi Muzii,
Team Leader,
Gruppo L10N,
Rome, Italy

info[at]gruppol10n.it
www.gruppol10n.it

Get the List of 4,500+ Translation Agencies Now!
No Recurring Membership Fees!

Luigi Muzii photo In the past few years, a new trend has emerged: standardizing procedures for a contractual relationship between the client and the service provider. The idea is this: following certain pre-established procedures when producing a translation will increase the likelihood of good quality.

In fact, the fundamental assumption in quality standards (namely ISO 9000) is that business processes can be improved to ensure the desired level of quality at each pass. And for this approach to work, general criteria are necessary to standardize the production process and appraise the quality level.

SPECIFICATION OF REQUIREMENTS

When dealing with quality, two basic principles must be acknowledged:

Quality is relative; people perceive different quality levels in the same product.
Quality levels are subject to constraints in requirements.

A specification of requirements is a document providing an adequate and unambiguous description of the task load for a project, together with a description of the desired results, the essential conditions to which the service must conform, and the characteristics or features of each deliverable.

Most quality problems in translation have little to do with mistakes, and more to do with a mismatch of assumptions and goals between the people requesting a translation and the people supplying it. Clarifying requirements beforehand helps prevent such mismatches, but gathering requirements from the user is not always a straightforward task.

On the other hand, if you can’t collect requirements, you don’t know your clients, and if you don’t know your clients, you can hardly please them.

The key to quality translation is really the ability to successfully negotiate between competing demands, to find the translation that fits a particular situation and that represents the best tradeoff between requirements that cannot all be met simultaneously.

The name of the European quality standard for translation services, EN 15038:2006, reads, “Translation services – Service requirements,” and its purpose is to establish and define the requirements for providing quality translation services. Admittedly, a key issue is quality assurance and the ability to trace its progress, yet the standard does not envisage service-level agreements or metrics.

A service-level agreement is a contract between a service provider and a buyer or user of that service (the client), and it specifies the level of service that is expected during the term of their agreement. It also defines the terms of the provider’s responsibility to the client, and either the type and extent of compensation, if those responsibilities are met, or the extent of penalty, if they are not met.

Despite the lack of specifications for translation quality metrics, the standard does require “that information about any specific linguistic requirements in relation to the translation project is registered. Such information can include requirements of compliance with a client style guide, adaptation of the translation to the agreed target group, purpose and/or final use, use of existing terminology, and updating of glossaries.”

METRICS

Metrics are a set of rules that allow users to measure how much a product (the translation) meets requirements, and metrics are generally used to measure performance. The primary goal of measuring, of course, is to create a standard against which something can be judged. What’s often forgotten is that metrics can be used not only to measure performance, but also to identify specific problems that are affecting performance.

Effective metrics must be objective (measurable), unbiased, and able to provide enough resolution (detail) to assess the factors that need improvement. This means that any two people who set out to calculate the value of a metric must be able to produce comparable results.

Typical metrics are SAE J2450, recently elevated to a standard, whose goal is just to provide “a tangible method for measuring the quality of translation deliverables as precisely as for any manufactured product.”

SAE J2450 provides for severe and minor occurrences of wrong terms (glossary violation or conflict with de facto standard translations), syntactic errors, omissions, word structure or agreement errors, misspelling, punctuation errors, and any linguistic errors related to the target language and that are not clearly attributable to the other categories.

Subjective metrics are hard to measure, because their value depends as much on opinion as on demonstrable facts. Translation quality can be a typical case of subjective assessment, as all translations are prone to subjective influences. This is due to the subjective conditions of the interpretation process and of translators’ personalities. Reviewers and editors are subject to the same influences.

TRANSLATION QUALITY ASSESSMENT

The definition of quality, as stated in ISO 8402:1994, 3.1 reads, “The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.”

Quality also can be defined as an integration of the features and characteristics that determine the extent to which output satisfies the client’s needs. And “needs” are not just those stated, but also those implied.

The most important implied need in translation is accuracy. People who use the services of translators don’t ask for an accurate translation; they just assume that it will be accurate. Another implied need is successful communication of the text’s message to the readers. And for both needs, the client is usually the de facto judge of quality.

Therefore, a translation is of adequate quality, supposedly, if the client does not complain about it.

Even though quality is always a very personal issue, consistent and acceptable translation quality can be achieved through quality-oriented process design and standardization. Such a process is known as quality assurance. It is a planned and systematic pattern of actions necessary to provide adequate confidence that the item or product conforms to established technical requirements. Quality assurance covers all activities, and does so in accordance with two basic rules: “fit for purpose” and “do it right the first time.”

In translation, quality assurance is the full set of procedures applied before, during, and after the translation production process, by all members of a translating organization, to ensure that quality objectives—those that are important to the client—are being met.

On the other hand, quality assessment is intended for establishing whether contract conditions have been met. Quality assessment is business oriented. Unlike quality assurance, which always occurs before the translation is delivered to the client, quality assessment may take place after delivery. Assessment is not part of the translation production process. It consists in identifying—but not correcting—problems in one or more randomly selected passages of text, in order to determine the degree to which it meets the agreed standards.

The unsuccessful attempt to introduce service-level agreements and metrics in the aforementioned European Standard, EN 15038:2006, lies on the belief that, generally speaking, the clients of a translation service do not have the necessary skills and competencies to drive the provision of service through requirements. In effect, they rely on the service provider to deliver a certain degree of intrinsic quality.

The refusal of metrics is just a direct consequence, as there are virtually no tools available to validate compliance to standards—however unstated.

Nevertheless, since there is no “perfect” translation, the intended purpose of a translation and its suitability remain the only judgment criteria which, for the sake of objectivity, must be accompanied by assessment metrics. The combination of process and quality assessment of translation work will eventually tell simply whether it is acceptable or defective.

Therefore, translation quality assessment (TQA) criteria should be agreed upon with the client, should be subject to requirements, and should be formalized in a separate document. And so far, TQA has been performed on the basis of a strict correspondence between source and target texts and on intensive error detection and analysis. While this is undoubtedly the best approach from a theoretical (and maybe pedagogical) point of view, it is absolutely uneconomical.

Thus, before taking on a job, three steps should be made to anticipate refusal or dislike:

Come up with a full understanding of the expectations (requirements) of the client.
Agree on a process with the client to correct any deviations from requirements.
Implement a process to prevent the same issues in the future.

MEASURABILITY

The most commonly-asked question about quality is this: how can quality be measured? To measure something, you must know what it is, and then you must develop metrics that measure it. And for people who have always thought of quality in their deliverables as a questionable subject, defining metrics is the hardest part.

ISO 8402:1994, 3.21, defines “defect” as “the non-fulfillment of intended usage requirements.” A defect is a characteristic that causes customers to depart from their normal work processes. The number of defects is one metric that can be used to indicate how and why a product or service is not conforming to specifications or not meeting requirements. In fact, measuring the number and magnitude of defects remains to be the best way to assess quality.

The first step, then, is to establish a model or definition of quality, and then transform it into a set of metrics that measure each of the elements of quality. And if something is not relevant to the quality model established, it is not a good use of time to develop metrics to measure it. Measuring things just because they can be measured is not useful.

A comprehensive set of metrics must measure quality from several perspectives and at several points during the production process. Striving for a single, all-encompassing metric is not only troublesome, it is likely useless, as a simple metric would not reveal all the problems. Creating multiple metrics that assess the various aspects of what is to be measured can help re-compose the overall framework; and knowing which parts of a process work well and which ones don’t allows taking measures to correct the problems.

Most quality components can be clearly described and precisely verified. The quality of the finished product corresponds to general customer satisfaction ratings, while the lack of quality can be determined by defects, such as technical errors; the quality of process comes from repeatability; and typical predictors of quality are in-process indicators such as editing.

RULES OF ENGAGEMENT

Because quality is so subjective, and its definition is such a relative thing, developing quality specifications for each new project is a good method for clearly setting quality parameters.

Translation quality should be tracked from different perspectives: the number of reviews, and the time spent on each of them, the number of errors found, productivity, and suitability.

Being able to track translation defects is not only an important condition for delivering high-quality services to clients, it also provides an efficient way to evaluate vendor performance. And it’s important to note that the reasons behind errors (why they happen) are separate from the measurement of errors.

From an academic perspective, a correct translation is a translation with no errors. From a practice-oriented perspective, a correct translation is a translation where total error points result in a quality index above a desired threshold. Therefore, one way to judge whether TQA on a project is complete is to measure translation defect density.

And when dealing with TQA, ideally a tool should be available to track any potential issues in a translation. Such a tool should guide the user in judging whether or not these issues are actual and guide them in deciding whether to take any corrective actions. And for a TQA tool to work, explicit and reliable assessment criteria are required. Also required are sampling rules that can be used for extracting representative allotments of text, something that is necessary when the entire project is too large or complex to perform quality controls on all content.

See Part 2

ClientSide News Magazine - www.clientsidenews.com