During the middle of the last century, researchers began to expand the study of linguistics. John Rupert, sometimes called the “first linguist of Britain,” developed a theory that encompassed his concept of both prosody, the idea that each person’s language is completely individual, and context, the specific way we put words together to create meaning.
These two observations partially explain why evaluating the quality of a translation is so difficult. Not only the prosody and context of the source language must be considered but also the prosody and context of the linguist creating the translation. The same source text can be given to half a dozen different linguists and get half a dozen different results, all correct, but some naturally written better.
At Tembua, we frequently explain this to clients who, understandably, think there is only a single correct translation for any phrase, what is often called “word-for word.” A browse through a good monolingual dictionary shows numerous synonyms for many entries, which often can be used interchangeably. Our goal, of course, is to produce translations that are more than simply an approximation of the source text in another language.
The translation should also be evaluated by the language’s effect on the reader/listener. Juliane House, a leading expert in translation quality assessment, suggests beginning a project by analyzing the source text along numerous dimensions. The target text must line up with the source along those dimensions.
Other researchers point to the effect that text has on readers. The translation should have the same effect as the source, whether it be to prove a point or bring about a sale.
For several decades, three rules for translation have been used (first formulated by Scottish historian Alexander Fraser Tytler). Paraphrased, they are:
- Register, because the language used in a love letter is different than that used in an instruction manual; and
- Style, obviously the most difficult both to achieve and to measure.
Tembua measures translation quality using these three rules. All translations are quantitatively reviewed for spelling and grammar. Our quality certifications require—and we willingly implement—a record of translators who produce errors. Numerous Q/A tools are available to check the mechanics of a translation but they go no further.
Following a spelling and grammar check, we evaluate the target text for completeness (rule 1). This is not as straightforward as it sounds. Many words contain several concepts which may take whole phrases to capture in the target language. The English word feast, for example, may conjure up a large number of complicated dishes and an elegant table setting with smiling people holding wine glasses. Or it may mean a meal with much more than the individual usually has to eat.
Register and style (rules 2 and 3) are addressed by using professionally trained, experienced linguists, each of whom works with a reviser to capture the source text accurately. In some cases, we also employ a subject matter expert to verify current industry-standard terminology and style. These team members work back and forth to polish a text so that the tone of the original is maintained.
Metrics have been developed in the translation industry for certain types of texts. J2450, for example, was a project of the Society of Automotive Engineers and identifies seven error types, categorizing them as serious or minor. While J2450 is a useful tool for automotive text, for which authors are taught to use a set vocabulary and repetitive phrases, it does not address style or actual errors in the source text.
Other metrics include the much more complex LISA (Localization Industry Standard Association) model and the work done by the Translation Service of the European Commission.
These metrics are time consuming, complex, expensive, and, in many cases, subjective. Today’s business climate rarely allows for resources to be used in this way, except in specific areas such as the biomedical device industry.
Tembua’s CAT (computer-assisted translation) tools are excellent for maintaining consistency with previously translated work that has been evaluated and accepted. This also speeds delivery and drops the price—important points for our clients!
We look for the day when source and target texts can be automatically evaluated along all the above dimensions and the translation given a number or grade that shows its quality. Today, the best measure of our quality is the number of clients who continue to send us work for many years.
Our membership in professional organizations allows us to track new research and techniques. These we use to improve the translation quality Tembua delivers to its clients!