Seeing the Wood for the Trees in Machine Translation Evaluation: An LSP Success Story from RWS

Track: TAUS | TS8 |
Friday, October 16, 2015, 3:30pm – 4:15pm
Held in: Mission City Ballroom 4
John Tinsley
Host: Jaap van der Meer

Perplexed about the real return on investment on machine translation (MT)? You are not alone! Most translation and localization organizations are seeking ways in which to measure the actual, real productivity to be gained from introducing MT into the translation workflow. But let’s face it, we have a lot of data on our hands with so many types of evaluations, from automatic measures to subjective assessments of fluency and adequacy. Despite all of these data points, we are all ultimately just trying to answer one simple question: is the MT useful and to what extent? In order to answer this, we need to cut through the data to get to the crux of the matter. In this session, Iconic Translation Machines will discuss a case study relating to a large-scale post-editing evaluation carried out in collaboration with the RWS Group. Over 20 translators across three sites in China post-edited Chinese to English patent translation using the TAUS Dynamic Quality Framework (DQF). The objective tests produced valuable data, which enabled RWS to build a business case for the adoption of MT for its patent translation business. Join us to hear how!

Takeaways: Attendees will get practical insight into how to design and implement an effective post-editing productivity evaluation using the TAUS DQF tool; hear an LSP view on how to analyze evaluation results and interpret them in a way that makes sense for the MT business case; and see an MT-provider view on what is often required from a development perspective in order to meet the evaluation criteria of a client.