English-Catalan Neural Machine Translation: state-of-the-art technology, quality, and productivity

Recent major changes and technological advances have consolidated machine translation (MT) as a key player to be considered in the language services world. In numerous instances, it is even an essential player due to budget and time constraints. Much attention has been paid to MT research recently, and MT use by professional or amateur users has increased. Yet, research has focused mainly on language combinations with huge amounts of online available corpora (e.g. English-Spanish). The situation for minoritized or stateless languages like Catalan is different. This study analyses Softcatalà’s new open-source, neural machine translation engine and compares it with Google Translate and Apertium in the English-Catalan language pair. Although MT engine developers use automatic metrics for MT engine evaluation, human evaluation remains the gold standard, despite its cost. Using TAUS DQF tools, translation quality (in terms of relative ranking, adequacy and fluency) and productivity (comparing editing times and distances) have been evaluated with the participation of 11 evaluators. Results show that Softcatalà's Translator offers higher quality and productivity than the other engines analysed.