Man vs Artificial Intelligence : Human Translators Knock out the Competition

The abilities of human translators still surpass that of artificial intelligence. However, this is certain not to last. Despite some failures, artificial intelligence has been known to beat human champions at games such as the game of Go (a Chinese board game) or, more recently, poker. However, in the domain of translation, it still has a long way to go…

Under the organisation of Sejong University’s International Interpretation and Translation Association (South Korea), four human professional translators went up against three translation programmes: Google Translate; Systran and the Naver Papago Translate app.
The human translators had 50 minutes to translate four literary and non-literary texts from English into Korean and vice versa. Internet access was allowed, given that none of these texts had ever been translated. Also connected to the internet, the three programmes did the same but 5 times quicker. The quality of translation was evaluated in terms of the three following criteria, with a maximum of 5 points for each: accuracy, expression, coherence and organisation. Out of a perfect score of 60 for the four texts, the human translators scored 49 points. The best out of the four artificial intelligence systems scored only 28 points. The specific scores for each programme have not been disclosed.
The organisers of this competition explain that the problem with artificial intelligence lies in its partial NMT approach (Neural Machine Translation; different from that of Statistical Machine Translation) which, coupled with the use of an artificial network of neurones and deep learning techniques, does not allow for sufficient comprehension of context. (Quoted by the Korean Central News Agency Yonhap).
They add that for texts translated using the NMT approach, grammatical errors are very common. A representative of Systran admits the programme’s difficulties in translating nuances and emotions of certain phrases.

by Jennifer Cleaver