Machine Translation Tool Wars

Today the debate around machine translation is stuck. It’s stuck on what engine to use, rather than looking at the whole process for optimizing MT. Welcome to the Machine Translation Tool Wars!

In this war of words, discussions – sometimes heated and not always evidence-based – contrast today’s two dominant approaches, and the competing software built up around them. Camps tend to be divided between supporters of rules-based machine translation and those of statistical machine translation, the first approach, as its name suggests, relying on grammatical rules that have been hard-coded into language-specific engines and the second using algorithms to predict the most likely translation.

The tool wars rage on – in conferences and online. RBMT and SMT face off against each other, their individual merits mostly argued by vendors of one solution or another. Converts are determined by which vendor is getting the most face time, or seems the most credible. Open source, naturally, gets a lot of traction due to the attractive pricing (free, if you don’t count learning curve.) Evaluations abound, but they are often supplied by vendors of one solution or another: in tests, a trained SMT engine may be pitted against an untrained RBMT engine, or vice versa.

These contests are also inevitably skewed by the fact that content types and language pairs tend to play a determining role in engine performance. One engine is better at French, another at Japanese, and both sides declare victory.

Behind the face-offs are the metrics – BLEU, NIST, GTM – which may or may not concur with each other, let alone with human evaluations. Critics complain that the playing field is not level and that certain metrics favor certain approaches. Since the very metrics used to measure quality may contradict a human judgment on whether a particular translation is good or not, automatic quality evaluation tools are just as inconclusive in declaring a winner in the MT wars as the face-offs have been.

Hype is another disruptive factor. Software vendors make claims that can only be tested with an expensive pilot. Language Service Providers (LSPs) may only be familiar with one particular tool, and so cannot offer customers the engine that performs best with their content, file formats or language pairs.

No wonder the market is confused!

I honestly believe – and have been told – that LexWorks (and our parent company – Lexclera Eurotexte) is a different sort of LSP. We are engine agnostic. Experience over nearly a decade has taught us that:

        1.

There is no such thing as one engine fits all.

        Rules-based performs best in some situations, SMT rules in others, and the Hybrid beats both much (but not all) of the time. The trick is knowing the right approach for the right content.
        2.

Machine translation is not a tool. Machine translation is an industrial process.

        And like any process, there are many facets to master, not just the choice of the engine.

We’re available if you have any questions.