Good Things in Small Packages

David Versus Goliath is story about an underdog who defeats a mighty opponent against overwhelming odds. What does it teach translators, teams and agencies about machine translation?

Despite six (6) decades of research and a proclamation of human parity, we continue to experience machine translations (MT) blunders and reality. A bigger is better mindset has prevailed through three (3) major MT technology generations. Coincidence?

First Generation MT

In MT’s humble beginnings, expert linguists painstakingly crafted rules for two languages. Software engineers encoded rules-based MT (RbMT) engines like SYSTRAN’s Desktop Translator. RbMT with its big teams of experts and Goliath-sized resources established the mindset that bigger is better when building MT engines.

The bigger is better mindset, that experts with Goliath-sized resources build MT engines, is RbMT’s lasting contribution to MT.

The Goliath-sized costs and low quality stopped RbMT’s growth. At the end of its life cycle, RbMT never achieved broad market acceptance.

Second Generation MT

In the early 2000s, artificial intelligence (AI), machine learning (ML), “big data” training corpora and data scientists replaced the expert linguists who crafted rules. Statistical machine translation (SMT) engines moved to powerful online servers because PCs couldn’t run ML and AI.

The bigger is better mindset persisted as SMT experts proclaimed only experts with Goliath-sized resources can build SMT engines.

Online SMT engines were designed as a jack of all trades, masters of none to service millions of people across virtually every possible topic. Customized SMT engines delivered better quality but they were even more expensive.

Third Generation MT

By late 2016, newer ML & AI and “Goliath data” became the expert linguists. Neural machine translation (NMT) engines moved to Goliath-sized super-computers and displaced online SMT.

The bigger is better mindset reaches new heights as NMT experts secure ever bigger Goliath-sized resources to build NMT engines.

Online NMT engines are still jacks of all trades, masters of none. Online services prefer NMT because it’s easier for them to maintain than SMT. RbMT’s bigger is better legacy continues.

Is Bigger Really Better?

Let’s see how the bigger is better transition from SMT to NMT improved the translator’s experience.

In mid 2016, Memsource published this study of Google and Microsoft SMT titled Machine and Professional Human Translations Identical. It shows translators accepted 5% of the MT suggestions as publishable without edits.

5% of Google SMT exactly matched the human translations.

2016 Memsource Study

Two years later and after Google had changed to NMT, thirty-one (31) Slate Rocks customers translated their works with Google’s NMT. We learned 5% of Google’s NMT exactly matched their own translations.

5% of Google NMT exactly matched the human translations.

2018 Slate Rocks Study

Translators experienced practically no change in the NMT that matched the translators’ work but according to the bigger is better mindset the number should have increased.

Why do so many translators report that NMT is better? I suspect there’s a strong placebo effect. Experts say NMT is better. Translators know they’re working with NMT. Therefore, NMT is better. This is just like a patient feels better after a doctor administers a placebo and tells the patient he’ll feel better.

Unfortunately, most translators don’t monitor their work and NMT vendors are biased towards their products. We will never really know the truth until language researchers use genuine scientific studies with double-blind processes.

Underestimated Small Packages

The same thirty-one (31) Slate Rocks customers translated their works with their personal, customized slate® desktop engine. We learned 34% of the customized SMT exactly matched their own translations.

34% of slate® desktop SMT exactly matched the human translations.

2018 Slate Rocks Study

A smaller SMT engine created a seven (7) fold increase in segments that matched the translators’ works! Why do translators report that SMT is not as good as NMT?

This Slate Rocks study was the first research to rely on objective criteria. Traditional language technology research uses subjective criteria and processes that are vulnerable to placebo effect. Researchers desperately need transformation to double-blind studies.

Through sixty (60) years of MT research and development, the bigger is better mindset has perpetuated two myths.

  • Myth: Only Goliath-sized resources can build MT engines.

    Fact: MT experts report MT engines work better within subject-constrained domains that require smaller resources.
  • Myth: Translators don’t have the skills to build MT engines.

    Fact: Intuitive Windows-compatible applications make it easy for translators to build MT engines using skills and resources they have.

MT itself does not require Goliath-sized resources. The use case determines the size of the required resources.

Deploy an MT engine to a Goliath-sized use case and Goliath-sized resources are required. Example: MT engine deployed to online servers servicing millions of demands across virtually every possible topic requires a training corpus of tens of millions of segments from thousands of different translators.

Deploy an MT engine to a David-sized use cases require David-sized resources. Example: An MT engine for a translation team of 3 or 4 colleagues working on insurance claims requires a training corpus with 3 or 4 years of those translators’ segment pairs.

David Versus Goliath can be interpreted as the underdog defeating a mighty opponent against overwhelming odds.

It also teaches how a small competitor with confident, like a translator, team or small agency, wins against the self-absorbed giant stuck in outdated beliefs. It teaches bigger isn’t always better. It demonstrates good things come in small packages.

Brought to you by Slate Rocks LLC

Learning slate® desktop takes less than an hour.

Using slate® desktop saves days, weeks, months and years.

Start saving now!