The thoughts reflected in this piece stem from over 10,000+ hours of Japanese-language E-Discovery related projects based in Tokyo, Los Angeles, Phoenix, New York and Washington, DC. As a lawyer licensed to practice in the United States, I’m also culturally and linguistically fluent in English and Japanese. I work directly with legal counsel, litigation support teams, paralegals and various E-Discovery software systems, including machine translation software. Most importantly I am deeply embedded within the multi-language review teams that carefully sift through the voluminous records that make up the corpus of electronic evidence that assist counsel in developing their factual understanding of their clients’ case posture.
I’m currently based in Tokyo where I’m the Senior Manager for The CJK Group. I rarely speak on public panels and find myself amazed how the actual day-to-day of what is often relegated as the most “unsexy” work of E-Discovery (i.e. document review) is hardly reflected in the industry conference circuit. This is my humble contribution. I will, however, be presenting on some of these issues at the next Global Forum’s Tokyo Summit on April 18th, 2019 at the Tokyo American Club.
Preliminary Thoughts
For law firms engaging in international litigation, foreign language documents can be a frustrating barrier to meeting discovery obligations and to compiling supportive evidence. The inability for an attorney to read original documents presents a preliminary hurdle on top of an already large number of tasks that must be carried out during the course of often expensive, multi-year legal proceedings. Even if counsel can read the original non-English documents, the volume of data prohibits a deep dive in any meaningful way. In other words, the arduous and time-sensitive task analyzing electronic records (for better or worse) is outsourced to a 3rd party entity under supervision of counsel. This is where I enter.
No technology seemingly offers the chance to eliminate this hurdle at minimal expense more than machine translation, or “MT.” Subsequent articles will take a closer look at the various technology assisted review “TAR” software. Nonetheless, returning to MT, companies promise that for merely a few cents per page, at the push of a button, documents will be “translated” into English at an “acceptable” accuracy to make a determination as to their importance. However, when considering the use of MT, (and there are many varieties utilized in the E-Discovery industry) it is important that attorneys understand the risks and benefits to this technology—as well as its limitations. (Part II can be found here)
Replacing Humans?
It is sometimes proposed that instead of replacing human translators and foreign language proficient attorneys, machine translation can be useful for separating important documents from unimportant ones. It is argued that the higher cost dual language attorneys can review a smaller number of documents by needing to look only at a small subset of the documents, which are identified as “most likely to be important” by looking at machine translated English language copies first. This approach comes with its own set of risks.
A Typical Scenario in broad strokes
Documents identified contain non-English material. These non-English documents may be selected from potentially key custodians, key word search may be applied and/or some form of analytics-based technology assisted review. There are a myriad of ways this could happen, as this process can be fraught with motions practice between parties in determining the scope of discovery. That is not my expertise nor the point of this article. Whatever the chosen work flow, a machine translation option would be run across a set of potentially relevant documents. These now English language documents would be reviewed by English-language Attorneys. Those reviewers, be that of contract attorneys or law firm associates, would make note of certain documents that appear to have been translated poorly or appear to be of particular factual significance. At this juncture, these “flagged” documents of significance would be the only ones which would be reviewed by foreign language speaking attorneys.
The merits of this approach are fairly obvious. It is more expensive to hire foreign language proficient attorneys to review documents than an English language one. There are practical reasons why this also occurs, such that if the data set presents 12 different languages and an expedited schedule. Perhaps you are receiving a production set from opposing counsel and you just need to gain a 30,000-foot aerial view of what the corpus of evidence looks like. The type of substantive matter may also warrant the use of MT more readily than other types of cases. There are countless scenarios. I’m merely presenting some of the risks. So long as the cost of the machine translation is significantly less than the cost savings of reviewing primarily in English, the document review project can be conducted at far less expense. Fair enough, that seems to make sense.
This approach brings certain risks that any litigator should be aware of before reaching for these cost savings. This is a subtle point but it’s entirely possible that documents of legal significance can appear entirely innocuous without knowing different possible translations or interpretations of a given phrase. I’ll explain this point further, below. Lastly, while we are keenly aware of methodologies such as the BLEU (Bilingual Evaluation Understudy) score, an algorithm that evaluates the quality of translation output from one language to another, my experience on fast moving litigation with so many moving pieces of data, or noise, often does not lend itself to a contained laboratory for easy translation. Think about it. We have data sources coming from spreadsheets with dense information buried in cells, chat logs on messaging platforms like Line, WeChat, KaKao Talk, QQ, WhatsApp, Wickr, Slack, Skype, Jabber (Cisco), etc. To add to this, there’s video, audio, dynamic web data on social media (Facebook, Twitter, YouTube, Instagram, TikTok, Snap, LinkedIn and many more), handwritten notes, slang, abbreviated terminology or shorthand, industry jargon, engineering manuals, emojis, text messages. Not only are these data sources fragmented across various platforms, but the language of the written, spoken or visual content is not English. These are challenges put forth irrespective of language, but greatly heightened when faced with the gauntlet of language, platform differentiation, culture & translation. The challenges, to say the least, is both in the technical and interpretative domains. This is easier said than done.
The Litigation Risk: Documents with Hidden Significance
The risks of culling documents with machine translation lie in documents whose significance would be lost when viewed only in English. Japanese documents can contain multitudes of different interpretations in a single phrase, demanding the listener or reader to deduce the meaning out of context. For further illustrations on this point, read my colleague Yoshitaka Kashiwagi’s article entitled “Machines, Semantics & E-Discovery: Sontaku (忖度) and Tokusai (トクサイ).” We have numerous links to other pieces we have explored on this topic, including one that examines “Problems Within the Translation Industry: Status, Quality and Professionalization.”
While having a foreign language fluent attorney look at documents which appear to have mistranslations or appear to be legally significant upon being looked at in English may capture some of the documents important to litigation, the risk would be in the documents which contain dual meanings or cultural subtleties like those mentioned above. For example, a document that uses golf wordplay and double meanings may appear simply as a golf reference—losing its secondary or more legally significant meaning, with nothing to indicate to the English language reader that the document could be used to make a legal argument.
It is important to also add that I’m not suggesting machine translation does not work, especially with its advancements with the use of neural networks (which are very impressive in their own right). My point is to emphasize that there are inherent limitations and these limitations can have severe impact on the development of factual understanding on cases where this really matters. This is where most litigation (at least for me) involving multi-languages takes place, where context and noise is significant—therefore necessitating careful human-experts to steward this process. At this juncture, I would point to a useful blog entitled “Neural MT Weekly,” produced by one of the leading MT software providers in their recent post “Improving Robustness in Neural MT.” As I’ve indicated above, we are sensitive that there are serious industry conferences and working groups exclusively devoted to enhancing the sophistication of neural networks so as to increase the accuracy of machine-based solutions. While this has produced an advancement in machine translation output, as evidenced in even the recent shift from a network architecture model from that of the recurrent neural network (RNN) or convolutional neural network. As this demonstrates an increase in BLEU scores on certain language pairs, (English-to-German and English-to-French) there still remains work to be done to apply this to my typical data environment. See these two articles for those wanting to “geek out” on these topics written by a group of deep learning artificial intelligence researchers at Google Brain: “Transformer: A Novel Neural Network Architecture for Language Understanding” and “Attention Is All You Need.” I am hopeful and I think on certain types of data environments with certain languages, relative accuracy can be achieved with increasingly limited human input and training data.
Returning to my work, to further clarify, I’m not implying that simply dispatching human translators will solve this either– having a human translator untrained in legal analysis is not the panacea. It takes a certain skill of language, cultural knowledge and legal acumen to sift through complex fact patterns in another language, let alone Japanese or Chinese. I think many of my non-bilingual colleagues under-appreciate how challenging this task can be. I would go even further that in evaluating the use of certain MT work streams, to have somebody like me (Yes, I’m promoting myself here!) to optimize the appropriate use of certain technologies. This is not easy to accomplish. Human language is complex on so many levels. Ask anyone trying to learn another language. Superficial proficiency is one thing but to truly understand the complex nature of a spoken and written language is a beautiful yet rewarding undertaking.
Why is Japanese Hard to Decipher?
Japanese is a particularly challenging language for machine translation software. Japanese often asks the listener to make interpretive decisions and select one of numerous possible meanings for a given phrase based on context. Resultingly, simple phrases can be ambiguous.
Take for the example, an extremely common Japanese phrase, “Yoroshiku Onegai Shimasu.” (よろしくお願いします). If asked, “what does this mean?” a Japanese speaker would be hard pressed to ask for considerable context.
Possible meanings could include anything from:
-Please do this
-I appreciate your business
-I appreciate your hard work
-Thank you in advance
-I look forwards to working with you
-Yes, I would be happy to marry you
It can be easy to imagine why yoroshiku onegaishimasu being translated as “please do this,” versus “I appreciate your hard work,” can have a dramatically different legal ramification, depending on the case. And, in some context, it will be obvious as to which definition is applicable and that an error has been made.
Consider, for example, a machine translation that produces the phrase “I would be happy to marry you,” repeatedly in a set of business emails (an actual occurrence on a case that I have worked on). While some of these innocuous outcomes can be resolved by post-editing (conducted by humans during translation quality control) built into the work stream, it underscores the value of a “lawyer-linguist-E-Discovery” professional when confronted with the situation of complex, subjective and context-based analysis during review. I call this the trifecta of E-Discovery Project Manager. Most often multi-lingual situations maintain 1 or 2 of the above skills. To maintain the highest possible accuracy and effectiveness, your team must be led, at minimum, by a Project Manager fluent in the underlying language (inclusive of English), licensed as an Attorney (US licensed preferably) and deeply familiar in managing non-English professionals and the demands of US law firms. As I referred above, the trifecta of an E-Discovery Project Manager is paramount. It also helps that you know how to operate at least 2 or 3 of the leading database document review repositories cold.
The More Challenging Cases: The Rub
Far more problematic than the obvious errors are cases in which the linguistic misinterpretation is more subtle. Of concern is a situation in which the misinterpretation is not obvious because the context permits multiple translations, and one of those multiple translations happens to be of great legal significance.
Imagine an email simply ending in yoroshiku onegaishimasu. Both “I appreciate your hard work” or “Please do this” may be applicable. If the machine translation chooses to translate the phrase as “I appreciate your hard work,” you may never know that “Please do this” was also arguable, even if your case could benefit from this alternative translation.
Again, I use this simple example of “yoroshiku onegaishimasu” to illustrate the problem. On live cases where facts are complex and communications are dispersed among various types of file types (emails, spreadsheets, chat portals, emojis, slang, audio, technical journals, handwritten notes, etc) the allure of machine translation becomes increasingly untenable. In other words, the corpus of data that is made available rarely presents themselves as a neatly, linear oriented email thread of communications. It’s quite messy, as high stakes litigation and investigations typically are. I would go even further if you add social media and other web-based dynamic data, not only can collection be challenging but the interpretation of the data that is collected.
So even if the word or phrase can be identified and “cleaned up” in post editing (assuming it’s even flagged) these data points appear in various unassuming places throughout corpus of ESI. Part of my role is to identify areas of concern, leveraging available technology to drill deep into certain areas that machines would have initially viewed as innocuous.
Ultimately, in a legal case, the legally significant translation should be consciously chosen or avoided. A machine translation, however, does not identify the most legally significant translation of a given phrase. It simply seeks to provide the “correct” translation according to its algorithm. Nonetheless, this potential piece of significant evidence could be overlooked, creating a certain feedback loop sending the foreign language review team and thus the case team down the wrong rabbit hole.
Some Additional Words of Confusion
A further challenge for a language like Japanese are that frequently, there are also words or phrases that have opposite meanings depending on context. An example would be the term bimyou (微妙). Bimyou could, depending on context, mean anything from “precise,” “subtle,” “undecided,” “yes and no,” “awkward,” “uncomfortable,” “I can’t say anything to that,” or “mixed feelings.”
It is important to understand that the contextual nature of Japanese, in this case, often lends itself to more than one “correct” translation. Other languages also pose similar issues. Multiple interpretations, as shown above, can be possible for a given phrase or paragraph, depending on the surrounding context. Additionally, cultural knowledge and expertise can be a key element in identifying potential issues. When dealing with a multi-country investigation, the trifecta of culture, language and legal make a big difference. Again, I emphasize the importance in maintaining the strong leadership skills of a Project Manager that exhibits the skills outlined above.
When Precision Matters: FCPA & Antitrust is no “Walk in the Park”
For example, I worked on an FCPA matter wherein identifying suspect gifts was a key issue. A member of my team noted that some of the names on receipts being submitted for reimbursement for dinner didn’t sound like restaurants. Upon further investigation, it was discovered that the purported “restaurant” listed on numerous receipts was a call girl service.
There was also an instance where a supervisor inquired if the official had been treated to “wakame” – seaweed. A person less familiar with Japanese culture might have assumed that this referred to food. The reviewer correctly identified it as slang for a part of a woman’s anatomy—and thus further reference to the call girl service. In other words, key insights gleaned from a large, complex case presents useful pieces of evidence across many types of documents in fragments. It is rarely in plain sight.
In these cases, and numerous others throughout my career, cultural knowledge of Japan has repeatedly turned out to be of key importance in FCPA cases.
Alternatively, in antitrust cases, it is not uncommon for defendants to use various code words or phrases. Golf terminology, slang, pop culture references, baseball references, gambling terminology, for example may be used. Furthermore, a lot of the useful evidence occurs over chat networks such as Line in Japan. As with anybody who is familiar with text messaging lingo, the language of text-based messaging is sometimes not even understood among English speakers. (FOMO, LMAO, IMHO, YOLO, LMK, etc)
I have personally seen spreadsheets with golf terminology that appear innocent on its surface turn out to be of key importance. The golf terminology in that particular case was a word play that implied the existence of a deal. The word was an extremely unusual choice in this sentence, which alerted the bilingual attorney examining the document to the alternative meaning. Often when a machine translation encounters an odd wording choice, software tries to make the sentence appear ‘normal’ or grammatically correct. By reading the phrase in the original Japanese, however, the legal significance of the phrase was apparent.
Had the document been presented with only the English version viewed by an American (or English-speaking Attorney) the document’s significance would have been entirely “lost in translation.”
Machine Learning, AI and Neural Network Translation—Not a Silver Bullet
Another argument for machine translation has been that these issues can be addressed through machine learning. Proponents of this approach argue that machine translations can be tweaked to improve over time and learn the quirks of a given set of facts, so that the machine translation can adequately translate the documents.
While this process can gradually improve the quality of the translations, it does not provide a “silver bullet” solution to the problem posed by machine translations’ inability to understand legal significance or easily identify odd usage of language. The problem lies in what is often the unique character of such documents.
It is common that in many major non-English document review projects, tens of thousands of documents, if not hundreds of thousands of documents require review. Among those documents, it is not uncommon that documents of truly high significance, that could potentially move the litigation’s chances one way or another amount to as few as a dozen documents or less. Also, the cases are very rarely simple black and white decisions. There are multiple levels of analysis, tagging schemas and various litigation matters (anticipated or filed) that require a numerous decisions at the document level.
While Statistical methods of translation has been replaced and improved upon by the Neural Network translation systems offered by many MT software developers, from my vantage deep in the weeds of complex Japanese-language E-Discovery, I operate very cautiously when MT solutions are presented to me. This is the case because common patterns of words or phrases are “taught” to an AI to understand how they should be translated, are problematic when the “samples” to be provided amount to what are often highly unique documents representing 0.01% of the total dataset.
In other words, a machine-learning based approach to machine translation does not eliminate the problem of dual meanings obscuring legally significant documents. As such, documents tend to be a tiny proportion of the overall dataset, and they are often made significant because they make use of unique or atypical usage of language. As a result, machine learning tends to be of minimal aid in making improvements to a translation algorithm in finding or appropriately translating such documents.
Conclusion: Understanding the Risks
None of this is to say that Machine translation cannot have a place in streamlining reviews and reducing costs. The simpler the factual issues, the fewer risks machine translations would present. My involvement has been on the side of more complex engagements that has placed a higher premium in my skills as a Bilingual Attorney Project Manager on large-scale foreign language document reviews. This is akin to hiring the most skilled and premier law firm. Yes, they will likely charge more per hour, but they will get the work done. We are valued for our hands-on approach and keen awareness of the problems at hand. Sure, we’ll leverage and optimize technology WHERE APPROPRIATE, but we have no qualms with getting our hands dirty.
A savvy Attorney needs to evaluate the risks presented by working through a machine translation. It generally cannot tell you when alternative similarly correct translations are possible—and it most certainly cannot tell you when some of those alternatives have greater legal significance. A truly informed weighing of the costs of a thorough bilingual analysis against the value it provides in mitigating litigation risk is now a necessary part of preparing for the discovery process in international cross border matters.
