Prior to The CJK Group, I had the opportunity to work for a variety of different agencies on Japanese eDiscovery Projects as a contract employee.
One thing that was shockingly common was to walk into a room, turn to the person next to me and say, “Hi, nice to meet you” in Japanese, and for the person to respond “Oh, I can’t actually speak in Japanese.” This was a Japanese review and the eDiscovery managed review provider was retained by a major US law firm to provide high quality Japanese document review. To be frank, this is something that occurs in virtually all foreign language reviews. Trust me, I know. I’ve been involved in tens of thousands of billable hours of foreign language review. The stories I can tell…
I would often think, how is this possible, let alone common?
A major factor is blind reliance on the ALTA test to determine linguistic competence among reviewers assigned to non-English projects. I’ll cover some other reasons for this but let’s dive into the ALTA test for now.
ALTA is a language proficiency testing service (ALTA Language Services, Inc.) that can test for various proficiency levels and is used to test language qualification for bilingual customer service representatives, nurse practitioners, law enforcement officers, sales representatives, flight attendants, personal bankers, physicians, government linguists, and legal document reviewers.
The ALTA test is a short multiple choice language test that can be completed in under an hour by most test takers. It requires that the test taker read a short passage in the target language, then answer a series of multiple-choice questions designed to test comprehension of that short passage.
For purposes of testing for competence in foreign language review, this format presents several problems.
First, the time requirements of reading the passage are quite generous. A reviewer who can read a short email but takes 4-5 minutes (instead of 30 seconds) to read it is not qualified to conduct review.
Second, as the test is not proctored and administered largely in an “honor system” environment, there are ways for the unscrupulous to engage in cooperative test taking, sharing of answers, etc. The incentive in foreign language pay, which is typically double or triple the rate of English document review, provides the motivation. Think $30 USD versus $95 USD. If you can get on enough of these reviews, you could make more than the Associate at Big Law.
This is not to say the ALTA test is not of value—it can provide a valuable gatekeeper role in excluding from considerations individuals who are clearly unqualified to conduct a foreign language eDiscovery reviewer role.
However, the ALTA test cannot take the place of a bilingual manager who can evaluate the work provided by the eDiscovery reviewers on an ongoing basis. To place blind faith in the ALTA test results as a marker of linguistic competence can be dangerous.
To make matters worse, the typical review model assigns an English-only Project Manager to manage a foreign language review. For obvious reasons, these managers cannot personally evaluate the quality of the work of a foreign language reviewer.
These managers can only evaluate the quality of the work of their team members by reliance on someone that speaks the language—but they have no personal ability to evaluate substantive review-based decisions. They do not speak or read the language and have no independent mechanism to examine the original electronic file in its native language.
While the manager can choose someone on the team to quality control the rest of the team and report back, the manager is incapable of personally verifying whether that person is competent.
I once worked on a team where the English-speaking manager had placed a person in charge of overseeing the rest of the team who clearly did not speak or know Japanese very well. Since the Project Manager spoke no Japanese, it was months before this error in personnel was discovered, requiring rechecking tens of thousands of documents at major client expense.
The ALTA test is a useful first step in evaluating the linguistic competency of an eDiscovery team member, but it can be a high-risk decision to rely upon it blindly. Be very careful and ask many questions if your vendor has only an ALTA test. It takes much more than an ALTA test to determine foreign language review aptitude.
