Hunton Insurance Recovery Blog

In the case The Estate of Gene B. Lokken v. UnitedHealth Group, Inc., No. 23-CV-3514 (JRT/SGE) (D. Minn.), the plaintiffs alleged that the defendant insurer had denied claims using an artificial intelligence program without human review. They sought discovery into the insurer’s use of AI. When the insurer refused, they moved to compel and the Minnesota federal court granted the motion. Although this case deals with health insurance, its principles are widely applicable to all other types of insurance. Insurers are increasingly using AI to evaluate or even deny claims without human review. They also use it to challenge policyholders’ expenses as too high. Courts are beginning to allow discovery into how AI was used in the claim process. Accordingly, requests for AI chat files, use policies, and documents concerning oversight of AI should now be a standard part of every policyholder’s discovery requests in coverage litigation.

The Court’s Decision

The plaintiffs sought seven categories of documents related to the insurer’s use of the AI program nH Predict to evaluate claims for post-acute care: “(A) discovery regarding the policies and procedures for post-acute care, (B) the development and use of nH Predict, (C) the business acquisition and financial data regarding the financial benefits of nH Predict, (D) internal and government investigations into UHC’s use of artificial intelligence (‘AI’), (E) naviHealth employee incentives for using nH Predict, (F) any oversight of AI use by UHC, and (G) information regarding the employees issuing the Notices of Medicare Non-Coverage (‘NOMNCs’).” The insurer refused to provide these documents, and the plaintiffs moved to compel.

The court granted some of the requests. The court granted the second and sixth requests in full, and denied the third and fifth requests. It granted the remaining requests in part, ordering the production of documents related to employee training on the use of AI, government investigations (but not internal investigations) into the insurer’s use of AI, and the identities of only certain employees. Notably, the court found “Plaintiffs are entitled to discovery of documents regarding how nH Predict works, its development goals and anticipated benefit, and whether it was designed to supplant physician decision-making.”

Analysis

This is the second recent decision in which a federal court permitted discovery into AI use. As discussed in a recent article by our colleagues, in the case USA v. Heppner, the Southern District of New York recently held that AI chat files are not privileged and are discoverable. A key concern of the court in that case was that the AI program’s terms of use allowed the developer to share the chat files with third parties, including the government. As a result, the user had no reasonable expectation of confidentiality, a requirement for maintaining privilege.

As AI’s role in society expands, it is only natural that it needs to be accounted for in litigation. Parties should now routinely seek discovery into the other side’s AI use, including chat files and policies regarding AI use. They should also take steps to ensure that no sensitive information that they do not want produced in litigation is entered into an AI platform without adequate safeguards. Relatedly, businesses should examine how their document retention and deletion policies apply to AI chats to avoid discovery sanctions for failure to maintain relevant documents.

Specific to insurance coverage disputes, the Lokken decision supports that policyholders should seek discovery of an insurer’s use of AI to evaluate and deny claims, or to challenge a policyholder’s covered costs. Despite narrowing some requests, the court allowed broad discovery into the role of AI in the claim process. While this particular case deals with health insurance, its general reasoning and holdings apply to all types of insurance.

For instance, the Lokken court allowed discovery into whether AI was used to “supplant physician decision-making.” In a case involving property or liability insurance, a policyholder could seek discovery into whether an AI program was used to supplant the adjuster’s decision-making authority. Insurers must carefully consider each claim on its merits, and a policyholder is entitled to know the basis for an insurer’s coverage decision. If an insurer erroneously and unreasonably denies a claim on the basis of AI output, with no or minimal human review, that might be evidence of bad faith in the claims process or in the decision itself. Using a software program and removing human decision-making could violate that rule if the AI program makes errors. For instance, if the denial is based on an AI hallucination that is not caught by a human, a policyholder could use that to argue that the insurer acted in bad faith. Regardless of whether the decision is made by a human, by AI, or both, it must be reasonable and supported by the facts and the policy terms. Discovery into a coverage decision based wholly or in part on AI is therefore material.

Conclusion

Policyholders, and litigants in general, should be considering the role of AI in litigation. It is important that companies be aware that any information that is entered into an AI program and the AI’s output may eventually be subject to disclosure. In particular, privileged information should not be entered into an AI program that shares user data or the company risks waiving privilege. On the other side of the coin, litigants should routinely request the opposing party’s AI chat files, use policies, and related documents in discovery, and depose company employees and Rule 30(b)(6) witnesses about the use of AI in relevant matters.

In the context of insurance, policyholders should explore what role AI played in reviewing and denying their claim. An insurer’s use of AI to deny claims or argue that certain costs are not covered could be evidence of bad faith if the denial was erroneous and no human verified its reasonableness.