The Wikimedia Foundation weighs in on the NYT's case against OpenAI

The Wikimedia Foundation has weighed in on the copyright lawsuit launched by The New York Times against OpenAI and Microsoft. The people behind Wikipedia have filed an amicus brief urging nuance in the case, which seeks to decide whether or not OpenAI and Microsoft's use of content published in The New York Times for any purpose, including AI model training, is a violation of copyright law.

The New York Times says that companies like OpenAI shouldn't be given free reign over any of their published content. OpenAI and Microsoft say that training and deploying AI models based on published data is necessary for technological development and would therefore fall within the definition of fair use.

The Wikimedia Foundation, however, is seeking a third way. The amicus brief agrees with Microsoft and OpenAI and suggests that a much more restrictive version of fair use could inhibit tech innovation. However, it also says that a ruling in favor of blanket fair use could lead to a closed web where knowledge is weakened due to a "lack of available secondary source materials."

The platform's alternative position applies fair use to two specific use cases for AI companies. This includes certain model training scenarios and something called retrieval-augmented generation (RAG). This is a technique that optimizes the output of large language models (LLMs) by referencing authoritative knowledge bases outside of its training data. However, the organization does not support RAG if an LLM is regurgitating pre-existing articles without attribution.

As for the specifics of model training, the Wikimedia Foundation argues that scraping copyrighted content should be considered fair use if it's being used when "training LLMs that are something new and different from the original sources." It is worth noting that Wikipedia itself is considered one of the highest-quality datasets used to train LLMs.

"A ruling that does not apply fair use principles carefully could threaten the sustainability of free knowledge, and stifle the development of open source technologies, ultimately impacting access to information for everyone in both cases," the Wikimedia Foundation's General Counsel Stephen LaPorte wrote. "The court needs to understand that Wikipedia is created through a collaborative process that requires a strong foundation of open and accessible human knowledge to thrive."