Skip to content

Authors Alliance

Northeastern Library, Partners Receive Grant to Deliver AI-Enabled Access to Digital Books

For over a year, the Northeastern University Library has been actively exploring how to use artificial intelligence in ways that advance learning and research, prototyping and implementing helpful connections to library collections and related tools.

A significant gap in this work has been the lack of book collections, so we are thrilled to announce that we are part of a major new grant that will provide students, faculty, and the public with robust, noncommercial forms of AI access to the 19 million books that have been digitized and stored in the nonprofit HathiTrust.

Here is the official announcement of this grant from HathiTrust:

Researchers, students, and the broader public will have new paths into the HathiTrust collection thanks to a Mellon Foundation award to develop responsible, equitable, and transparent services that leverage AI for the public good and scholarly discovery. Two respected partners will join HathiTrust in this effort: Northeastern University Library, which brings deep expertise in digital scholarship, and Authors Alliance, a nonprofit that supports authors who want their work to benefit the public. With a firm commitment to lawful, noncommercial, public-interest uses, this project will make HathiTrust’s 19 million digitized volumes more useful and accessible to the academic community and the public.

Transparent Books: AI Infrastructure for Discovery and the Public Interest responds to the growing need for shared, trusted, academy-led infrastructure enabling sophisticated discovery and research of digitized collections. The project aims to connect readers to relevant titles more efficiently, thereby enhancing their reach and impact, and will consider a range of interests, including publishers, authors, scholars, and members of the public.

Over the course of the $5 million, 5-year grant, HathiTrust and its two grant partners will:

  • Enhance Book Discovery: Apply AI tools to the HathiTrust corpus to improve descriptive metadata (e.g., subjects, languages, summaries) and text quality, and to develop interactive search to facilitate deeper understanding of books, both as individual items and in aggregate.
  • Provide Verified Resources: Offer lawful, legitimate, verified, and transparent sources of text and images from books at scale for academic and noncommercial use cases, for purposes such as text mining, computational analysis, and the creation of specialized language models.
  • Advance Shared Principles: Develop principles for legal frameworks, research integrity, accountability, and environmental impact to serve as a shared foundation for decision-making that centers academic values and the public good.
  • Create Sustainable Services: Develop financially sustainable services that support academic and noncommercial research.

Mike Furlough, HathiTrust’s Executive Director, said, “For decades, libraries have sustained their values in the face of rapid and powerful advances in technology. Libraries created HathiTrust to ensure that the results of mass digitization would address the unique needs of researchers and the academy. This project continues that longstanding effort to collaboratively make the scholarly record more accessible and usable for today and into the future.”

Transparent Books also builds on the 2024-2026 Public Interest Corpus project, led by Authors Alliance and Northeastern University, which examined how research libraries can make book data available for computational research in lawful ways that serve the public interest rather than reinforce commercial control of large-scale digital collections. Through Transparent Books, HathiTrust will use the framework identified in the Public Interest Corpus project to deliver its corpus for noncommercial AI research and development.

Authors Alliance Executive Director Dave Hansen shared, “Our role in Transparent Books is to help articulate principles for use of the corpus, convene the authors, researchers, and institutions the project depends on so their views can be meaningfully reflected in the projects’ designs. Ultimately, we want to lower barriers for noncommercial, academic researchers so they can build models and tools that actually research and benefit the public.  And we want to see this happen in a way that ensures authors’ works and ideas are discovered, attributed, and built upon.”

The project will collaborate closely with academic libraries and researchers to address the complexities of applying artificial intelligence tools to scholarly collections. Creating a path to lawful, legitimate access to in-copyright books data without corporate mediation demands a deep understanding of libraries’ current use of AI tools and their aspirations to offer meaningful engagement with their collections for academic and public communities.

Dan Cohen, Dean of the Library and Professor of History at Northeastern University, highlighted, “Students, faculty, and members of the public who treasure books as rich sources for teaching and learning will value the new kinds of access and use this initiative will provide. At the Northeastern University Library, we have been thoughtfully integrating collections with AI tools to help readers find materials of interest and to enable advanced forms of research, and the lack of digitized books has been a major gap we aim to fill.”

Mellon Foundation Awards $350,000 Grant to Fund Plan for AI Book Training Commons

The Northeastern University Library and Authors Alliance have received a $350,000 grant from the Mellon Foundation to plan a public-interest book training commons for artificial intelligence.

With the increasingly essential role that artificial intelligence plays in society, the importance of including information from books in AI’s large language models becomes more pivotal. The more than 129 million books written over the past 500 years are vital training data sources for AI, providing a quality, breadth, and diversity of content related to human thinking that can strengthen AI’s scope and accuracy.

The main goal of this project is to develop a plan for either establishing a new organization or identifying the relevant criteria for an existing organization (or organizations) to attempt the work of creating and stewarding a large-scale public interest training commons of books.

“We are grateful for the Mellon Foundation’s generous support for this important new project,” said Dean of the Library Dan Cohen. “We and the Authors Alliance are excited to convene writers, publishers, Librarians, technologists, and other stakeholders to explore the best way for books to be incorporated in an ethical and productive way that serves the public.”

Northeastern University Library’s role will be to support and coordinate the project and to host one of the two meetings of stakeholders.

With this work, planners hope to answer key questions, including:

  • What are the right goals and mission for such an effort, taking into account both the long and short-term;
  • What are the technical and logistical challenges that might differ from existing library-led efforts to provide access to collections as data;
  • How to develop a sufficiently large and diverse corpus to offer a reasonable alternative to existing sources;
  • What a public-interest governance structure should look like that takes into account the particular challenges of AI development;
  • How do we, as a collective of stakeholders from authors and publishers to students, scholars, and libraries, sustainably fund such a commons, including a model for long-term sustainability for maintenance, transformation, and growth of the corpus over time;
  • Which combination of legal pathways is acceptable to ensure books are lawfully acquired in a way that minimizes legal challenges;
  • How to respect the interests of authors and rightsholders by accounting for concerns about consent, credit, and compensation; and
  • How to distinguish between the different needs and responsibilities of nonprofit researchers, small market entrants, and large commercial actors.

The Authors Alliance is an organization focused on creating resources and opportunities for authors interested in sharing their work broadly, in the interest of the public good.