Science has an AI problem. This group says it can fix that.- Princeton Engineering

AI has the potential to help doctors find early markers of disease and policymakers avoid decisions that lead to war. But a growing body of evidence has revealed profound flaws in the way machine learning is used in science, a problem that has swept dozens of fields and implicated thousands of erroneous papers.

Now, an interdisciplinary team of 19 researchers, led by Princeton University computer scientists Arvind Narayanan and Sayash Kapoor, has published guidelines for the responsible use of machine learning in science.

“When we move from traditional statistical methods to machine learning methods, there are a lot more ways to shoot yourself in the foot,” said Narayanan, director of Princeton’s Center for Information Technology Policy and professor of computer science. . “If we don’t have an intervention to improve our scientific standards and reporting standards when it comes to machine learning-based science, we run the risk of not just one discipline, but many different scientific disciplines rediscovering these crises one after another.”

The authors say their work is an effort to end this simmering crisis of credibility that threatens to engulf nearly every corner of the research enterprise. An article detailing its guidelines was published on May 1 in the journal Science Advances.

As machine learning has been adopted in virtually every scientific discipline, with no universal standards safeguarding the integrity of these methods, Narayanan said the current crisis, which he calls the reproducibility crisis, could become much more serious than the crisis of replication that emerged in social psychology. more than a decade ago.

The good news is that a simple set of best practices can help solve this new crisis before it gets out of control, according to the authors, who come from computer science, mathematics, social sciences and health research.

“This is a systematic problem with systematic solutions,” said Kapoor, a graduate student working with Narayanan and who organized the effort to produce the new consensus-based checklist.

The checklist focuses on ensuring the integrity of research using machine learning. Science depends on the ability to independently reproduce results and validate claims. Otherwise, new work cannot build reliably on top of the old work and the entire enterprise will collapse. Although other researchers have developed checklists that apply to discipline-specific problems, notably in medicine, the new guidelines start with the underlying methods and apply them to any quantitative discipline.

One of the main conclusions is transparency. The checklist asks researchers to provide detailed descriptions of each machine learning model, including the code, the data used to train and test the model, the hardware specifications used to produce the results, the experimental design, the project objectives and any limitations of the study results. The standards are flexible enough to accommodate a wide range of nuances, including private data sets and complex hardware configurations, according to the authors.

Although the increased stringency of these new standards may delay the publication of any studies, the authors believe that widespread adoption of these standards would increase the overall rate of discovery and innovation, potentially greatly.

“Ultimately, what concerns us is the pace of scientific progress,” said sociologist Emily Cantrell, one of the lead authors, who is pursuing her doctorate. at Princeton. “By ensuring that published articles are of high quality and form a solid foundation on which to build future articles, this potentially accelerates the pace of scientific progress. Focusing on scientific progress itself and not just publishing articles is where our emphasis should really be.”

Kapoor agreed. Mistakes hurt. “On a collective level, it’s just a huge waste of time,” he said. That time costs money. And that money, once wasted, could have catastrophic downstream effects, limiting the types of science that attract funding and investment, harming ventures that are inadvertently built on flawed science, and discouraging untold numbers of young investigators.

In working to reach a consensus on what should be included in the guidelines, the authors stated that they aimed to strike a balance: simple enough to be widely adopted, comprehensive enough to catch as many common errors as possible.

They say researchers could adopt the standards to improve their own work; peer reviewers could use the checklist to evaluate articles; and journals could adopt the standards as a requirement for publication.

“The scientific literature, especially in applied machine learning research, is full of avoidable errors,” Narayanan said. “And we want to help people. We want to keep honest people honest.”

# # #

The paper, “Consensus-Based Recommendations for Machine Learning-Based Science,” published May 1 in Science Advances, included the following authors:

Sayash Kapoor, Princeton University;
Emily Cantrell, Princeton University;
Kenny Peng, Cornell University;
Thanh Hien (Hien) Pham, Princeton University;
Christopher A. Bail, Duke University;
Odd Erik Gundersen, Norwegian University of Science and Technology;
Jake M. Hofman, Microsoft Research;
Jessica Hullman, Northwestern University;
Michael A. Lones, Heriot-Watt University;
Momin M. Malik, Center for Digital Health, Mayo Clinic;
Priyanka Nanayakkara, North West;
Russell A. Poldrack, Stanford University;
Inioluwa Deborah Raji, University of California-Berkeley;
Michael Roberts, University of Cambridge;
Matthew J. Salganik, Princeton University;
Marta Serra-Garcia, University of California-San Diego;
Brandon M. Stewart, Princeton University;
Gilles Vandewiele, University of Ghent; It is
Arvind Narayanan, Princeton University.

IT HURTS