When sycophancy and bias meet medicine

Once upon a time, two villagers visited the fabled Mullah Nasreddin. They hoped that the Sufi philosopher, famed for his acerbic wisdom, could mediate a dispute that had driven a wedge between them. Nasreddin listened patiently to the first villager’s version of the story and, upon its conclusion, exclaimed, “You are absolutely right!” The second villager then presented his case. After hearing him out, Nasreddin again responded, “You are absolutely right!” An observant bystander, confused by Nasreddin’s proclamations, interjected, “But Mullah, they can’t both be right.” Nasreddin paused, regarding the bystander for a moment before replying, “You are absolutely right, too!”

In late May, the White House’s first “Make America Healthy Again” (MAHA) report was criticized for citing multiple research studies that did not exist. Fabricated citations like these are common in the outputs of generative artificial intelligence based on large language models, or LLMs. LLMs have presented plausible-sounding sources, catchy titles, or even false data to craft their conclusions. Here, the White House pushed back on the journalists who first broke the story before admitting to “minor citation errors.”

It is ironic that fake citations were used to support a principal recommendation of the MAHA report: addressing the health research sector’s “replication crisis,” wherein scientists’ findings often cannot be reproduced by other independent teams.

Read full article

Comments