MIT - Critical Data

10/23/2026

Of the 68 published clinical trials of large language model interventions identified in this BMJ systematic review, 43 tested patient-facing health education and 39 enrolled patients as the participants. These are the trials where a confident wrong answer reaches someone with no training to catch it. Yet among the 44 randomised trials, 19% described how harms would be assessed and 26% reported any harms at all. Among the 42 chatbot trials, 31% checked the outputs for harmful content, and 41% of randomised trials included an error analysis. The people on the receiving end were also rarely in the room when the trials were designed. Patient and public involvement was reported in 2% of randomised trials under CONSORT and 7% of all trials under TRIPOD-LLM. Patients helped develop the prompts in 14% of chatbot trials and took part in evaluating the outputs in 25%. A trial can clear ethics review and reach a journal while the only patient role is to be the subject.

The trials also come from a narrow slice of the world. Corresponding authors were based in Asia for 36 of the 68 reports and in North America for 20. Europe accounted for 10. Africa and Oceania had one trial each, and South America had none. Recruitment sites followed the same pattern: one published trial recruited in Africa, none in South America. When we tried to compare reporting completeness by region, Europe, Africa and Oceania had to be pooled into a single group of 12 trials, so the paper can say nothing about how LLM trials are reported across most of the world. The models under test, mostly OpenAI’s GPT family, are already being used by patients on every continent. 

https://www.bmj.com/content/395/bmj-2026-100855

10/21/2026

Journals and conferences should reject papers about prediction models in healthcare that the authors cannot say what they are for. Hundreds of clinical prediction models published, few externally validated, fewer still in use. Behind every one sits an assumption that rarely gets checked, that an accurate forecast will produce a better decision. We used anticoagulation during ECMO to show how it fails. Each morning the ICU team sets an aPTT target that trades bleeding risk against the risk of a clotted circuit. A model trained on those patients learns outcomes that already contain the clinicians’ choices. If patients judged to be at low bleeding risk were given higher targets and later bled, the model learns that low heparin predicts bleeding, the reverse of the physiology, and a clinician who lowers the target in response withholds anticoagulation from patients who were tolerating more of it. No AUC reveals this. Whether acting on a prediction helps anyone is a causal question, and it needs a target trial emulated in existing data before any prospective deployment.

Four questions should be answered before a prediction model paper is sent to reviewers. Which decision is the model meant to inform, and what would a clinician do differently? What harms does that action carry, and above what predicted risk is it worth taking? Has the effect of the altered decision been evaluated, or at least has a study been designed to evaluate it? A field building crystal balls at industrial scale could and should stand to slow down. We need fewer prophets and more experiments. Editors and conference organizers are the ones who can make that happen, one rejection at a time.

https://www.nature.com/articles/s44387-026-00171-6

10/14/2026

10/01/2026

A written examination cannot tell us whether a system is ready to manage patients without clinician review. These tests say nothing about longitudinal accountability, medication reconciliation, escalation of care, or who answers for harm. Patients and communities must help set acceptable error thresholds and have the authority to require modification or withdrawal of systems that fall short.

AI is moving power away from the people who controlled access to knowledge and expertise. For a long time that control kept existing power structures in place, because only gatekeepers could reach what they knew. Now anyone with a phone and an internet connection can. We should welcome this change with open minds and the courage to act on it.

https://jamanetwork.com/journals/jama/fullarticle/2854673

08/27/2026

Medical education should treat AI as an opportunity to reimagine what physicians are for. The curriculum has spent a century selecting for people who can hold and retrieve biomedical facts under pressure, and that selection pressure is what produced the hidden curriculum: deference, compliance, and the individualist framing of patient problems all follow from a training model built around scarce expertise. If the scarcity goes away, the pedagogy loses its justification.

Education in general faces the same reckoning. Once the thing being tested is cheap to generate, assessment has to move toward what a person does with knowledge in a specific context: whose evidence base was used, which populations it excluded, what the model cannot see. We coin this agency “algorithmic literacy” and place it alongside evidence-based medicine.

We predict an overhauling of knowledge gatekeeping. Patients who find an AI more patient, more available, and more willing to explain are not confused about their own experience, and physicians who treat that preference as a threat will lose the battle. The latter is AI’s gift to patients.

What remains for physicians is the work that requires standing in a specific place: the authority to say no to and the agency to redesign a system that is harming patients. A model can generate a structural differential diagnosis, but it cannot testify at a zoning hearing or refuse to discharge a patient into unsafe housing.

https://www.tandfonline.com/doi/full/10.1080/0142159X.2026.2653205

07/07/2026

The first Critical Data Datathon in the Dominican Republic 🇩🇴✨

A weekend of learning, collaboration, and problem-solving alongside incredible doctors, researchers, engineers, and data scientists. While language barriers and different backgrounds brought their own challenges, they also reminded us that innovation happens when people are willing to listen and work together.
It wasn’t perfect, and that’s what made it meaningful. Every conversation taught us something new, and each challenge helped build a stronger foundation for the future.
Looking forward to seeing this community grow and making the next datathon even better ❤️

05/22/2026

Please post:��When you open a chatbot at midnight, worried about a symptom, you are not thinking about network traffic. But your browser is. We captured every HTTP/HTTPS request generated during routine interactions with Claude, ChatGPT, Gemini, and Grok: logging in, starting a chat, sharing a conversation. Between 9% and 36% of those requests went to third-party domains: analytics services, behavioral tracking, in some cases advertising networks. Every platform contacted external analytics during ordinary use. None of this is forbidden. It lives in the terms of service you clicked through.

The gap this study surfaces is not technical. It is a governance failure. These platforms were designed as consumer products, and their data practices follow from that: built for product analytics, not for the weight of what people actually bring to them. Over 40 million people use AI chatbots daily for health questions, many after clinic hours, many in communities where care is hard to reach. They are not patients in any legal sense. That is the problem. HIPAA does not apply because the law was never written for this.

The ask: transparency reports, guidance from medical associations, regulatory frameworks that meet people where they are actually seeking care.

Objective: This study presents an analysis of network traffic across four commercial LLM platforms to document which external domains they contact during routin

04/17/2026

AI as a Catalyst: Reimagining Innovation | May 1–2, 2026

On May 1–2, MIT Critical Data brings AI as a Catalyst to the heart of Silicon Valley, not to celebrate the innovation ecosystem as it stands, but to fundamentally challenge it. This isn’t another tech conference. It’s a provocation. We are gathering founders, funders, clinicians, artists, community voices, and boundary-crossers to reimagine how we create knowledge, how we innovate, how we think, how we communicate, and most importantly, how we connect and relate to one another. Half of our workshops will confront a question the venture world rarely asks out loud: why do we accept a system where 99% of ideas die after billions of dollars of investment and years of blood, sweat, and tears, and why do we keep calling that system “successful”? Could there be a better way to nurture ideas, one that doesn’t treat failure as an acceptable mass casualty event?

We are looking for funders and founders courageous enough to sit with that discomfort and imagine alternatives, and just as urgently, we are looking for the voices that have been historically excluded from the innovation table, the communities most likely to benefit from or be harmed by the technologies being built in their name. If you believe the future of innovation demands not just better tools but better values, join us.

Details and registration:
https://criticaldata.mit.edu/events/san-francisco-2026

03/24/2026

Instead of building AI that knows everything, we should be building AI that makes us better: more humble, more curious, more creative. How can we engineer virtues directly into clinical AI systems, equipping them with self-awareness modules that detect overconfidence, flag uncertainty, and prompt clinicians to seek fresh perspectives rather than passively accept a machine’s verdict?

The implications reach far beyond medicine. If we accept that the purpose of AI is not simply to automate cognition but to catalyze our evolution as a species, then the virtues we encode into these systems matter enormously.

The consortium behind this work practices what it preaches. The initiative spans all the continents except Antarctica, deliberately weaving together students, patients, data scientists, clinicians, social scientists, indigenous knowledge holders, and artists. Ultimately, the biases baked into AI are biases baked into who gets to design it. Let us stop building AI that thinks for us and does stuff for us, and start building AI that helps us humans think together, more wisely, and with the kind of courage our most complex challenges demand.

https://news.mit.edu/2026/creating-humble-ai-0324
Image: MIT News; iStock

03/24/2026

In this paper, we examine how a single vendor came to control the digital backbone of American healthcare. But let us be clear: this is not about tearing down a company. We have nothing against technology, innovation, or AI. We believe deeply in the promise of digital tools to democratize access to expertise, to extend the reach of the best clinical knowledge to communities that have never had it. What we are calling for is the building and bridging of communities so that they gain the agency to shape these technologies rather than simply be shaped by them. When one company controls how health data is captured, exchanged, and monetized, the question is not whether the technology works. It is who it works for, and who gets to decide.

As we develop AI for healthcare, we must be deliberate about the systems that generate and capture the data on which everything downstream depends. The electronic health record is not a neutral tool; it encodes assumptions about whose experiences count, whose pain gets documented, whose outcomes get measured. That is why the entire pipeline, from care delivery to data capture, from data curation to modeling, from validation to deployment and continuous monitoring, must involve a diverse set of actors. This means patients, clinicians, data scientists, ethicists, community health workers, and most importantly, those who have been historically marginalized from the design table. Health data is a public good. Its governance should reflect the communities it is meant to serve, not the commercial priorities of any single entity. The path forward is not less technology. It is more inclusive stewardship of the infrastructure on which all of us depend.

https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001143&?utm_id=plos111&utm_source=internal&utm_medium=email&utm_campaign=author