Our latest news:
Project led by the Harvard Medical School and MIT team – 2023
Pancreatic cancer is one of the deadliest cancers, with a 5-year survival rate below 10%. Early diagnosis is extremely difficult, as symptoms are subtle and lesions are often invisible in standard examinations during the initial stages. However, multiparametric MRI (diffusion, T1/T2, perfusion) is beginning to reveal subtle biomarkers, undetectable to the human eye but exploitable by algorithms.
In 2023, a joint team from Massachusetts General Hospital, Harvard Medical School, and MIT’s Computer Science & AI Lab (CSAIL) developed a multimodal deep learning model capable of early pancreatic cancer detection on multiparametric MRI, even before clinical signs appear.
The Model: PanCan-Net
Project led by INSERM & Inria in collaboration with the Health Data Hub – 2024
Patient trajectory modeling is a key challenge in predictive medicine: anticipating a patient’s evolution based on their history allows for earlier treatment adjustments and even prevention of clinical deterioration. However, in practice, these trajectories are often incomplete, noisy, or biased (loss of follow-up, variable interventions, etc.). This is where digital twins become relevant: artificial yet realistic patients evolving in a temporally coherent space aligned with real-world cases.
In 2024, a Franco-European consortium led by INSERM U1138, Inria Saclay, and the Health Data Hub developed a pipeline based on temporal diffusion models, tailored for generating longitudinal clinical trajectories.
The Model: TwinDiff
Launched in 2023 in pilot hospitals in the United States (Cedars-Sinai, UCSF)
Postoperative monitoring is a critical point in the patient journey. Poor management of early complications or side effects can lead to avoidable readmissions. However, caregivers often lack the time to provide intensive follow-up at home. It is in this context that medical conversational agents are emerging, particularly with the advent of specialized healthcare LLMs, capable of conversing with patients, detecting weak signals, and alerting medical teams.
Hippocratic AI, a startup founded by former executives from Google Health, Amazon Alexa, and Epic Systems, developed in 2023 a HIPAA-certified voice AI agent for postoperative telephone monitoring.
The Agent: Hippocratic Care Agent v1.3
A LLM Designed for Clinicians, Not Just Demos
While large general-purpose language models like GPT-4 or Claude demonstrate their power across various tasks, their use in medical environments remains challenging: hallucinations, lack of traceability, and absence of clinical validation make them risky to use. To bridge this gap, the MIT, in collaboration with Mass General Brigham and Harvard Medical School, developed PubMedGPT-X, a language model specifically trained for hospital applications, with one priority: reliability.
This open-source LLM is based on a Transformer-type architecture optimized for a rich biomedical vocabulary. It has been trained on tens of millions of documents: scientific articles from PubMed, clinical guidelines, anonymized hospital records, consultation notes, and therapeutic protocols.
Unlike its general-purpose counterparts, PubMedGPT-X is designed to produce texts aligned with medical standards—structured, contextualized, and verifiable.
In 2024, the model was deployed in real-world conditions in three university hospitals in Boston, within internal medicine, oncology, and geriatrics departments.
Clinicians use it directly through an interface integrated into the Electronic Health Record (EHR). Three use cases were tested:
Early feedback is significant: clinicians save an average of 30 to 40% of time on document writing while producing more standardized, readable, and compliant content according to regulatory requirements.
The project team has implemented an integrated factual verification system: each model output is cross-checked with a medical evidence database (guidelines, referenced publications) and flagged if inconsistencies appear. A confidence score is provided for each response, facilitating human review.
The project is led by Dr. Regina Barzilay, an AI and cancer medicine specialist, in collaboration with Prof. Mark Sendak (Duke Health) for clinical integration. The model is currently being validated for broader deployment in several U.S. public hospitals.
PubMedGPT-X could become a backbone for documentation in healthcare institutions: automatically structuring information, supporting care continuity, and improving traceability in overloaded environments. By opening its code and training data, the team also aims to create an ethical and sovereign standard against closed LLMs dominated by Big Tech.
Understanding a patient as a whole
One of the major challenges in medicine is not only analyzing an image, a biological result, or a clinical note separately, but interpreting them together. Medical practice is inherently multimodal: a diagnosis relies on images, medical history, symptoms, lab results, and sometimes also genetic information or social context. Until now, AI has processed these elements in silos. Today, a new generation of models aims to merge them into a unified representation. This is the ambition of the Med-Flamingo project, led by Google DeepMind in collaboration with the University of Cambridge and King’s College Hospital. The goal: to design a multimodal medical model capable of reading a radiological image, understanding its clinical context, and formulating a hypothesis or response to a medical question in natural language.
Med-Flamingo is based on an architecture derived from Flamingo, DeepMind’s general-purpose multimodal model. Adapted to medicine, it has been trained on a corpus of over 500,000 text-image pairs, including MRIs, X-rays, CT scans annotated with their reports, as well as doctor-patient dialogues.
The model receives an image as input, accompanied by free text (e.g., “62-year-old patient, history of smoking, acute chest pain”) and must produce a response: a diagnostic hypothesis, a therapeutic direction, or an explanation for educational use. In early tests, the model demonstrated an impressive ability to generalize: detecting a pneumothorax on an X-ray, justifying a specific anticoagulant dose, or formulating a response to the patient in simple language.
The team has initiated an experiment in two London hospitals. The model is used as a co-pilot in emergency departments to assist in rapid decision-making. For example, after analyzing an image and the patient’s file, the AI provides a prioritized summary of possible diagnoses, with a confidence level and links to corresponding guidelines. The model is not autonomous: it only intervenes at the clinician’s request and does not issue orders or prescriptions, ensuring compliance with regulatory frameworks. Each response is logged and can be audited later.
The potential is vast, but several obstacles remain. The model needs to learn to manage uncertainty, gaps in the file, variations in image quality, and contradictions between sources. It also needs to adapt to local contexts: the same symptom does not always have the same meaning depending on the country, practices, or populations. Discussions are ongoing with the UK’s NHS to expand the model’s use to general medicine and with the EMA to initiate a regulatory validation framework for multimodal AIs.
With this type of model, we enter a paradigm of global patient understanding, where AI becomes a transversal clinical assistant, capable of contextualizing, prioritizing, and explaining. It is no longer just a technical tool but a reasoning partner, which could transform the way doctors navigate the complexity of modern clinical cases.