PubMedVision-Plus

A large-scale medical and biomedical image–caption dataset.

🤗 PubMedVision

⚡ Introduction

PubMedVision-Plus is a large-scale medical image–text dataset built from scientific literature, pairing original figures with expert-authored captions to support medical multimodal pretraining.

• Medical knowledge at scale. Drawing on approximately 15M PubMed papers, the collection is refined through LLM- and VLM-based quality filtering into a 12M high-quality Core.
• Broad visual coverage. From radiology and pathology to microscopy, ophthalmology, and endoscopy, the data span diverse medical specialties, with modality and anatomical-region labels retained in the browsing sample.

License notice: Images and captions come from medical journal articles. Please follow the source journals’ image-use and attribution requirements. See the PMC reuse guidelines.

📚 Data

Each Core record includes a unique ID, a relative image path, and its corresponding caption. The browsing sample additionally retains modality and anatomical-region labels and embeds the images.

Data	Image–caption pairs	Construction
Core	12,444,904	Medical literature collection, followed by LLM/VLM quality filtering.

Original figures and author-written captions are preserved without generative rewriting, avoiding hallucinations introduced by synthetic captions. Filtering targets defective images and problematic image–text pairs.

🚀 Usage

PubMedVision-Plus/
├── README.md
├── core-*.parquet       # IDs, captions, and relative image paths
├── images/images-*.tar # Original image archives
└── sample/sample.parquet # Examples with embedded images

The Core collection contains 12,444,904 pairs. The browsing sample currently contains 50 examples.

Load the image-embedded sample Parquet from the downloaded repository:

from datasets import load_dataset

dataset = load_dataset("parquet", data_files="sample/sample.parquet", split="train")
image, caption = dataset[0]["image"], dataset[0]["caption"]

For the full Core release, extract all image tar shards from the repository root (four archives in parallel):

find images -maxdepth 1 -type f -name '*.tar' -print0 \
  | xargs -0 -P 4 -I {} tar -xf '{}' -C .

Extraction preserves the relative image paths in the Parquet metadata. Load core-*.parquet to access IDs, captions, and the corresponding image paths.

📄 Related work

PubMedVision-Plus is introduced in Incentivizing Medical Vision Capabilities from Large-Scale Multimodal Pre-training (NeurIPS 2026). If you use this dataset, please cite:

@inproceedings{chen2026incentivizing,
  title={Incentivizing Medical Vision Capabilities from Large-Scale Multimodal Pre-training},
  author={Chen, Junying and Cai, Zhenyang and Yang, Yunjin and Cai, Kunyuan and Wang, Rongsheng and Wang, Xidong and Yu, Geyang and Feng, Xiangyi and Wang, Benyou},
  booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
  year={2026}
}