LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure Profiles
EMNLP 2025: Findings
BibTeX
@inproceedings{ng-etal-2025-lamp,
title = "{L}a{MP}-Cap: Personalized Figure Caption Generation With Multimodal Figure Profiles",
author = "Ng, Ho Yin Sam and
Hsu, Edward and
Anantha Ramakrishnan, Aashish and
Kveton, Branislav and
Lipka, Nedim and
Dernoncourt, Franck and
Lee, Dongwon and
Yu, Tong and
Kim, Sungchul and
Rossi, Ryan A. and
Huang, Ting-Hao Kenneth",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-emnlp.521/",
doi = "10.18653/v1/2025.findings-emnlp.521",
pages = "9818--9832",
ISBN = "979-8-89176-335-7",
abstract = "Figure captions are crucial for helping readers understand and remember a figure{'}s key message. Many models have been developed to generate these captions, helping authors compose better quality captions more easily. Yet, authors almost always need to revise generic AI-generated captions to match their writing style and the domain{'}s style, highlighting the need for personalization. Despite language models' personalization (LaMP) advances, these technologies often focus on text-only settings and rarely address scenarios where both inputs and profiles are multimodal. This paper introduces LaMP-Cap, a dataset for personalized figure caption generation with multimodal figure profiles. For each target figure, LaMP-Cap provides not only the needed inputs, such as figure images, but also up to three other figures from the same document{---}each with its image, caption, and figure-mentioning paragraphs{---}as a profile to characterize the context. Experiments with four LLMs show that using profile information consistently helps generate captions closer to the original author-written ones. Ablation studies reveal that images in the profile are more helpful than figure-mentioning paragraphs, highlighting the advantage of using multimodal profiles over text-only ones."
}