Haocun Ye — Homepage · 叶浩存 | Multimodal LLMs, ICT CAS

01News

Recent news

Aug 2026
CRAFT — The Diagnosis Is Already in the Notes — accepted to Findings of EMNLP 2026 (first author).
Aug 2026
KnowMeBenchV2 accepted to the EMNLP 2026 main conference (co-author).
2025
Received the ICT Outstanding Master’s Student Award.
2025
Invention patent granted: a training method for large models in ophthalmology.
Feb 2025
VersaFusion presented at AAAI 2025 (first author).

02Publications

Selected publications

Learning Without Looking: Image-Dependent Gains from Image-Free RLVR

Haocun Ye et al.
In preparation
When RLVR lifts a vision-language model’s visual-reasoning scores, is the model learning to see — or just learning to take the test? Standard benchmarks can’t tell the two apart. A controlled train × test visual-visibility design separates the sources of the gain, and FlipTrack — paired images differing in one key visual fact, scored so the answer must flip — makes text-prior-only strategies score zero by construction. The recurring pattern: scores can rise while reliance on the image quietly degrades. Code and benchmark will be released with the paper.

The Diagnosis Is Already in the Notes: A Channel-Wise Audit of Diagnostic Shortcuts in Clinical-Text Benchmarks

Haocun Ye, Xinlong Jiang, Yuanyuan Hu, Shubai Chen, Tingyu Wu, Zhenkun Zheng, Caijiao Yi, Teng Zhang, Liuxin Wang, Tong Zhao, Heping Xu, Weiwei Dai, Yiqiang Chen

Findings of EMNLP 2026

Diagnosis-redacted clinical records still recover the diagnosis at 0.944 macro-AUROC. CRAFT is a portable audit protocol that reports each channel’s shortcut floor, so fusion gains are read against what the text alone already gives away.

[Code] [Blog]

[BibTeX]

@inproceedings{ye2026diagnosis,
  title     = {The Diagnosis Is Already in the Notes: A Channel-Wise
               Audit of Diagnostic Shortcuts in Clinical-Text Benchmarks},
  author    = {Ye, Haocun and Jiang, Xinlong and Hu, Yuanyuan and Chen, Shubai
               and Wu, Tingyu and Zheng, Zhenkun and Yi, Caijiao and Zhang, Teng
               and Wang, Liuxin and Zhao, Tong and Xu, Heping and Dai, Weiwei
               and Chen, Yiqiang},
  booktitle = {Findings of the Association for Computational Linguistics:
               EMNLP 2026},
  year      = {2026},
  note      = {Code: \url{https://github.com/yhc98002-bit/The-Diagnosis-Is-Already-in-the-Notes}}
}

KnowMeBenchV2 overview: five-task ladder from information extraction to grounded abstention, with per-task diagnostic results

KnowMeBenchV2: Evidence-Grounded Person-Centric Long-Video Understanding

Tingyu Wu, Guangyu Cao, Qizhen Lan, Zhisheng Chen, Ziyan Weng, Bingkun Zhu, Miao Su, Chenglong Li, Zhengwei Xie, Huacan Wang, Sen Hu, Haocun Ye, Nan An, Zaoqu Liu, Ronghao Chen

EMNLP 2026

An evaluation-only benchmark for evidence-grounded, person-centric long-video understanding.

[BibTeX]

@inproceedings{knowmebenchv2_2026,
  title     = {KnowMeBenchV2: Evidence-Grounded Person-Centric
               Long-Video Understanding},
  author    = {Wu, Tingyu and Cao, Guangyu and Lan, Qizhen and Chen, Zhisheng
               and Weng, Ziyan and Zhu, Bingkun and Su, Miao and Li, Chenglong
               and Xie, Zhengwei and Wang, Huacan and Hu, Sen and Ye, Haocun
               and An, Nan and Liu, Zaoqu and Chen, Ronghao},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods
               in Natural Language Processing (EMNLP 2026)},
  year      = {2026}
}

VersaFusion: A Versatile Diffusion-Based Framework for Fine-Grained Image Editing and Enhancement

Haocun Ye, Xinlong Jiang, Chenlong Gao, Bingyu Wang, Wuliang Huang, Yiqiang Chen
AAAI 2025
A tuning-free dual-branch framework for fine-grained editing: a guidance branch reconstructs the source and stores its self-attention K/V, a generation branch edits under feature-correspondence guidance and masks, with proximal negative-prompt inversion in place of null-text optimization. One set of weights covers outpainting, resizing, style transfer, drag and face editing. AAAI 39(9), 9445–9453.
[Paper] [DOI] [Blog]
[BibTeX]

@inproceedings{ye2025versafusion, title = {VersaFusion: A Versatile Diffusion-Based Framework for Fine-Grained Image Editing and Enhancement}, author = {Ye, Haocun and Jiang, Xinlong and Gao, Chenlong and Wang, Bingyu and Huang, Wuliang and Chen, Yiqiang}, booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence}, volume = {39}, number = {9}, pages = {9445--9453}, year = {2025}, doi = {10.1609/aaai.v39i9.33023}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/33023} }

Patent · Granted A Training Method for Large Models in Ophthalmology — invention patent behind the ophthalmology LLM training pipeline.

03Projects

Things I’ve built

01 / 04

Learning Without Looking · FlipTrack

Is RLVR teaching the model to see, or to take the test? Training Qwen2.5-VL with GRPO under four visual conditions — real image, caption, text-only, gray placeholder — and testing each with and without images separates where the gains come from. FlipTrack, a counterfactual benchmark of paired images whose answers must flip, scores text-prior-only strategies at zero by construction. The pattern: benchmark scores can rise while the model’s reliance on the image quietly degrades.

RLVR
GRPO
Qwen2.5-VL
Benchmark design

02 / 04

CRAFT — The Diagnosis Is Already in the Notes

Does image–text fusion actually use the image? On 5,904 real visits across Aier Eye Hospital sites, a deliberately weak probe — frozen encoder, one linear head — still recovers the diagnosis from diagnosis-redacted records at 0.944 macro-AUROC; even a view keeping only which fields were filled in reaches 0.854, because which exams were ordered is itself a clue. CRAFT packages this as a portable audit: report each channel’s shortcut floor before claiming a fusion gain. I led the trustworthy-evaluation work; the code is open-source and is becoming a pre-release dataset check.

Clinical NLP
Shortcut learning
Benchmark audit

Code → Blog post →

03 / 04

Ophthalmology LLM Training Pipeline

General-purpose LLMs answer ophthalmic questions vaguely, and public ophthalmic QA corpora are scarce — so this industry project built the corpus first: 610k items filtered from 11 open medical QA sets, 208k QA pairs generated from 212 OCR’d textbooks, paraphrase-augmented to 9.0M instructions. Then full-parameter SFT of Qwen2.5-14B on 2 × 8 A800s (DeepSpeed ZeRO-3, bf16, global batch 1024, RDMA / NCCL). A four-dimension LLM-judge score — medical accuracy, completeness, clinical usefulness, professionalism — rose from 2.6 to 4.1; the method was granted an invention patent. I was a core member and took the project through its final wrap-up.

SFT
DeepSpeed ZeRO-3
Data pipeline
Qwen2.5-14B

VersaFusion editing examples from the AAAI 2025 paper: seven editing tasks with source inputs and edited outputs

04 / 04

VersaFusion: Tuning-Free Fine-Grained Image Editing

Text-to-image models generate well but edit poorly: text instructions miss their target, and drag-style methods either inherit GAN backbones or fine-tune per image. VersaFusion’s dual branches edit with no tuning at all — 5.23 s preparation versus 63.56 s for DragDiffusion, with the best point-alignment error and FID among general-purpose methods — and one set of weights covers outpainting, resizing, style transfer, drag and face editing. (AAAI 2025)

Diffusion models
Image editing
Stable Diffusion

Paper → Blog post →

04About

Beyond the benchmarks

I’m Haocun Ye, a PhD student in artificial intelligence at the Institute of Computing Technology, Chinese Academy of Sciences, advised by Prof. Yiqiang Chen.

I work on multimodal large models — how they are trained, how they are post-trained with reinforcement learning, and how they should be evaluated. The question that drives most of my work is simple to state and hard to answer: when a benchmark score goes up, did the model actually acquire the ability the benchmark claims to measure?

Away from the lab I like experimenting in the kitchen, and I’m happiest on days off spent with family, travelling with friends, or around a board-game table — ideally with a dog nearby. In quieter hours I play single-player games (GTA, The Witcher 3, Sekiro). I know I have my share of flaws, and I try to keep improving — both in how I work with others and in myself.

Education: PhD student (integrated master’s–PhD track), Institute of Computing Technology, Chinese Academy of Sciences · 2023 – 2028 (expected)
Advisor & lab: Prof. Yiqiang Chen · Beijing Key Laboratory of Mobile Computing and Pervasive Devices
Award: ICT Outstanding Master’s Student Award, 2025
Languages & tools: Python · C++ · Linux / Git · PyTorch · DeepSpeed · Transformers / Diffusers
Training & post-training: Full-parameter / LoRA SFT · multi-node multi-GPU training (ZeRO-3, NCCL) · multimodal RLVR (GRPO / EasyR1) · diffusion inversion & editing
Data & evaluation: LLM-driven data filtering / cleaning / synthesis · benchmark construction & trustworthy evaluation (counterfactual, shortcut & leakage audits) · LLM-as-judge
Models: Qwen3.5 / Qwen2.5 series (incl. VL) · InternVL3 · Gemma-3 · Stable Diffusion

05Contact

Get in touch

Open to research collaboration and conversations about multimodal models, post-training, and evaluation. Email is the fastest way to reach me.

despair_eyes@qq.com

Institutional: yehaocun23s@ict.ac.cn

Haocun Ye叶浩存

Learning Without Looking: Image-Dependent Gains from Image-Free RLVR