MedsCast at ICML 2026 Structured Data for Health Workshop

MedsCast will participate in ICML 2026 through the Structured Data for Health workshop in Seoul, South Korea.

We will present preliminary research titled Scaling ECG-Language Models: A Comparative Study of PEFT-Based Adaptation Across Backbone Architectures.

Workshop Details

Workshop: Structured Data for Health
Date: Saturday, July 11, 2026
Venue: Hall D2, COEX, Seoul, South Korea

Conference Details

Conference: ICML 2026
Dates: July 6-11, 2026
Venue: COEX Convention & Exhibition Center, Seoul, South Korea

Research Direction

The paper studies how ECG-language models can be adapted with Parameter-Efficient Fine-Tuning (PEFT) across different open language model backbones.

Our setup connects a frozen ECG-pretrained wav2vec2 encoder to open language model backbones through a projection layer, then compares adaptation strategies including projection-only fine-tuning, joint PEFT with LoRA, and few-shot connector PEFT with in-context examples.

Backbones And Adaptation Strategies

The exploratory study compares four open language model backbones across two families and two size tiers: Qwen3.5-9B and Gemma 4-E4B as smaller models, and Qwen3.6-27B and Gemma 4-31B as larger models.

We compare three adaptation strategies: projection-only fine-tuning with the LLM frozen, joint PEFT with connector tuning plus LoRA on the LLM, and few-shot connector PEFT with in-context examples.

Preliminary Findings

Under the tested hyperparameters, projection-only fine-tuning was more stable than joint PEFT for large quantized models, which degraded after cosine learning-rate restarts.

Performance was non-monotonic across the heterogeneous backbones we evaluated. In the preliminary experiments, the 9B model achieved 46.79% exact-match accuracy on PTB-XL, while Gemma 4-31B achieved 74.19 ROUGE-L. In-context examples showed architecture-dependent effects: the 27B few-shot model improved over its projection-only baseline across reported metrics, while the 9B few-shot model traded precision for fluency.

Clinical-AI Relevance

The failure-mode analysis suggests a clinically relevant operating-point difference between backbone families. In the evaluated setting, Qwen models showed higher specificity, while Gemma 4 models showed higher sensitivity.

These differences matter because clinical AI systems are not judged only by aggregate benchmark scores. Operating point, failure mode, uncertainty behavior, and validation population all affect whether a model is useful in a real clinical workflow.

Download the paper to read more.