BMdataset: A Musicologically Curated LilyPond Dataset
| Authors | Matteo Spanio et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.10628 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Symbolic music research has relied almost exclusively on MIDI-based datasets; text-based engraving formats such as LilyPond remain unexplored for music understanding. We present BMdataset, a musicologically curated dataset of 393 LilyPond scores (2,646 movements) transcribed by experts directly from original Baroque manuscripts, with metadata covering composer, musical form, instrumentation, and sectional attributes. Building on this resource, we introduce LilyBERT (weights can be found at https://huggingface.co/csc-unipd/lilybert), a CodeBERT-based encoder adapted to symbolic music through vocabulary extension with 115 LilyPond-specific tokens and masked language model pre-training. Linear probing on the out-of-domain Mutopia corpus shows that, despite its modest size (~90M tokens), fine-tuning on BMdataset alone outperforms continuous pre-training on the full PDMX corpus (~15B tokens) for both composer and style classification, demonstrating that small, expertly curated datasets can be more effective than large, noisy corpora for music understanding. Combining broad pre-training with domain-specific fine-tuning yields the best results overall (84.3% composer accuracy), confirming that the two data regimes are complementary. We release the dataset, tokenizer, and model to establish a baseline for representation learning on LilyPond.
Engineering Breakdown
Plain English
This paper introduces BMdataset, a curated collection of 393 Baroque music scores in LilyPond format (a text-based music notation language) transcribed by musicology experts from original manuscripts, containing 2,646 movements with rich metadata about composer, form, and instrumentation. The authors built LilyBERT, a BERT-based language model adapted specifically for symbolic music by extending its vocabulary with 115 LilyPond-specific tokens and pre-training on masked language modeling. The key finding is that despite the dataset's modest size (~90M tokens), the pre-trained model shows transferable knowledge when tested on the out-of-domain Mutopia corpus through linear probing, demonstrating that text-based music notation can work effectively with transformer models where MIDI datasets have dominated.
Core Technical Contribution
The core novelty is demonstrating that LilyPond—a human-readable text format for music notation—is a viable and potentially superior alternative to MIDI for symbolic music machine learning, contrary to the field's decade-long dependence on MIDI-based datasets. The authors created the first large-scale expert-curated LilyPond dataset with musicologically meaningful metadata (form, instrumentation, sectional structure), which preserves semantic information that MIDI formats often lose. LilyBERT adapts CodeBERT (designed for programming languages) to music by treating musical notation as a specialized code, extending the tokenizer to handle 115 music-specific tokens, and demonstrating that this approach transfers to unseen corpora. This shifts the paradigm from treating music as time-series audio or pitch sequences to treating it as structured symbolic text, enabling language model techniques that capture higher-level musical structure.
How It Works
The pipeline begins with expert musicologists manually transcribing Baroque manuscripts into LilyPond format, a text-based notation where musical events are represented as commands (e.g., c4 for a C quarter-note, \relative c' { ... } for relative octave positioning). The transcribed scores are paired with rich metadata annotations covering composer identity, musical form (sonata, fugue, etc.), instrumentation, and sectional boundaries—this metadata is critical because it grounds the symbolic representation in musicological meaning. LilyBERT extends CodeBERT's vocabulary from standard tokens to include 115 LilyPond-specific tokens representing musical primitives, then pre-trains a bidirectional encoder using masked language modeling on the full 393-score dataset (~90M tokens total), learning to predict masked LilyPond tokens from context. The resulting encoder produces contextualized embeddings for every token in a score; these embeddings capture both low-level notation patterns and learned musical structure. Evaluation uses linear probing—freezing the pre-trained weights and training a simple linear classifier on top of the embeddings to predict downstream tasks (like form classification or instrumentation)—tested on the held-out Mutopia corpus to assess generalization.
Production Impact
For teams building symbolic music understanding systems, this work eliminates the need to commit exclusively to MIDI; you can now process music directly from LilyPond sources (or convert to LilyPond) and leverage pre-trained transformer embeddings rather than hand-engineering features. The approach dramatically improves interpretability: because LilyPond is human-readable text, you can inspect what the model learned by examining which tokens it attends to, making it easier to debug and validate that the system captures musically meaningful patterns rather than artifacts. Integration is straightforward—LilyBERT is released on Hugging Face with standard transformer APIs, so existing fine-tuning pipelines (LoRA, prompt-based adaptation) work directly. Trade-offs are favorable for musicological applications: the modest dataset size (2,646 movements) keeps training cost low, but the expert curation ensures high-quality labels, making downstream fine-tuning efficient compared to MIDI-based systems that often struggle with annotation quality. However, this is not a drop-in replacement for real-time performance tasks—LilyBERT is optimized for symbolic understanding (form recognition, style classification), not score generation or real-time performance synthesis.
Limitations and When Not to Use This
The dataset is restricted to Baroque music, so the model's learned representations may not transfer well to contemporary classical, jazz, or non-Western music traditions where stylistic conventions differ drastically. The 2,646 movements (~90M tokens) is modest by modern standards; while the authors show transfer to Mutopia, larger pre-training corpora might yield stronger representations. The paper does not evaluate on downstream generative tasks (e.g., score completion or composition), only on classification tasks via linear probing—it's unclear whether the learned embeddings support high-quality music generation or whether they only capture style/form attributes. LilyPond itself is relatively niche and requires active maintenance of a tokenizer; if used for production systems, you depend on LilyPond's syntax stability and community support. The expert transcription process is labor-intensive, limiting scalability; automatic MIDI-to-LilyPond conversion would introduce errors that could degrade pre-training quality, so expanding the dataset requires sustained musicological expertise.
Research Context
Symbolic music research has historically relied on MIDI datasets (MAESTRO, MuseNet training corpora) because MIDI is a de facto standard, but MIDI discards notational intent (e.g., whether a note should be a B-sharp or C-natural) and relies on proprietary encoding schemes. This paper builds on the CodeBERT lineage (NLP models adapted for code understanding) and applies transformer pre-training methodology to music, similar to prior work like Compound Word Transformer for music, but with a focus on notation-preserving formats. BMdataset fills a gap: prior symbolic datasets (Kern corpus, MusicNet) are either small or not curated for representation learning, whereas this dataset combines scale with expert musicological annotation. The work opens a research direction toward notation-preserving models, potentially enabling better transfer learning for downstream music understanding tasks and suggesting that other structured symbolic domains (mathematics, logic) might benefit from similar text-based pre-training approaches.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::