SongSong: A Time Phonograph for Chinese SongCi Music from Thousand of Years Away
| Authors | Jiajia Li et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2602.24071 |
| Download | |
| Categories | cs.SD, cs.CL |
Abstract
Recently, there have been significant advancements in music generation. However, existing models primarily focus on creating modern pop songs, making it challenging to produce ancient music with distinct rhythms and styles, such as ancient Chinese SongCi. In this paper, we introduce SongSong, the first music generation model capable of restoring Chinese SongCi to our knowledge. Our model first predicts the melody from the input SongCi, then separately generates the singing voice and accompaniment based on that melody, and finally combines all elements to create the final piece of music. Additionally, to address the lack of ancient music datasets, we create OpenSongSong, a comprehensive dataset of ancient Chinese SongCi music, featuring 29.9 hours of compositions by various renowned SongCi music masters. To assess SongSong's proficiency in performing SongCi, we randomly select 85 SongCi sentences that were not part of the training set for evaluation against SongSong and music generation platforms such as Suno and SkyMusic. The subjective and objective outcomes indicate that our proposed model achieves leading performance in generating high-quality SongCi music.
Engineering Breakdown
Plain English
SongSong is the first music generation model designed specifically to recreate ancient Chinese SongCi music, addressing a gap where existing models only handle modern pop. The paper introduces a three-stage pipeline: predicting melody from SongCi text, generating singing voice and accompaniment independently based on that melody, then combining them into a final composition. To overcome the scarcity of historical music data, the authors created OpenSongSong, a dataset containing 29.9 hours of SongCi compositions from renowned classical composers. This work demonstrates that domain-specific music generation requires both architectural innovation and dataset curation for stylistically accurate historical music synthesis.
Core Technical Contribution
The core novelty is a domain-specific music generation architecture that decouples melody prediction from voice and accompaniment generation—a departure from end-to-end models that struggle with distinctive ancient musical structures and rhythms. Rather than treating all music generation as a unified problem, SongSong recognizes that SongCi has unique tonal and rhythmic characteristics requiring separate modeling of melodic and instrumental components. The paper's second major contribution is OpenSongSong, the first large-scale dataset of ancient Chinese SongCi with 29.9 hours of professionally composed pieces, which is itself a research artifact that enables future work in historical music understanding. This combination of architecture (modular pipeline) and data (historical music corpus) represents the first systematic approach to ancient music generation.
How It Works
The SongSong pipeline operates in three sequential stages. First, a melody prediction module takes SongCi text (classical Chinese poetry with specific musical constraints) as input and outputs a musical melody that respects both the linguistic rhythm of the text and traditional SongCi melodic patterns—this stage essentially learns the relationship between classical Chinese phonetics and expected pitch contours. Second, two parallel generation modules process the predicted melody: one synthesizes the singing voice (human vocals) following that melody contour, while the other generates orchestral accompaniment that complements it, likely using separate models trained on the OpenSongSong dataset. Finally, a composition module combines the singing voice and accompaniment into a coherent final audio piece, handling synchronization, mixing levels, and ensuring musical cohesion. The key architectural insight is that treating melody prediction, voice synthesis, and accompaniment generation as separate tasks allows each to specialize in the distinct characteristics of SongCi music, rather than forcing a single model to learn all three mappings simultaneously.
Production Impact
For engineers building music generation systems, SongSong demonstrates that domain specialization is critical when moving beyond contemporary music into historical or culturally-specific styles—a generic pop music model will fail on ancient genres with different harmonic structures and rhythmic conventions. In production, adopting this modular approach means building separate inference pipelines for melody, voice, and accompaniment, which increases latency compared to end-to-end models but provides interpretability (you can audit what melody was predicted before generation) and allows independent model updates. The data requirement is substantial: creating OpenSongSong required 29.9 hours of curated historical compositions, meaning you cannot apply this approach to a new historical music genre without significant manual annotation and collection effort. The primary trade-off is architectural complexity (three models instead of one) versus output quality and stylistic authenticity—for production systems targeting cultural authenticity over speed, this is likely worthwhile.
Limitations and When Not to Use This
SongSong's applicability is constrained to SongCi and potentially similar structured poetic-musical forms; it does not generalize to other historical music traditions (Japanese Noh, Indian classical, Western medieval) without retraining and new datasets. The paper does not clearly address how the model handles textual ambiguity in SongCi (the same poem can be set to multiple valid melodies depending on performance tradition), which is a fundamental challenge in classical music that may cause the melody prediction stage to produce deterministic outputs that miss stylistic variation. The reliance on OpenSongSong means performance is bottlenecked by dataset size and quality—29.9 hours, while substantial for a specialized domain, is small compared to modern music generation datasets, potentially limiting the model's ability to learn rare compositional patterns or subtle stylistic nuances. Follow-up work must address controllable generation (letting users specify melodic preferences or instrumentation) and cross-genre transfer learning to make this approach applicable beyond SongCi.
Research Context
This work builds on the recent surge in neural music generation (models like Jukebox, MusicLM, and similar sequence-to-sequence approaches) but identifies a critical limitation: existing models are trained on contemporary Western music corpora and fail to capture non-Western or historical musical structures. SongSong's modular architecture is conceptually related to other music AI systems that separate melody, harmony, and timbre modeling, but the application to ancient Chinese music and the creation of OpenSongSong pushes this direction into new cultural and historical territory. The paper implicitly raises an important research question: whether monolithic end-to-end generative models are appropriate for music styles with strong structural constraints (like SongCi, which has meter and tonal patterns tied to classical Chinese phonology), suggesting that domain-aware decomposition may be necessary for historical music. This opens a research direction in culturally-specific music AI, encouraging the community to build models and datasets for non-Western musical traditions rather than treating Western pop as universal.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::