StreamMUSE v0 accepted by RTAS 2026
The v0 paper, Real-Time Language Model Jamming: A Case Study for Live Music Accompaniment Generation, was accepted by RTAS 2026.
StreamMUSE
Real-Time Music Accompaniment Generation System
StreamMUSE studies how language-model generation can stay synchronized with a live musical stream, producing accompaniment that is both timely and musically coherent.
Goal
Real-time accompaniment generation as a systems problem
StreamMUSE focuses on generation that reacts to a live musical stream while staying aligned with the musical clock.
StreamMUSE explores live music accompaniment as a real-time generation problem. Instead of generating an entire accompaniment offline, the system must continuously listen to the incoming musical context and produce the next accompaniment frames on time.
This makes the task both musical and system-oriented: the output should be coherent with the melody and previous accompaniment, while the inference loop must stay responsive under latency constraints.
Our goal is to understand this tradeoff and design a system that satisfies real-time constraints while achieving the best possible musical quality.
News
Latest updates
Versions
Project versions
Each version has its own page for system notes, media, MIDI examples, and publication status.
Demos
Recorded demos and media slots
Local videos, YouTube/Bilibili embeds, and MIDI assets share one data-driven media structure.
Video
Real-time demo 1
Real-time demo 2
Real-time demo 3
Real-time demo 4
Publication
StreamMUSE v0
The first accepted StreamMUSE paper.
Real-Time Language Model Jamming: A Case Study for Live Music Accompaniment Generation
2026 IEEE 32nd Real-Time and Embedded Technology and Applications Symposium (RTAS)
Bowen Zheng, Andrew H. Yang, Jiaqi Ruan, Jia He, Xinyue Li, Yuan-Hsin Chen, Ziyu Wang, Xiaosong Ma
@inproceedings{zheng2026realtime,
title = {Real-Time Language Model Jamming: A Case Study for Live Music Accompaniment Generation},
author = {Zheng, Bowen and Yang, Andrew H. and Ruan, Jiaqi and He, Jia and Li, Xinyue and Chen, Yuan-Hsin and Wang, Ziyu and Ma, Xiaosong},
booktitle = {2026 IEEE 32nd Real-Time and Embedded Technology and Applications Symposium (RTAS)},
year = {2026},
doi = {10.1109/RTAS68450.2026.00032}
}