AI Music Generation
The fast-moving world of generating music with machine learning — from prompt-to-song web apps like Suno and Udio to open models like MusicGen and Stable Audio you run in code. This hub explains the landscape, compares the tools, and links hands-on setup guides.
Hands-on tutorials
Full walkthroughs for running each open model yourself — install, every generation parameter, troubleshooting and licensing.
Generate Music with MusicGenBoth install routes, every generation parameter, melody conditioning, and getting past the 30-second limit.Generate Full Songs with ACE-StepApache-2.0 songs with vocals — the web UI, the Python API, and retake / repaint / edit.Run Stable Audio Open LocallyStereo 44.1 kHz audio: negative prompts, audio-to-audio, batch sample packs, low-VRAM options.Separate Stems with DemucsEvery model by SDR, the flags that improve quality, the Python API, and stem pipelines.Generate Songs with YuE2The staged plan/synthesise API, editing the ABC score before rendering, and zero-shot covers.Train Your Own RAVE ModelDataset prep, every training config, real-time export, and playing the latent in Max/MSP.
Before you start
Two constraints decide which of these models you can actually use. Both are collected as reference tables.
GPU & VRAM RequirementsWhat every open model actually needs — documented figures separated from estimates.Model Licensing MatrixCode licence vs weights licence for every model, and what non-commercial really rules out.
The tools at a glance
| Tool | Type | Access | Licence / cost | Vocals | Best for |
|---|---|---|---|---|---|
| Suno | Text → song | Web app + API | Subscription | Yes | Full songs with vocals from a prompt |
| Udio | Text → song | Web app | Subscription | Yes | High-fidelity songs with fine control |
| MusicGen | Text / melody → instrumental | Local (Python) | Open, non-commercial weights | No | Self-hosted instrumental generation — tutorial |
| Stable Audio Open | Text → audio | Local (Python) | Open (community licence) | No | Local loops, textures and SFX — tutorial |
| Stable Audio | Text → audio | Web app + API | Subscription | No | Commercially-cleared audio and SFX |
| Riffusion | Text → music | Local (Python) | OpenRAIL-M weights | No | Spectrogram-based experiments (unmaintained) |
| YuE2 | Lyrics → score → song | Local (Python) | Non-commercial weights | Yes | Editable compositions, best open song quality — tutorial |
| ACE-Step | Text → music | Local (Python) | Open (Apache-2.0) | Yes | Fast open generation for products — tutorial |
| Magenta | Symbolic / MIDI | Local (Python) | Open (Apache-2.0) | No | Research and MIDI generation |
| RAVE | Neural synthesis | Local (Python) | Open, non-commercial | No | Real-time timbre transfer in a DAW |
The main kinds of AI music tool
- Text-to-song (with vocals) — describe a song (and often supply lyrics) and get a finished track with singing. Suno, Udio and the open YuE2 and ACE-Step live here.
- Text-to-instrumental / audio — generate instrumental music, loops or sound effects from a description. MusicGen, Stable Audio Open and Riffusion.
- Melody & continuation — condition on a hummed or played melody and let the model orchestrate around it (MusicGen-melody).
- Stem separation — pull an existing track apart into vocals, drums and bass with Demucs or Spleeter. Guide →
- Neural synthesis & timbre transfer — RAVE and DDSP learn an instrument's timbre and let you play through it. Guide →
- Symbolic / MIDI generation — models that write notes rather than audio, like Magenta.
Background reading
- Best AI Music GeneratorsSuno, Udio, MusicGen, Stable Audio and more — picked by what you're trying to do.
- Open-Source AI Music ModelsThe models you can actually run yourself, with licences and hardware needs.
- AI Music Prompt GuideHow to write text-to-music prompts that get the sound you want.
- Suno vs UdioThe two leading text-to-song apps, compared.
How to choose
- Want a finished song fast, no code? Use Suno or Udio.
- Want to run it yourself / build a product? Reach for open models — MusicGen, Stable Audio Open, ACE-Step.
- Sound design and loops? Stable Audio (Open or the web app) is tuned for short audio and SFX.
- Working with existing tracks? Start with stem separation, then remix or resample.
Licensing & ethics. Open code is not the same as open weights: several models (MusicGen, RAVE) release weights under non-commercial licences, so check before selling output. Web services set their own commercial terms per plan. And be transparent when a track is AI-generated — some platforms require it, and listeners increasingly expect it.
See the whole ML & generative category in the directory, or the wider tools directory.