README.md

metadata

title: SonicVerse
emoji: 🖼
colorFrom: purple
colorTo: red
sdk: gradio
sdk_version: 5.25.2
app_file: app.py
pinned: false

🎼 SonicVerse

An interactive demo for SonicVerse, a music captioning model, allowing users to input audio of up to 10 seconds and generate a natural language caption that includes a general description of the music as well as music features such as key, instruments, genre, mood / theme, vocals gender.

---

🚀 Demo

Check out the live Space here:

---

🚀 Samples

Short captions

---

📦 Features

✅ Upload a 10 second music clip and get a caption

✅ Upload a long music clip (upto 1 minute for successful demo) to get a long detailed caption for the whole music clip.

---

🛠️ How to Run Locally

# Clone the repo
git clone https://github.com/AMAAI-Lab/SonicVerse
cd SonicVerse

# Install dependencies
pip install -r requirements.txt

# Alternatively, set up conda environment
conda env create -f environment.yml
conda activate sonicverse

# Run the app
python app.py

---

💡 Usage

To use the app:

1. Select audio clip to input
2. Click the Generate button.
3. See the model’s output below.

---

🧹 Built With

• Hugging Face Spaces
• Gradio
• Mistral 7B
• MERT 95M

---