audiocraft
Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.
Free
About audiocraft
Overview
AudioCraft is an open-source library developed by Facebook Research for audio processing and generation using deep learning. It provides state-of-the-art models such as MusicGen for text-to-music generation, AudioGen for text-to-sound effects, and EnCodec for high-fidelity neural audio compression. Designed for researchers and developers, AudioCraft leverages PyTorch and can be used to create custom audio assets for games, virtual reality, and multimedia applications.
Key Features
- MusicGen: Generates music from textual descriptions with controllable melodic conditioning.
- AudioGen: Produces realistic sound effects from text prompts (e.g., footsteps, ambient noise).
- EnCodec: High-quality neural audio codec for efficient compression and reconstruction.
- Multi Band Diffusion: Enhances EnCodec decoded audio with diffusion-based super-resolution.
- MAGNeT: Non-autoregressive model for faster text-to-music generation.
- Training Pipelines: Complete PyTorch components for training custom audio generation models.
Use Cases
- Dynamic Soundtrack Creation: Generate endless, context-adaptive background music for games.
- Procedural Sound Effects: Create on-the-fly sounds for player actions, environmental changes, or UI feedback.
- Voice-over and Dialogue: Synthesize character voices or narration using fine-tuned models (with additional voice synthesis models).
- Audio Asset Prototyping: Rapidly prototype audio assets during game development without requiring a composer or sound designer.
Use Cases
- ▸Procedural generation of background music for adaptive game soundtracks
- ▸On-demand sound effects generation for player actions and environmental cues
- ▸Rapid audio asset prototyping during game development
- ▸Dialogue and voice-over synthesis for narrative games
- ▸Audio asset compression and streaming optimization
Features
- ✓Generates music from text prompts with melodic conditioning
- ✓Creates realistic sound effects from text descriptions
- ✓State-of-the-art neural audio compression via EnCodec
- ✓Diffusion-based audio decoding for high fidelity
- ✓Non-autoregressive generation for faster inference
- ✓Provides pre-trained models for easy integration
- ✓Supports training custom models on user datasets
Installation
Install audiocraft to get started:
$git clone https://github.com/facebookresearch/audiocraft.git
Or visit the project page for detailed setup instructions.
Similar Tools
tilemapgen
**Overview**
Tilemapgen is a command-line tool that harnesses **Stable Diffusion** to generate isometric dungeon tiles and full tilemaps. It provides a streamlined workflow for creating consistent tile sets by separating texture generation from tile rendering. The tool is designed to be flexible, accommodating various tile geometries beyond isometric when provided with appropriate geometry inputs.
**Key Features**
- **Swatch Generation:** Create texture swatches using custom text prompts and ne
PixelVibe
AI pixel art and sprite generation by Rosebud AI. Generate game-ready pixel art characters, tilesets, and items.
stable-diffusion-webui
Stable Diffusion web UI
Meshy
AI-powered 3D model generation. Create 3D assets from text prompts or 2D images. Export to Unity, Unreal, Blender.
Leonardo AI
AI image generation for game assets. Generate concept art, textures, sprites, and UI elements with fine-tuned models.
Luma AI
Neural radiance fields for game asset capture. Turn photos and videos into 3D models, environments, and textures.