Inworld Voice Isolator v1: Open-Source AI for Cleaning Audio

Inworld has released Voice Isolator v1, a dual-model, open-source AI for cleaning noisy speech in both recorded and live audio, available on Hugging Face.

By Sci Herald Editorial Team ·

Released on Hugging Face by inworld — view the original study

Inworld Voice Isolator v1: Open-Source AI for Cleaning Audio

TL;DR: Inworld has released Voice Isolator v1, a dual-model AI system for cleaning noisy audio. It offers both offline processing for the highest quality and a real-time version for live streams, all under an open-source license.

Key Facts

Inworld Voice Isolator at a Glance

FeatureSpecification
TaskAudio-to-audio (voice isolation)
Libraryinworld-voice-isolation
LicenseApache-2.0
Primary UseDenoising speech recordings and live audio streams
Model Variantsvoice-isolator-v1 (offline), voice-isolator-v1-realtime

What's New

The Inworld Voice Isolator is a new entry into the open-source audio processing landscape. This initial release, version 1, is specifically designed for denoising—it removes background noise, music, and competing talkers from an audio signal to isolate a primary speaker's voice. A key innovation is the release of two distinct models built on a shared architecture, each optimized for a different use case. This allows developers to choose the right tool for pre-recorded content or low-latency live applications. Future version 2 is planned to add audio restoration capabilities.

How It Compares

The model card provides a direct comparison between its two variants, detailing their intended purposes and operational parameters.

voice-isolator-v1voice-isolator-v1-realtime
made forrecorded audio and anything not latency-sensitive, at the best qualitylive audio, as a stream in your pipeline or production
how it runsoffline: every frame sees its whole 8 s chunka causal stream with 20 ms of look-ahead
parameters64M58M

The offline model processes audio in 8-second chunks, using the full context of each chunk to achieve the highest quality isolation. The real-time model is causal, meaning it processes audio as it arrives with a minimal 20-millisecond look-ahead, making it suitable for live calls and streaming pipelines.

How to Run It

The recommended way to use the Inworld Voice Isolator is through its dedicated Python library, inworld-voice-isolation. The model weights are downloaded automatically on first use. The library supports CUDA for NVIDIA GPUs, Apple's Metal Performance Shaders (MPS) on Apple silicon Macs, and will fall back to CPU if needed.

The process is straightforward for cleaning a pre-recorded file. After installing the package, a simple command line interface (CLI) command processes the audio.

``bash pip install inworld-voice-isolation voice-isolate sample_14.wav sample_14_clean.wav ``

For developers looking to integrate isolation into a Python application, the library provides a simple API.

```python from inworld import voice_isolation

# Isolate a recording offline offline = voice_isolation.VoiceIsolator("inworld/voice-isolator") wav, sample_rate = voice_isolation.load_audio("sample_14.wav") voice = offline.isolate(wav, sample_rate) voice_isolation.save_audio("sample_14_clean.wav", voice, sample_rate, subtype="FLOAT") ```

For real-time processing, such as handling a live microphone feed, the streaming API can be used with the real-time model.

``python live = voice_isolation.VoiceIsolator("inworld/voice-isolator", mode="streaming", lookahead_frames=2) session = live.stream() # Push audio chunks to the session and receive processed output ``

License and Commercial Use

The Inworld Voice Isolator is released under the Apache-2.0 license. This is a permissive open-source license that allows for free commercial use, modification, and distribution. Users must include the appropriate copyright notices and state any significant changes made to the code. Third-party datasets used in training are credited in a separate NOTICE file.

Limitations

According to the model card, this is a denoising model, not a restoration or speaker selection model. It will remove non-speech elements and competing talkers (typically keeping the louder voice), but it will not reconstruct parts of the audio signal that were lost in the original recording, such as missing bandwidth. The card also notes that the models have been tested on high-end NVIDIA data center GPUs (A100, H100, B200, B300) and CPU, but performance on smaller consumer GPUs is not yet confirmed.

Related on Sci Herald

Frequently Asked Questions

Is the Inworld Voice Isolator free for commercial use?

Yes. The model is released under the Apache-2.0 license, which explicitly permits commercial use, modification, and distribution without any royalty fees.

Can I run the voice isolator on a live microphone feed?

Yes, but you must use the voice-isolator-v1-realtime model variant in "streaming" mode. This model is designed for low-latency processing with only 20 ms of look-ahead, making it suitable for real-time applications like live calls.

What is the difference between the two model variants?

The main difference is latency versus quality. The standard v1 model is for offline processing and uses an 8-second context window for each chunk of audio, yielding the highest quality. The v1-realtime model is for streaming and processes audio as it comes in with minimal delay, which is necessary for live applications but may trade off some quality.

Does the model work with music or only speech?

The primary design goal is to isolate a human voice, including non-verbal sounds like laughter and breaths. While it will remove background music, it is not designed for the specific task of separating vocals from a music track, though subtracting the isolated voice from the original mix can be used to extract a residual.

What are the system requirements to run this model?

The model has been tested on high-end NVIDIA data center GPUs (e.g., A100) and on CPU. For optimal performance, a compatible GPU with CUDA support or an Apple-silicon Mac with PyTorch 2.6+ is recommended. The library will automatically use the best available hardware.

Source: inworld/voice-isolator on Hugging Face