1.3.0 - GPU Support
This release introduce whisper.cpp update to 1.5.1, GPU inference support and other minor improvements.
Whisper.cpp updated to 1.5.1
whisper.cpp 1.5.1 got a lot of improvements and bug fixes including better GPU usage.
Check original release notes for more information.
GPU Support
Whisper now supports GPU acceleration. This can drastically improve performance for some hardware.
| Model | CPU | CUDA |
|---|---|---|
| tiny | 1188 ms | 185 ms |
| small | 8992 ms | 517 ms |
| large-v2 | 60325 ms | 1946 ms |
Tests of "jfk.wav" transcribing on Windows with Intel Core i5-12400F and Nvidia Geforce RTX 2070 Super.
| Model | CPU | Metal |
|---|---|---|
| tiny | 1113 ms | 189 ms |
| small | 6319 ms | 860 ms |
| large-v2 | 40608 ms | 3888 ms |
Tests of "jfk.wav" transcribing on Apple M1 Pro.
For Windows and Linux you would need Nvidia GPU and installed CUDA Toolkit (tested with 12.2.0). Unity project compiled with enabled CUDA expects your end-users to have Nvidia GPU and CUDA libraries. Trying to run build without it will result error.
For MacOS you would need ARM CPU, like M1 or newer. iOS Metal inference isn't yet supported. In case of Intel or older hardware, whisper.cpp should fallback to CPU inference.
To activate GPU inference, go to Project Settings => Whisper => Enable CUDA or Enable Metal. For more information, check README.
Other
Ubuntu libs now compiled on Ubuntu 20.04. This might cause problems with Ubuntu 18.04. If you need support for earlier versions of Ubuntu or other distros, consider recompiling libs from source.
New loop mode for microphone was added. It creates a new endless non-stopping stream using Unity build-in circular microphone loop. This is very useful for whisper streaming transcription. To activate it - set Loop in MicrophoneRecord to "true".
What's Changed
- Endless loop microphone and memory leak fix by @Macoron in #55
- Updated whisper.cpp to 1.5.0 by @Macoron in #60
- Add CUDA support for Windows by @Macoron in #61
- Add CUDA support for Linux by @Macoron in #63
- Metal support for MacOS by @Macoron in #64
- Updated whisper.cpp to 1.5.1 by @Macoron in #65
Full Changelog: 1.2.1...1.3.0