This repository hosts pesto~, a Max/MSP external object designed for streaming pitch estimation using the PESTO model.
PESTO (Pitch Estimation with Self-supervised Transposition-equivariant Objective), developed by Alain Riou et al. at Sony CSL Paris, is a deep learning model for fundamental frequency estimation. This Max wrapper enables musicians, developers, and researchers to integrate PESTO's robust, low-latency (typically under 5ms) pitch tracking directly into their Max patches. Whether for interactive music systems, audio analysis tools, jitter visuals, or live performance.
To use pesto~ with a pre-built release, first download the latest release from the releases page and unzip it into your Max packages folder. For Max 8, this is typically located at:
- macOS:
~/Documents/Max 8/Packages/ - Windows:
C:\Users\[YourUsername]\Documents\Max 8\Packages\
The release comes pre-bundled with a handful of models, 128, 256, 512, and 1024 samples at 44.1kHz sample rate (the default for Max and Ableton). The preprocessing step within the PESTO model is sensitive to sample rate, and to avoid any extra overhead I chose not to include any sample rate conversion. If you are using a different sample rate, or would just like a different chunk size, you can very easily export more scripted models from the original PESTO repository. See the Exporting Models section below for more details.
To use pesto~, create a new object in Max with the syntax pesto~ <chunk_size>. The chunk size determines how many samples are processed at once - smaller chunks reduce latency while larger chunks improve accuracy. We recommend values between 128 samples (minimum stable size) and 1024 samples (acceptable latency for most applications). You must specify a chunk size argument, though using 0 will automatically select the smallest available chunk size.
pesto~ has three outlets:
- Pitch: Outputs the estimated midi pitch value.
- Confidence: Outputs the confidence of the pitch estimation, ranging from 0 to 1.
- Amplitude: Outputs an continuous note amplitude.
You can change settings during runtime by sending these messages to the object:
chunk <chunk_size>to adjust the processing chunk sizemodel <modelname.pt>to load a specific model file
pesto~ will continuously output pitch, even if the confidence and amplitude are both very low, so we also include a couple of useful attributes: @conf <value> and @amp <value>. These provide automatic confidence and amplitude thresholding, returning a heavily negative midi value from the pitch outlet when the confidence or amplitude is below the specified value.
All functionallity is available in the reference, and an example help patch is included in the help folder.
If you encounter issues with Max not loading the external on an Apple Silicon Mac due to security restrictions, you may need to unquarantine the .mxo file. This can be done within the pop-up, alternatively you can open Terminal and run:
xattr -r -d com.apple.quarantine externals/pesto~.mxo/Contents/MacOS/* To use pesto~ with different sample rates or chunk sizes not included in the pre-built release, you need to export new JIT (TorchScript) models from the original PESTO repository. Follow the instructions below or refer to the rea
8000
dme in the original repository for more details.
- Clone the Official PESTO Repository:
Clone the official PESTO repository:
git clone https://github.com/SonyCSLParis/pesto.git cd pesto - Export the Model:
Run the
export_jit.pyscript, specifying the checkpoint, your desired sample rate (in Hz) and chunk size (hop size in samples). For example, to export a model for a 44.1kHz sample rate and a chunk size of 512 samples:This will create apython -m realtime.export_jit 'mir-1k_g7' --sr 44100 --hop 512.ptfile in thepesto/assets/directory, named according to the convention<DATE>_sr<SAMPLE_RATE_IN_KHZ>k_h<CHUNK_SIZE>.pt(e.g.,20250528_sr44k_h512.pt). - Place Models:
Move the exported
.ptmodel file(s) into themodelsfolder within thispesto~package's directory (e.g.,.../Packages/pesto/models/). - Load the Model in Max:
In Max, you can load the new model by sending the message
model <modelname.pt>to thepesto~object.
Note! in the load_model() function in export_jit.py, you may need to add the argument mirror=1.0, This centres the incoming chunks in the CQT window to ensure minimal latency!
Building pesto~ from source involves cloning the repository, downloading dependencies (LibTorch and CMake), and then running the specific CMake commands to your OS.
If you haven't already, clone this repository to your local machine. It's recommended to clone it directly into your Max Packages folder. For Max 8, the typical locations are:
- macOS:
~/Documents/Max 8/Packages/ - Windows:
C:\Users\[YourUsername]\Documents\Max 8\Packages\
Open your terminal or command prompt, navigate to your chosen directory (e.g., inside Max 8/Packages/), and run the following command to clone the repository and its submodules:
git clone https://github.com/QosmoInc/pesto_tilde.git --recursiveDownload the appropriate LibTorch distribution for your architecture from the PyTorch website.
- Apple Silicon Macs: Download the latest "macOS - Arm64" version.
- Windows: Download the "Win - shared" version.
Extract the downloaded LibTorch archive and place the libtorch folder into the root directory of this project (e.g., ...\Packages\pesto\libtorch).
Open a terminal or command prompt in the packages's root directory.
For Apple Silicon Macs:
mkdir build
cd build
cmake -DCMAKE_OSX_ARCHITECTURES=arm64 ..
makeFor Windows (Visual Studio):
Ensure you have CMake and Visual Studio (with C++ development tools) installed.
mkdir build
cd build
cmake -G "Visual Studio 17 2022" -A x64 ..
cmake --build . --config Release(Adjust the Visual Studio version in the generator flag -G if you have a different version installed.)
This will generate the pesto~.mxo (macOS) or pesto~.mxe64 (Windows) file in the externals folder and move the appropriate .dll or .dylib files. If you cloned the repository into your Max 8 packages folder, the external should be immediately available in Max. Otherwise, copy the externals, models, docs and help (and libs on Windows) folders to a new folder in your Max 8 packages folder.
Contributions, bug reports, and optimisations are welcome! Especially builds for Intel Mac or Linux! Please feel free to open an issue or submit a pull request.
- PESTO Model: Alain Riou, Stefan Lattner, Gaëtan Hadjeres, and Geoffroy Peeters. "PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective." ISMIR - International Society for Music Information Retrieval. 2023. https://github.com/SonyCSLParis/pesto