Proposal: add client for NMFMorphing from live input #13
Conversation
|
Hello @elgiano , thanks for the PR! It looks like a cool idea, so we will review shortly. I take it you have tested this in SuperCollider? |
|
Yes, I tested it in SuperCollider and it works beautifully. By the way, one thing that I found also for NMFMorph is that to have good results (aka not so much crackle which I guess comes from phase estimation) I need a hopSize of at least fftSize/8. And another thing, here is the sc class, which I based on FluidNMFMorph, with two comments: FluidNMFMorphIn : FluidRTUGen {
*ar { arg in, source = -1, target = -1, autoassign = 1, interp = 0, windowSize = 1024, hopSize = -1, fftSize = -1, maxFFTSize = 16384;
// 1. here I use .asUGenInput so that I can pass Buffer objects directly, for instance when using Ndefs for quick testing
// MultiOutUGen subclasses do it by default on all arguments in *multiNew, normal UGen doesn't
source = source.asUGenInput ?? {-1};
target = target.asUGenInput ?? {-1};
^this.new1('audio', in.asAudioRateInput(this), source, target, autoassign, interp, windowSize, hopSize, fftSize, maxFFTSize);
}
init {arg ...theInputs;
inputs = theInputs;
// 2. I don't really know what specialIndex is for, but setting it to -1 wouldn't work here (all parameters would be shifted)
// specialIndex = -1;
}
checkInputs {
if(inputs.last.rate != 'scalar') {
^(": maxFFTSize cannot be modulated.");
};
^this.checkValidInputs;
}
}
|
|
Nice, thanks. Yes, the phase synthesis works better with a lot of overlap, although so far we tend to have uniform defaults for all spectral processing. |
|
Fair enough! But I would consider mentioning it in the help files / examples :) |
|
Thanks indeed for this @elgiano - and yes, optimal FFT settings for each algo, with their trade-offs, for various signal types, is something on our radar for the next year where we try to get these concepts across in creative ways. |
|
@tremblap |
|
Oh this idea of internship has a lot of potential indeed! I'll need to speak to the team first, but feel free to contact me on my academic email (p.a.tremblay@hud.ac.uk) with what you have in mind. |
Adds a new client to do NMF Morphing on live audio input
81c73d0 to
fed85fe
Compare
Hi and thanks for this amazing library, I'm really into it and reading some of your papers too! Great work!
I wanted to do NMF morphing on some live input, after having pretrained bases for both source and target. Here I wrote an additional client that combines NMFMatch and NMFMorph, so that it can get activations live from an audio input and multiply them with interpolated bases from source and target.
I thought it could be useful to share it, so here it is. If you like it and you want to pull it I could write a SuperCollider help file for it as well.