Transverb: adapt to "distance" changes without crackly glitches

[sophiapoirier]

Here is my collection of prototypes of different approach to deal with "distance" parameter changes in a more sonically interesting manor. My goals are for it to:

1. no longer glitch in an uninteresting way (just sounding like a crummy bug)
2. do something actually interesting/fun sounding instead

Here there is a new (temporary) parameter for "distance change mode" that let's you choose between five algorithms:

reverse
Move playhead in reverse until reaching the new distance.

distance varispeed
In the buffer, take the period of audio between the current read position to the write position and resample it (stretching or compressing the length as needed) to last the duration of the new "distance" setting, and then copy that resampled audio to the delay buffer between the newly distanced read position and the write position.
The idea inspiring this is the way that analog bucket brigade delays work, where they behave as though they are operating with a fixed storage for the delay audio, and when you lower the delay time, then the same storage is used in full but at a faster rate, so whatever was already in the delay buffer now is pitched higher, but new incoming audio delays at regular speed. And conversely increasing the delay time slows down the existing audio in the delay buffer. for example: (https://www.youtube.com/watch?v=KSaZEqfy0ac&t=1171)

buffer varispeed
Like distance varispeed, but do that transformation to the entire buffer, not just the segment between reader and writer.

looping buffer varispeed
Like buffer varispeed, except that in the case where the distance change is shortening, resampling the entire buffer does not provide enough resampled audio to refill the whole buffer, so keep looping that resampled audio as much as needed for the output to fill the buffer.

ad hoc varispeed
Taking the same resampling rate as used in the previous varispeed modes, rather than rewriting the buffer, offset the active delay head speed with that resampling rate for as long as it takes to reach the target distance.

I am curious to hear your opinions of these! And I will share mine, plus some other findings:

I think that reverse really only succeeds at the goal of eliminating glitches, but it isn't that especially fun or interesting sounding.

I was really happy after implementing distance varispeed, but it was usually glitching at the start of the resampling. I thought that maybe rounding errors for partial sample read positions, or the resampling algorithm that looks back and forward, might not like that new transformation boundary right at the read head, and lead me to implement buffer varispeed. And then looping buffer varispeed as a logical extension.

The buffer resampling algorithms are definitely my favorite sounding. They can get pretty enjoyably chaotic and unpredictable as you move the distance parameter around because the buffer resampling compounds on itself, differently in segments of the buffer upon each change. But then I looked at the CPU meter and saw it peaking while dragging the distance slider around. It depends how big your buffer parameter is set in Transverb, and also what the general audio processing buffer size for plugins is in your hold (small Transverb buffer sizes are less of a hit, and larger host buffer sizes make the hit easier to absorb). This also points to the fact that, if you are dragging the parameter around over time, you will get decidedly different audio results depending on what the host buffer size is, which I don't love.

Much as I really like the sound I got with these, these problems are pretty major. I thought I could make the update rate for distance parameter changes time-fixed, so that behavior doesn't depend on the host sample rate. That's not too hard, but it doesn't solve the performance problem, even if it means fewer full passes of resampling, those will still be big spikes each time they do happen.

Then I tried thinking about how to amortize the resampling. This is certainly possible until you start accumulating new distance changes while still playing through a portion of buffer recently resampled from a distance change. It might be possible to simulate that, like build up a map of rates over time or something, but it hurt my brain to solve that problem. If you have ideas as the doctor of mathematics between us, I'm keen to hear!

Anyway, those serious problems led to ad hoc varispeed mode. I find this one enjoyable and it does address my two goals, but the pitch changes aren't as wild and cumulative, not as fun, but I think it works pretty well.

[sophiapoirier]

Oh and I should have mentioned: reverse and ad hoc varispeed modes are not yet implemented for TOMSOUND.

[sophiapoirier]

...and then after I wrote that comment about how math that's too hard for me and how maybe accumulating rates in a map could be an option, I thought about it more and actually I think that is a viable solution. The other thing I forgot to mention is that you would need to keep invalidating that map at the write position at each sample frame increment, but that could work. I think it might still lose something, but now I am curious to try this. The map basically would just be another float array of maximum buffer size, which is some significant additional memory usage, but at least it would only be need per delay head, not per channel, i.e. the DSP cores could share them. Hmmm I will experiment with this when I get the chance...

[sophiapoirier]

rebased on main for the latest fix there (I found that sometimes when modulating "distance" a bunch on a stereo instance, the right channel DSP core might see a newer value for the distance parameter at audio render time than the left channel had and get very different time results and land at an out-of-phase distance as well)

Also btw I implemented the idea I described in the preceding comment and it's definitely flawed and does not work. Adaptive read speed changes fundamentally cannot be the same as actually resampling lengths of the buffer. I am going to next try to make it performantly viable by amortizing the resampling as needed, though that is a pretty complex task (for which I'm yet to become motivated).

[tom7]

Thanks for trying to fix that glitchy thing. Any of these is an improvement!
I think reverse sounds fun, although it you definitely get a recognizable "reverse audio" sound, which seems a bit undermotivated (nothing else in transverb is reversing sound), especially since it seems to play in reverse regardless of which direction I move the distance.
The ad hoc varispeed does make sense, since the buffer would pitch up as the head moved forward, or down as it moved reverse. The problem I see with this one is that if you're tweaking the parameter, the sound changes quite a bit and then kinda instantly snaps back, which is weird because it's hard to tweak it to get a sound you want.
The distance varispeed option always sounds clicky to me. Maybe there's something wrong with it in this revision?
The buffer varispeed sounds in some ways better than ad hoc (it seems to ease out of the pitch change, which I prefer) but sometimes does some weird stuff I can't understand, particularly if you have a long buffer size and move the distance from say 2500 ms to 50 ms quickly, then after it apparently did the change it "comes back" with some space invader noises, which are funny but confusing.
At first looping buffer varispeed was my favorite, but if you slam it from 2500ms to 0ms, it just gives you some harsh noise, which I think is kinda bad.

It might be the case that I'm getting more or fewer parameter change notifications than you while sliding the slider. Does it reset and resample each time? Or if it's still in the middle of a change, does it just like update the destination?

I might not have wrapped my head around this parameter (it has been a while) but I was sort of surprised not to see one of the options be like "interpolate between the old and new position" e.g. with a Bezier (or I I'm fond of the asymmetric but stateless approach where each step you set current = (desired * alpha) + (current * (1 - alpha)) for some alpha in [0,1]). Does this basically amount to the varispeed options but you are being smarter about resampling? Or something else I don't see?

Other thought: The varispeed options are effectively changing the speed of the read head temporarily, and as you are keen to remind me, the dist parameter doesn't really do much unless the speed is zero anyway. Perhaps there is a version of this that is also modifying the speed parameter as well to make sense of it all? (I guess I am imagining a multi-control where you can control speed and distance together..)

[sophiapoirier]

It is great getting your thoughts and experience with this so far! Some replies and other thoughts inline:

Thanks for trying to fix that glitchy thing. Any of these is an improvement! I think reverse sounds fun, although it you definitely get a recognizable "reverse audio" sound, which seems a bit undermotivated (nothing else in transverb is reversing sound), especially since it seems to play in reverse regardless of which direction I move the distance. The ad hoc varispeed does make sense, since the buffer would pitch up as the head moved forward, or down as it moved reverse. The problem I see with this one is that if you're tweaking the parameter, the sound changes quite a bit and then kinda instantly snaps back, which is weird because it's hard to tweak it to get a sound you want.

It's interesting to hear your initial response to these two approaches, because for me the "unpredictable" algorithms (which is basically all of them other than reverse, to varying degrees) speak more to my interests, but hearing that it's a downside for you, it points that maybe this will be something that does make sense to offer modes for, to suit preferences. Reverse mode is always going to be a relatively unobtrusive mode, and anything catchup/slowdown flavor will I think always get into more unpredictable territory.

The distance varispeed option always sounds clicky to me. Maybe there's something wrong with it in this revision?

No, it's just mediocre. I haven't yet figured out why it is doing this, but I eventually ruled out obvious bugs and it still happened and so it is basically what got me developing the whole-buffer resampling algorithms, because this one always record-crackle-sounding glitched at the start. I guess it's still there in case I get some realization that it's fixable, but it might not be. My best guess is, because actual resampling of the buffer occurs based on read positions, and those are fractional values, something is usually slightly off when you round the resampling amount to fixed samples to write to. It doesn't seem to me like that should sound so consistently bad, but it certainly does behave poorly.

The buffer varispeed sounds in some ways better than ad hoc (it seems to ease out of the pitch change, which I prefer) but sometimes does some weird stuff I can't understand, particularly if you have a long buffer size and move the distance from say 2500 ms to 50 ms quickly, then after it apparently did the change it "comes back" with some space invader noises, which are funny but confusing.

Yeah, this is the stuff that I am enjoying most. :) All of the algorithms with "buffer" in their names are actually resampling portions of the buffer in-place, so it's a destructive rather than just transient thing, meaning that if you have a delay head speed above zero, it is potentially going to repeat that transformed portion of the buffer, and it will continue to live on with feedback (if in use).

And then the other part is that each adjustment of the distance parameter triggers a new round of this resampling, which is where it can start compounding and lead to little bits of "space invaders" sounds in there. This is gated by your host's audio buffer size, so whatever the new parameter you reach on a slider movement at the start of the next buffer, that is what is worked with that time.

It's definitely a problem that it is dependent on host buffer size and, if we continue to support one of the algorithms like this, should be intentionally strided in absolute time and not just sample frames as dictated by the host. But that's a refinement I haven't tried tackling yet.

At first looping buffer varispeed was my favorite, but if you slam it from 2500ms to 0ms, it just gives you some harsh noise, which I think is kinda bad.

Hmm now that I reproduce that, it makes sense. If you have to move resample a very large range to a very small range, it will be speeding up dramatically, and with the looping algorithm, it will keep repeating that tiny sped up segment enough time to fill the original length, i.e. many times leading to a high-pitched grating sound. I agree, this is quite displeasing and I think probably at least is a reason making the non-looping flavor of this better than the looping.

It might be the case that I'm getting more or fewer parameter change notifications than you while sliding the slider. Does it reset and resample each time? Or if it's still in the middle of a change, does it just like update the destination?

I think what you're asking is about the host-audio-buffer-size dependency I described two comments back?

I might not have wrapped my head around this parameter (it has been a while) but I was sort of surprised not to see one of the options be like "interpolate between the old and new position" e.g. with a Bezier (or I I'm fond of the asymmetric but stateless approach where each step you set current = (desired * alpha) + (current * (1 - alpha)) for some alpha in [0,1]). Does this basically amount to the varispeed options but you are being smarter about resampling? Or something else I don't see?

Yeah this is basically what ad hoc varispeed does, though not curved in Bezier or any other fashion.

Other thought: The varispeed options are effectively changing the speed of the read head temporarily, and as you are keen to remind me, the dist parameter doesn't really do much unless the speed is zero anyway. Perhaps there is a version of this that is also modifying the speed parameter as well to make sense of it all? (I guess I am imagining a multi-control where you can control speed and distance together..)

Well to start, I would differentiate the "buffer" varispeed options, which instead of changing the speed of the read head are destructively stretching or compressing the length of a portion of the buffer.

a clumsy set of metaphors to describe the various ways of getting the read head to a specific distance from the write head:
Reverse is where you walk backwards the same way you entered the building until you reach the room you passed through that you now are supposed to be inside.
Ad hoc varispeed is where a guided tour group is roaming the building, and you either passed them or are fell behind, and now you either walk faster to catch up with them or you slow down until they catch up with you after wrapping around the circular building for another lap.
Any of the buffer varispeed modes involve the building itself shifting beneath your feet until you are standing where you should be, and the shifting is accomplish by stretching or squishing areas of the building, like it's an inflatable building or something.
(I feel like there was a better metaphor to be had than a person in a building, but I'm going to just leave it be.)

And yeah, so if it's anything other than unity playback speed, then any target distance reached is totally transient. But maybe the journey towards that moment can be interesting at least? The buffer algorithms all you to reach the target distance immediately, because the terrain reshapes and so then you can move the read head while retaining the same point in the (now stretched or compressed) timeline. All of the other modes operate in a manor where they will eventually hit that target distance, and once they do, whatever transient operation was bringing the read head there is completed.

So that all said, because I guess I wanted to make sure understanding of the existing options is shared, I'm not sure I actually am able to picture what sort of mode you envision where the actual speed parameter would change in tandem? Like how would that look from a user perspective? And would the speed change be momentary or stick?

[sophiapoirier]

I was just thinking about the Bezier curve suggestion, and applying that to ad hoc varispeed, but then I realize that the tricky thing would be determining the alpha value in current = (desired * alpha) + (current * (1 - alpha)) because it is challenging to predict the length of time that will be spent offsetting speed to reach the new distance when regular playhead speed factors and can be changing. 😕