Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added an optional audio file parameter to the AudioStream class (issue #264) #317

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jnitsun
Copy link

@jnitsun jnitsun commented Apr 26, 2024

Enhanced the AudioStream class by introducing an optional parameter to accept an audio file.

Fixed #294.

Made alterations to the AudioStream class that now takes in an optional audio file parameter. If the parameter is not specific, it works as it did before.

Enhance the AudioStream class by introducing an optional parameter to accept an audio file. This enhancement allows users to initiate audio streaming directly from a specified file, facilitating more flexible usage scenarios.
@psobot
Copy link
Member

psobot commented Apr 26, 2024

Hi @jnitsun! Thanks for this PR - I always appreciate it when folks send in new code to review.

I know this is your first pull request, so forgive the amount of detail below - these are all friendly notes that I hope will be helpful.

  • You might notice in the "Files changed" view that this PR seems to change a lot of things other than just adding the feature you want to add; for instance, all of the formatting is changed in multiple files. The repo's CONTRIBUTING.md says that we use clang-format (specifically the LLVM style - although it's not listed explicitly there). If you run clang-format -i --style LLVM <filename(s)> before committing, these extraneous style changes should go away.
  • You add two new parameters to the AudioStream class - both of which include documentation (great!) - but you haven't changed the docstring of the AudioStream class itself, so users won't know how to use the new functionality.
  • Your change would technically solve the bug you linked (AudioFile in input of AudioStream #294) - but it changes the AudioStream API in a way that causes a couple of problems:
    • AudioStream has both an input and output, and is designed so that people can process an audio stream through a Pedalboard. In adding this file-playback functionality, the input_audio_stream argument is silently ignored while the provided audio file loops indefinitely. This violates separation of concerns and makes the API confusing for users. It's not intuitive for a newcomer to Pedalboard to expect that the AudioStream class is also capable of reading a file from disk and playing its contents to the output device in a loop; you'd expect the AudioFile class would deal with reading audio from disk instead.
    • This implementation reads the entire audio file into memory before playing it back - which requires enough memory to decode the whole file first. This makes it very easy for users to accidentally run out of memory when playing back large audio files, and is unnecessary as JUCE allows users to stream audio from disk.

I haven't spent much time thinking about a design for this API, but if I were to apply the design principles used for the rest of Pedalboard, I'd suggest a design that allows users to pass audio buffers (i.e.: np.ndarray objects) into an AudioStream for playback:

  • Make the input_audio_device argument Optional[str] = None instead, so that users don't have to pass an input audio device in the case where they want to play back audio directly.
    • Adding this None default parameter will also require the output_device_name to become Optional[str] = None (as positional arguments must come before keyword arguments), and you'll have to add a check in the constructor to throw an explicit error if the output device name is missing.
  • Add an instance method called write to AudioStream that takes an np.ndarray and copies it into an internal thread-safe buffer for playback
    • if input_audio_device was provided, write should throw an exception explaining that this AudioStream object is streaming audio from <device_name> and cannot be written to.
  • While the AudioStream is running, it plays back the available audio in the internal buffer (through the provided plugins)
  • Add a property (with .def_property_readonly) to indicate how many samples remain in its internal buffer, so callers can decide when to provide more samples.
  • The AudioStream close method should be changed to block the close of the stream until the provided buffer is finished playing; and could optionally take a force: bool = False parameter to stop the stream despite it having some buffered data.

Here's an example of how someone might use this API, showing how it would be consistent with the existing Pedalboard API:

with AudioFile("my_file.mp3") as f:
    with AudioStream(output_device_name="MacBook Pro Speakers") as stream:
        while f.tell() < f.frames:
            # Read audio from the file, decode it, and write it to the AudioStream for playback in chunks:
            while stream.samples_buffered < 8192:
                stream.write(f.read(1024))
    # Before this print() is called, stream.close() will implicitly be called,
    # which will block until all audio has been played back.
    print("Playback done!")

I know that the Pull Request you've made here will technically work (and has tests - thank you for that!) - but merging it as-is would create an API that is likely to confuse users and doesn't align with the design principles of the existing API. The API I've proposed here is probably going to be a lot of work to implement, so I don't expect you to make all of those changes in this PR unless you're feeling particularly ambitious. 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AudioFile in input of AudioStream
2 participants