Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sum-up of isolated events and soundscape are not equal, when Reverb=None #150

Open
fkutsukos opened this issue Feb 1, 2022 · 0 comments

Comments

@fkutsukos
Copy link

fkutsukos commented Feb 1, 2022

Hello,

I am working on source separation and i have noticed that the sum of the isolated events are not equal to the soundscape, even when the reverb attribute is set to None.

According to the documentation:
The audio of the isolated events is guaranteed to sum up to the soundscape audio if and only if reverb is None!

Below is the code snippet and some plots, confirming the problem is located at the foreground events.
Instead when only the background is present, the soundscape and the sum of isolated files is equal.

import jams
import numpy as np
import librosa
from matplotlib import pyplot as plt
import os
import scaper
import soundfile as sf
import glob


scaper_audio_root = os.path.join('data', 'scaper-master', 'tests', 'data', 'audio')

soundscape_duration = 10.0
seed = 123
foreground_folder = os.path.join(scaper_audio_root, 'foreground')
background_folder = os.path.join(scaper_audio_root, 'background')
sc = scaper.Scaper(soundscape_duration, foreground_folder, background_folder)
sc.ref_db = -20

sc.add_background(label=('const', 'park'),
                  source_file=('choose', []),
                  source_time=('const', 0))

sc.add_event(label=('const', 'siren'),
             source_file=('choose', []),
             source_time=('const', 0),
             event_time=('const', 0),
             event_duration=('const', 1),
             snr=('const', 0),
             pitch_shift=None,
             time_stretch=None)

audiofile = 'soundscape.wav'
jamsfile = 'soundscape.jams'
txtfile = 'soundscape.txt'


sc.generate(audiofile, jamsfile,
            allow_repeated_label=True,
            allow_repeated_source=True,
            reverb=None,
            peak_normalization=True,
            disable_sox_warnings=True,
            no_audio=False,
            txt_path=txtfile,
            save_isolated_events=True)


x_events_folder = os.path.join('soundscape_events')
x_events = f_events = glob.glob(os.path.join(x_events_folder, '*.wav'))

x_from_events = None
for x_event in x_events:
    x_from_events_partial, sr = librosa.load(x_event)
    if x_from_events is None:
        x_from_events = x_from_events_partial
    else:
        x_from_events = x_from_events + x_from_events_partial

x, _ = librosa.load('soundscape.wav')

print('result:', np.array_equal(x, x_from_events))

X = np.abs(librosa.stft(x))
plt.imshow(np.log(X), aspect='auto', origin='lower')
plt.title('X')
plt.show()

X_FROM_EVENTS = np.abs(librosa.stft(x_from_events))
plt.imshow(np.log(X_FROM_EVENTS), aspect='auto', origin='lower')
plt.title('X_FROM_EVENT')
plt.show()

plt.imshow(np.log(np.subtract(X, X_FROM_EVENTS) + 1e-7), aspect='auto', origin='lower')
plt.title('subtract')
plt.show()

import soundfile as sf

jam = jams.load(jamsfile)
ann = jam.annotations.search(namespace='scaper')[0]

soundscape_audio, sr = sf.read(ann.sandbox.scaper['soundscape_audio_path'])
isolated_event_audio_paths = ann.sandbox.scaper['isolated_events_audio_path']
isolated_audio = []

for event_spec, event_audio_path in zip(ann, isolated_event_audio_paths):
    # event_spec contains the event description, label, etc
    # event_audio contains the path to the actual audio
    # make sure the path matches the event description
    assert event_spec.value['role'] in event_audio_path
    assert event_spec.value['label'] in event_audio_path

    isolated_audio.append(sf.read(event_audio_path)[0])

# the sum of the isolated audio should sum to the soundscape
print(np.array_equal( sum(isolated_audio) , soundscape_audio))

image
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant