Audio not Working when adding WAV File to Audio Track

[philipph77]

I am trying to add a WAV file for this song to the audio track of my otio timeline. The code runs without error, however, when opening the timeline later, the audio is not working. Below is a minimal code example in python.
The only thing that I can think of why this is not working, is that the clips start_time and duration are not set correctly. Unfortunately I could not find how to set the for an audio-only file that does not have an fps attribute.

import os
from pathlib import Path
from zipfile import ZipFile

from hachoir.parser import createParser
from hachoir.metadata import extractMetadata
import opentimelineio as otio


def write_zipfile(archive_name: str, timeline_file: Path, songfile: Path, clip_files: list[str], logdir: str) -> None:
    # writes the timeline_file together with the songfile and clip_files into one zip archive
    with ZipFile(archive_name, "w") as archive:
        for clip_file in clip_files: 
            archive.write(os.path.join(logdir, clip_file), os.path.join("media",clip_file))
        archive.write(os.path.join(timeline_file), timeline_file.name)
        archive.write(songfile, os.path.join("media",songfile.name))

def get_song_duration(songfile_path: Path):
    # returns the duration of the songfile in seconds
    song_parser = createParser(str(songfile_path.resolve()))
    song_metadata = extractMetadata(song_parser)
    return song_metadata.get("duration").total_seconds()

def main(songfile_path: Path):
    # create the timeline and track objects
    tl = otio.schema.Timeline(name="Storyline")
    tr_audio = otio.schema.Track(name="Audio", kind=otio.schema.Track.Kind.Audio)
    tl.tracks.append(tr_audio)

    # create the audio clip and append to the audio track
    song_duration = get_song_duration(songfile_path)
    cl_audio = otio.schema.Clip(
        name=str(songfile_path.stem),
        media_reference=otio.schema.ExternalReference(
            target_url="media/"+str(songfile_path.name),
            available_range=otio.opentime.TimeRange(
                start_time=otio.opentime.RationalTime(0, 1),
                duration=otio.opentime.RationalTime(song_duration, 1),
            )
        ),
        source_range=otio.opentime.TimeRange(
            start_time=otio.opentime.RationalTime(0, 1),
            duration=otio.opentime.RationalTime(song_duration, 1),
        )
    )
    tr_audio.append(cl_audio)

    # export timeline to davinci
    fcp_xml_timeline_file = "otio_timeline_fcp.xml"
    otio.adapters.write_to_file(tl, fcp_xml_timeline_file, adapter_name="fcp_xml")
    archive_name = "davinci_only_audio_fcp_xml.zip"
    write_zipfile(archive_name, Path(fcp_xml_timeline_file), songfile_path, [], "")

    # export timeline to OTIO
    fcp_xml_timeline_file = "otio_timeline.otio"
    otio.adapters.write_to_file(tl, fcp_xml_timeline_file)
    archive_name = "davinci_only_audio_otio.zip"
    write_zipfile(archive_name, Path(fcp_xml_timeline_file), songfile_path, [], "")
    return



if __name__ == "__main__":
    songfile_path = Path("assets/songs/Infraction- Fantasia Logo Version 1.wav")
    main(songfile_path)

[apetrynet]

Hi @philipph77

Like you mentioned, I think this has to do with the rate of your clips.
Have you tried setting the "cl_clip" rate to whatever the sample rate of the wav file is?
And I wonder what unit you get back from get_song_duration()?
If seconds and you find the samplerate, you could try:
duration=otio.opentime.from_seconds(song_duration, samplerate)

[apetrynet]

Hmmm. You should be able to use arbitrary rates. The function you mention only checks for valid smpte timecode rates.
Did using the samplerate in the from_seconds method produce an error?

[philipph77]

The function does not throw an error and returns
otio.opentime.RationalTime(value=1.9404e+06, rate=44100)
if I pass this as duration (as shown below), is still get the same behavior (audio track appears in red and no audio is playing).

song_duration_seconds = get_song_duration(songfile_path)
song_duration=otio.opentime.from_seconds(song_duration_seconds, 44100)
 cl_audio = otio.schema.Clip(
        name=str(songfile_path.stem),
        media_reference=otio.schema.ExternalReference(
            target_url="media/"+str(songfile_path.name),
            available_range=otio.opentime.TimeRange(
                start_time=otio.opentime.RationalTime(0, 1),
                duration=song_duration,
            )
        ),
        source_range=otio.opentime.TimeRange(
            start_time=otio.opentime.RationalTime(0, 1),
            duration=song_duration,
        )
    )

[apetrynet]

You'll need to set the start_time rate to use the samplerate as well. It's still set to 1

        media_reference=otio.schema.ExternalReference(
            target_url="media/"+str(songfile_path.name),
            available_range=otio.opentime.TimeRange(
                start_time=otio.opentime.RationalTime(0, 44100),
                duration=song_duration,
            )
        ),
        source_range=otio.opentime.TimeRange(
            start_time=otio.opentime.RationalTime(0, 44100),
            duration=song_duration,
        )

[philipph77]

Setting this to 44100 will throw the following error:
ValueError: invalid timecode rate

[apetrynet]

Oh, that must be something in the time range behavior. I'll need to investigate more.
I'm out for a bit, will look at this again later.

[philipph77]

I just found out that when setting the rate to an arbitrary value (e.g. 30 or 1), and exporting the timeline as otio file (without any adapters) the audio track works as expected in DaVinci Resolve. Thus, this might be an issue with the fcp_xml adapter rather than otio itself

[apetrynet]

Hi!
Yes, I can confirm it's an adapter thing. I tried adding a clip to an audio track using sample rate for all ranges and it worked fine in otio native format.

You can try the following and see if it fits your workflow:

# Set this to whatever frame rate the rest of your project is
FPS=30
# Sample rate of audio file
SAMPLERATE=44100

myrange=otio.opentime.TimeRange(
    start_time=otio.opentime.RationalTime(0, SAMPLERATE).rescaled_to(FPS), 
    duration=otio.opentime.from_seconds(song_duration, SAMPLERATE).rescaled_to(FPS)
)

This way should work as both otio and fcp_xml.
FYI there is an otioz adapter that zips the timeline including media files. Resolve should be able to read these as well.

[philipph77]

I tried rescaling the start time and duration to the videos fps, unfortunately, the audio track still doesn't work in Resolve for me.
Thanks for the otioz hint, I will definitely look into that! :-)

[apetrynet]

I tried rescaling the start time and duration to the videos fps, unfortunately, the audio track still doesn't work in Resolve for me.

Sorry to hear that. I'm afraid I'm out of ideas.
Hope the otioz adapter works for you :)