Display transcript text and follow along the audio

[tonytamsf]

fix #4935

Description

Display contents of podcast:transcript tag as a full dialog view using RecylerView.

1. When tapping the currently playing transcript paragraph, tapping it will pause or start the podcast
2. When tapping another paragraph, will skip and start playing
3. better handling of scrolling, so the user can scroll to future and past and not have the software automatically scroll back to the current stat of the audio

Yet to do

• allow for copy and pasting
• allow for sharing and episode at a certain time from the transcript

Future

• allow for searching for text in the transcript

Image & Video

Version 0.9

Follow audio checkbox

audio.follow.mp4

Version 0.1

transcript-window-480.mov

[Matth7878]

Nice. A suggestion after looking at the screenshot. I think text should be justified to make it easier to read.

[keunes]

@Matth7878 Full justified text is only acceptable when hyphenation is applied correctly. Otherwise you get 'rivers' (vertical lanes of white space) which aren't great, especially for folks with dyslexia. I don't know if good hyphenation (which of course is langue-dependant) is doable in Android

Some resources on this:
https://blog.prototypr.io/text-alignment-best-practises-c4114daf1a9b
https://yesimadesigner.com/justification-vs-alignment/
https://uxmovement.com/content/6-surprising-bad-practices-that-hurt-dyslexic-users/

[keunes]

@tonytamsf Nice! I think I'd put it in a full window, though, instead of a modal. Gives just that little extra space, and with a top bar we get a logical place to add a search icon (in future).

[tonytamsf]

@tonytamsf Nice! I think I'd put it in a full window, though, instead of a modal. Gives just that little extra space, and with a top bar we get a logical place to add a search icon (in future).

@keunes
If we did the full window, do we have to handle any of the player controls like forward, pause?

Thoughts about dialog vs full fragment view @ByteHamster ?

[keunes]

Hi @tonytamsf

If we did the full window, do we have to handle any of the player controls like forward, pause?

Not in any different way from a dialog, I would say. (I believe we did discuss having player controls but potentially in future.)

Maybe a full screen dialog would be easier to implement than a dedicated fragment?

https://m3.material.io/components/dialogs/specs

[ByteHamster]

Thanks! It's nice seeing the transcript run by on by own phone. It already works great.

I added a bunch of comments below. The CI server also left a comment about a SpotBugs violation (that's new with the rebase :) )

[tonytamsf]

address most comments, still have a few more to go

[ByteHamster]

I just pushed a change to use a simple array with binary search. Seems to work fine and reduces the number of different data structures significantly.

I think something broke recently (between my previous commit and [before] the current one). The transcripts are concatenated for much longer sequences than just 5 seconds. To be exact, it seems that all text of one speaker is in one single segment

[tonytamsf]

I've added support for follow audio, which will only show up when a user manually scrolls pass 5 positions

[tonytamsf]

@keunes and @ByteHamster - please play with this version and give feedback, I'm reasonable happy with this version

[ByteHamster]

Thanks for adding the "follow audio" checkbox, that's nice to have!

I think there are some small quirks left. Would be great if you could have a look.

• When scrolling manually, the checkbox stays checked and it jumps back to the playback position. I think the box should automatically un-check when scrolling manually.
• Sometimes the dialog remains empty. I can reproduce this when turning off the network connection but I also had it happen with network connection.
• When there are many 1-line segments, the currently playing one is sometimes cut off on top. Maybe that happens because scrolling to the position-1 has some kind of offset. Maybe there is a way to scroll to the first visible position? Not sure.
• Sometimes when auto-scrolling, it keeps the playing episode below the screen, not on top of it. On my device this mainly happens when manually scrolling to the top, but it also happens when freshly opening the screen
• The logs print a bunch of InterruptedIOExceptions. These are harmless but probably confuse future contributors (we had something similar with the chapter screen). I would just print a single log line for InterruptedIOExceptions instead of a full stack trace.
• The forceRefresh parameter is unused, it always does a force-refresh. I think it should only do a force-refresh when clicking the corresponding button. Otherwise it cannot be viewed offline, for example. I think it should use the same logic as the online chapters

Showcasing the scrolling below the screen and the cut-off first episode: https://drive.google.com/file/d/1-ik8dUhYOB_3rEmiMwNnhLB6YZ40BAOY/view?usp=sharing

[keunes]

Very cool! Some UI comments:

• When highlighting, I wouldn't highlight the timestamps and speakers. They're a bit distracting.
• When changing highlights, the animations (colour changes) can be a wee bit faster. It now sometimes feels like the highlights are lagging a bit, which I think is just an impression.
• The text is a bit small to read, seems smaller than the podcast description text. I would align the two.
• When there's speaker names, maybe we could extend the maximum segment time? I initially thought that maybe we should collate everything into one segment as long as it's the same speaker. That would work nice for the Podcasting 2.0 podcast (as Adam and Dave frequently hand over the speaking stick), but I think in other contexts the segments would then become too long (e.g. long-form interviews). Not sure what the time limit should be.
• The 'Follow audio' toggle doesn't display initially, only when you start scrolling (is my impression). I would just display it always (especially when turning off automatically as suggested by @ByteHamster).

Side issue: the automatic transcripts of Podcasting 2.0 are… not great ^^ Hope the speech recognition tech will improve further (or people edit manually before releasing).

[tonytamsf]

@ByteHamster and @keunes Thanks for testing and the feedback, I'll work on changes to fix. One thing I'm struggling with is the checkbox and the Recylerview doesn't seem to work well together inside a layout inside of the dialog. The RecyclerView seems to overlap with the checkbox even thought I'm using a LinearLayout. If you have any ideas, please let me know.

The other feedback, I'll track down and fix probably in about 1 week.

[tonytamsf]

@keunes and @ByteHamster - could you test this for transcript again. I think I have addressed all your comments.