Project suggestion: Chafa backend for SDL2

[clort81]

I think there was a TCT renderer for half blocks, with libsdl1.2
It would be a fun project to develop a backend for sdl2 that can render to half blocks, quad blocks, or even braille.

Here's mednafen - Dragon Quest 3 rendered in braille (conversion from screenshot).

dragonquest3_text.ans.gz

[hpjansson]

That does sound like a fun project. I think Caca had a backend that could render via SDL or some other raster device. Might be a bit out of scope for this repository, though -- if it were me, I'd set up a separate project and make it depend on Chafa.

Also lol at the horned up hares/bunicorns.

[clort81]

Understand it's offtopic but some games run surprisingly well.
Pac-Manfrom x virtual framebuffer, live, controllable via x11vnc.

For any interested people, the update speed can be accelerated by using dirty buffers, and code optimizations for the limited 256-char braille set..

[hpjansson]

Maybe I misunderstood -- I thought you meant a backend that would render characters to an SDL surface instead of to a terminal. On the other end, writing an SDL driver that renders SDL surfaces to the terminal via Chafa would also be doable, cool and probably not too hard (but still outside the scope of this repo). You could write a VNC client or Wayland remote viewer, or rewire Xephyr if you want a less hacky way to redirect X11 -- anything that could be made to draw to a framebuffer can be easily routed through Chafa.

By the way, I was able to run a few Steam games in Xvfb routed to Chafa at one point, even simple GL rendered ones. LLVMpipe performs well enough. But decent input for an arcade game is afaict effectively impossible, as terminal protocols have no keyup events, only keydown.

[clort81]

Maybe I misunderstood -- I thought you meant a backend that would render characters to an SDL surface instead of to a terminal.

Ahh, well libsdl-ttf can already render font glyphs to a sdl surface. What would be fun is using a limited chafa style conversion to render to terminal. Libsdl1.2 could do it with ascii, and with a patch also 'sixel'.

I've been looking at the 'tct' truecolor terminal rendering in 'mpv' media player, as well as the 'caca' ascii rendering in libsdl 1.2. I can sort of understand what they're doing. Limiting the chafa-style conversion to 'just braille' ought to make the search faster.

But decent input for an arcade game is afaict effectively impossible, as terminal protocols have no keyup events, only keydown.

Afaik, if the program running in the terminal is running under xorg, it has access to x events, so keyup can be detected.

[hpjansson]

Ahh, well libsdl-ttf can already render font glyphs to a sdl surface. What would be fun is using a limited chafa style conversion to render to terminal. Libsdl1.2 could do it with ascii, and with a patch also 'sixel'.

Indeed, that sounds like a fun project :-)

I've been looking at the 'tct' truecolor terminal rendering in 'mpv' media player, as well as the 'caca' ascii rendering in libsdl 1.2. I can sort of understand what they're doing. Limiting the chafa-style conversion to 'just braille' ought to make the search faster.

I think someone suggested adding a Chafa backend to mpv or replacing the existing tct one, but afair (second hand) the mpv maintainers weren't too keen on accepting more work on text backends.

Anything still using Caca or plain ASCII is probably far behind the state of the art at this point, so could probably benefit from a Chafa backend.

But decent input for an arcade game is afaict effectively impossible, as terminal protocols have no keyup events, only keydown.

Afaik, if the program running in the terminal is running under xorg, it has access to x events, so keyup can be detected.

Right, but a proper terminal application should also accept keyboard input from the terminal session where the output goes, not via a side channel. I just discovered that the Kitty protocol supports keyup events as an option (the application must ask for it specifically), so it would be possible to do in Kitty.

[clort81]

Hey hpjansson, some chafa+blapinus video (114MB) "8-Bit sunset"

https://0x0.st/HLcq.mkv
music an excerpt from Branflakes : Vocaloid - Meltdown
It is better than mpv / mplayer's half-block TCT rendering for sure but it'd need 3d denoising between frames to look less noisy. :)

[hpjansson]

Love it!

Regarding noise, another thing that may help is a "delta compression" feature I've been thinking about. It would do partial updates of the image instead of redrawing the entire image each frame.

The primary use case is to reduce bandwidth use for slow links, but it could be useful here too, as cells that didn't see significant perceptual change would be more stable.

Each cell would be evaluated and given a score according to:

• Improvement in squared error if the current symbol were to be replaced.
• Time (in number of frames) since this cell was last updated ("staleness").
• Bandwidth cost of update (negative factor).
  • Length of UTF-8 sequence.
  • Extra cursor movement if the cell is not next to another cell that's also being updated.
  • Extra SGR seq to change color. Since these seqs are long, it might be beneficial to cluster updates by color (with possible quantization in directcolor mode), though this would be fairly complex.

The highest-scoring cells would then be updated. The update threshold could be adjusted by the user or an algorithm to satisfy either a bandwidth quota or maximum global error.