Understanding color screens
There are several different technologies that could be used to build full color EPDs. The 2 most common technologies are CFA and ACeP.
CFA stands for color filter array, which is basically colored glass/ film on top of the screen pixel. This is also the technology used on most color LCDs. Eink Kaleido, Eink Triton and color DES are based on this technology. The main advantage is that it's relative simple to control, and the low level driving is the same with the greyscale panels. Thus it has the same level of refreshing time (100~200ms), and same level of greyscale (16 level translate to 16^3=4096 colors). The draw back is that the CFA filters out some light (due to its colored nature), the screen reflectivity is negatively affected by the CFA. The screen ends up being quite dark. Up to now, all color E-readers uses CFA-based display. NekoInk supports color DES screen, though it should also be able to support Kaleido and Triton with some minor software modifications.
ACeP stands for Advanced Color ePaper, is a technology developed by Eink. There are 2 variants, Eink Gallery and Eink Gallery Palette. The former achieves 32K colors, the latter only has 7 colors. It uses CMY color pigments similar to color printers to achieve full color display. The advantage is it's much brighter compared to CFA based solutions. The disadvantage is it's much more difficult to drive, and quite slow: 1st gen Eink Gallery screen takes 30s (!) to refresh, and 2nd gen device demoed at SID DisplayWeek could achieve ~1s refresh speed, though that's still much slower than CFA based solutions. The NekoInk might be able to drive parallel ACeP panels (Atelier) with software-only changes, but to be honest I don't know.
The GDEW101C01 is a 10.1" color DES screen made by Good Display. It uses CFA to produce color image. As a result, to the EPDC hardware, it's just a normal greyscale panel with bunch of pixels. But the pixels are colored depending on their location due to the CFA. As there is no hardware handling of such screens, it's up to software to send the correct color value to corresponding pixels.
Color DES is a bit different from a typical color TFT LCD in terms of CFA design. Typically on TFT LCDs, one RGB pixel is called a pixel, and each R/G/B component is called as a sub-pixel. On color DES, the sub-pixel is ended up being called as pixel, and each pixel is either red, green, or blue. In display industry, such pixels are more commonly referred as dots.
Pengo, CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0, via Wikimedia Commons
In the above photo, the DES pixel arrangement is the same as XO-1.
This doesn't necessarily mean that the effective resolution needs to be divide by 3. This arrangement is slightly more "efficient" than RGB strip in terms of perceptive resolution. You could find more detailed analysis in the paper Comparing the Effective Resolution of Various RGB Subpixel Layout.
If the image is displayed on color DES without any processing, it would look like a greyscale image. This is similar when you just send the same color value to R/G/B components on a color LCD display.
To get color, send only the color component that correspond to the pixel color.
For example, for pixel 01, the pixel on the screen is green. Then only the green component from the frame buffer should be sent to the screen.
To get basic 4096 color image display, this is everything needed. However generally to improve image quality, few more steps are applied:
The previous described image displaying process is essentially a down-sampling process for each color components: Only 1/3 of the pixel are sent to the screen. Then this becomes a classic signal processing issue: sampling the signal would cause frequency components above Nyquist to fold back to the low frequency part. Or in simpler words, you would see jagged edges/ aliasing, which are things that wasn't present in the original image. To prevent this, low pass filtering (blurring) needs to applied first to the image to remove these high frequency components. The following diagram from OLPC wiki shows a simple way of implementing a such filter:
DES panels only supports up to 16 level of greyscale with the current waveform. This translate to 4096 colors, which isn't too good. Dithering could be applied to produce better image (increasing SNR, in signal processing sense). See https://en.wikipedia.org/wiki/Dither for background information.
The basic idea is when rounding down the color (selecting the closest color from the 16-level greyscale from 256-level input), the error value (difference) is calculated and added to neighboring pixels (so these error would be considered when processing these pixels later). The whole process is called error-diffusion. Error diffusing uses a diffusion kernel, which specifies the target and percentage of the diffused error. There are many classic kernels, such as Floyd–Steinberg, Stucki, and Sierra etc. commonly used for image dithering. However they won't work with the case here: the dithering process should push/ diffuse the error only to pixels with the same color. As a result, I came up with a kernel on my own, implemented in the imgview tool provided in the repo. It seems to work fairly well though you could always try to tweak that and come up with your own.
Another thing to consider is gamma. The dithering process involves a step of selecting the closest value. However, selecting the closest numerical value is not necessarily mean the closest color. The image typically is in sRGB space, which is non-linear. This causes the simple rounding to pick the wrong color, also calculating the wrong error value. One solution is to work in the linear space. This is also known as gamma-aware dithering. You could read more related info online, for example here: https://learnopengl.com/Advanced-Lighting/Gamma-Correction.
This repo provided a tool called imgview which do all these and display PNG image on the NekoInk. Due to the low contrast and low reflectivity of the EPD panel, gamma-aware processing could generally be turned off without much visible artifacts. LPF could also be turned off for non-text displays.