input() recognize Carriage return (\r, 0x0D) but not Line Feed (\n, 0x0A)

[DeqingSun]

CircuitPython version

Adafruit CircuitPython 9.0.4 on 2024-04-16; Adafruit KB2040 with rp2040

Code/REPL

if supervisor.runtime.serial_bytes_available:
    data = input()
    print(data)

Behavior

On windows, enter will be encoded as \r\n (0x0D, 0x0A)
However, the input seems only recognize the \r, and the 0x0A cause the code to freeze at the input()

I've tried to send 0x0D only, the input has response.
Then I tried 0x0A, the input has no response.

Description

I tried to search why this happen on github. It seems the input is linked with mp_builtin_input in modbuiltins.c.
And it leads to readline_process_char.
It processes \r

circuitpython/shared/readline/readline.c

Line 218 in 126a1a4

} else if (c == '\r') {

but not \n

Additional information

It seems circuit python uses the same code as readline_process_char

The Unix port of Circuit python uses fgets to read line. And fgets actually look for \n

https://github.com/lattera/freebsd/blob/401a161083850a9a4ce916f37520c084cff1543b/lib/libc/stdio/fgets.c#L90

And python version of input() will take care \n or \r\n

https://github.com/python/cpython/blob/19d468a3ab1d57c07a4283d67d439908574aa0cc/Python/bltinmodule.c#L2317

[dhalbert]

The situation about what constitutes an end of line for input is complicated. CircuitPython is reading characters either from a host computer or a UART port. Unlike CPython ("regular" Python) which knows what it's running on, we cannot know which kind of host computer (Windows, Linux, macOS, etc.) we are receiving input from. Or, we might be receiving input from a UART, or a raw keyboard.

We inherited the input() behavior from MicroPython. So I believe the current behavior is deliberate: input is triggered by a \r, and \n is discarded. This covers what gets sent from the different operating systems. If you don't try to see what's in the buffer before trying an input, then it behaves more or less as you'd expect: the Enter key causes the input to finish.

What is your use case, and what do you want the behavior to be? Note that there may be other ways to get what you want, such as using stdin.read(), or using the secondary USB CDC channel to exchange data between the board and the host.

[DeqingSun]

Thanks for your reply. The problem I had today was solved by using stdin.read(1) and managing my buffer to deal with newline. It works well for my job and I'm done with it at least for now.

More thought about this. If I use a serial monitor or my own code, as long as I can fully control the data, it can be fine to just send \r. However, here is a case that is worth some improvement for other users:

I have a code similar to this:

while True:
    if time.monotonic() - previousLightSensorSampleTime > 0.1:
        previousLightSensorSampleTime = time.monotonic()
        // print some data peridocally
        
    while supervisor.runtime.serial_bytes_available:
        print(input())

I used Thonny to run the script and I believe other users can use other IDE and have similar issue.
At the beginning the console can print the sensor data. no problem.
Then I type something in console, and press enter. My input got printed. But my data print also paused. Every time I press enter, I got one sensor print.

So here comes a problem. For IDE like Thonny, I do not have any choice on how the "Enter" is encoded. And it got encoded as \r\n in Windows. the \n became a blocking character that made serial_bytes_available to be true, but input() got waiting. For experienced user like me it is fine. But for a user with no idea about line ending, this is frustrating. (Also frustrating for me for a couple of minutes )

Not only Windows, Arduino also do \r\n

https://github.com/arduino/ArduinoCore-avr/blob/63092126a406402022f943ac048fa195ed7e944b/cores/arduino/Print.cpp#L126-L129

Maybe the input function can peek if there is \n after \r and just remove it? At least, I feel, the \n should be left in the buffer to trigger serial_bytes_available

[dhalbert]

We'll think about this, thanks. I'm not sure it can be perfect, in case the \n is delayed. But if it's there, we could discard it.

For your own purposes, sending data, as I mentioned, there is a secondary CDC channel which you can enable which transmitsand receives bytes, and has no extra printing or REPL handling. See https://learn.adafruit.com/customizing-usb-devices-in-circuitpython/circuitpy-midi-serial#usb-serial-console-repl-and-data-3096590