rebar3 tree formats its output very badly when called together with OTP26

[phiotr]

Pre-Check

• If you are filing for a bug, please do a quick search in current issues first
• For bugs, mention if you are willing or interested in helping fix the issue
• For questions or support, it helps to include context around your project or problem
• Think of a descriptive title (more descriptive than 'feature X is broken' unless it is fully broken)

Environment

• Add the result of rebar3 report to your message:

Rebar3 report
 version 3.22.1
 generated at 2023-10-02T13:04:58+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: 
Entered as:
  
-----------------
Operating System: x86_64-pc-linux-gnu
ERTS: Erlang/OTP 26 [erts-14.1] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit:ns]
Root Directory: /home/piotr/.evm/erlang_versions/otp_src_26.1.1/lib/erlang
Library directory: /home/piotr/.evm/erlang_versions/otp_src_26.1.1/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.12.2
certifi: 2.11.0
cf: 0.3.1
common_test: 1.25.1
compiler: 8.4.1
crypto: 5.3
cth_readable: 1.5.1
dialyzer: 5.1.1
edoc: 1.2
erlware_commons: 1.6.0
eunit: 2.8.2
eunit_formatters: 0.5.0
getopt: 1.0.2
inets: 9.0.2
kernel: 9.1
providers: 1.9.0
public_key: 1.14.1
relx: 4.8.0
sasl: 4.2.1
snmp: 5.15
ssl_verify_fun: 1.1.6
stdlib: 5.1.1
syntax_tools: 3.1
tools: 3.6

-----------------
Escript path: /opt/rebar3/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit fmt format get-deps help install install_deps list lock new path pkgs proper release relup report repos send shell state tar tree unlock update upgrade upgrade upgrade vendor version xref 
...Rebar3 report
 version 3.22.1
 generated at 2023-10-02T13:04:58+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)

• Verify whether the version of rebar3 you're running is the latest release (see https://github.com/erlang/rebar3/releases)
• If possible, include information about your project and its structure. Open source projects or examples are always easier to debug.
  If you can provide an example code base to reproduce the issue on, we will generally be able to provide more help, and faster.

Current behaviour

Today, when playing with Erlang OTP 26.1.1 and the latest rebar3 (v.3.22.1) I discovered it prints the tree command's output in a very ugly way:
(I picked up an ecron dependency app as a short example)

[piotr@G3-3590:ecron]$ rebar3 --version
rebar 3.22.1 on Erlang/OTP 26 Erts 14.1

[piotr@G3-3590:ecron]$ DEBUG=1 rebar3 tree
===> Evaluating config script "rebar.config.script"
===> 26.1.1 satisfies the requirement for minimum OTP version 23
===> 26.1.1 satisfies the requirement for minimum OTP version 23
===> Compile (apps)
===> 26.1.1 satisfies the requirement for minimum OTP version 21
===> Compile (apps)
===> Compile (apps)
===> Evaluating config script "/home/piotr/.../_build/default/lib/ecron/_build/default/plugins/coveralls/rebar.config.script"
===> Evaluating config script "/home/piotr/.../_build/default/lib/ecron/_build/default/plugins/jsx/rebar.config.script"
===> Compile (apps)
===> Expanded command sequence to be run: [app_discovery,install_deps,lock,tree]
===> Running provider: app_discovery
===> Found top-level apps: [ecron]
	using config: [{src_dirs,["src"]},{lib_dirs,["apps/*","lib/*","."]}]
===> Compile (apps)
===> Not adding provider default proper from module rebar3_proper_prv because it already exists from module rebar3_proper_prv
===> Evaluating config script "/home/piotr/.../_build/default/lib/ecron/_build/default/plugins/coveralls/rebar.config.script"
===> Evaluating config script "/home/piotr/.../_build/default/lib/ecron/_build/default/plugins/jsx/rebar.config.script"
===> Compile (apps)
===> Not adding provider coveralls send from module rebar3_coveralls because it already exists from module rebar3_coveralls
===> Running provider: install_deps
===> Verifying dependencies...
===> Running provider: lock
===> Running provider: tree
\x{2514}\x{2500} ecron\x{2500}0.6.1 (project app)
   \x{2514}\x{2500} telemetry\x{2500}1.1.0 (hex package)

These problems started in v3.22.0. In rebar3 v3.21.0, the output from tree command looks fine:

[piotr@G3-3590:ecron]$ rebar3 --version
rebar 3.21.0 on Erlang/OTP 26 Erts 14.1

[piotr@G3-3590:ecron]$ rebar3 tree
===> Verifying dependencies...
└─ ecron─0.6.1 (project app)
   └─ telemetry─1.1.0 (hex package)

Even version 3.22.1 produces a well-formatted ASCI leader on OTP 25.3:

[piotr@G3-3590:ecron]$ rebar3 --version
rebar 3.22.1 on Erlang/OTP 25 Erts 13.2

[piotr@G3-3590:ecron]$ rebar3 tree
===> Verifying dependencies...
└─ ecron─0.6.1 (project app)
   └─ telemetry─1.1.0 (hex package)

Just a combination of the newest rebar3 with the newest Erlang OTP is broken.

Expected behaviour

rebar3 tree should print out a readable dependency leader on the systems with the newest Erlang OTP.

[ferd]

This code hasn't changed since 2019, so I would actually imagine it's based on the shell encoding issue that has changed to work with the new OTP mechanisms when the IO ports for OTP-26 got changed: #2794

These values are already encoded as unicode, so that's a bit surprising:

rebar3/apps/rebar/src/rebar_prv_deps_tree.erl

Lines 64 to 72 in a16f41a

	Prefix1 = case Rest of
	[] ->
	io:format("~ts~ts", [Prefix, <<226,148,148,226,148,128,32>>]), %Binary for └─ utf8%
	[Prefix, " "];
	_ ->
	io:format("~ts~ts", [Prefix, <<226,148,156,226,148,128,32>>]), %Binary for ├─ utf8%
	[Prefix, <<226,148,130,32,32>>] %Binary for │ utf8%
	end,
	io:format("~ts~ts~ts (~ts)~n", [Name, <<226,148,128>>, Vsn, type(Source, Verbose)]), %Binary for ─ utf8%

I am currently not able to replicate this but my own terminal is set with LC_ALL=en_US.UTF-8. I would imagine the problem you have happens because the encoding is undefined, we're pushing utf-8, and your terminal is actually set on an ASCII environment value or some other charset that isn't UTF-8.

Can you confirm what your environment is with regards to locale? We can probably find a way to hit compatibility, but only if we can figure out which variations cause trouble.

[phiotr]

Good catch, Mr. Fred. Indeed, I am using Polish language locales in the operating system. And default ANSI-C encoding as a fallback (LC_ALL=C) when anything else fails.

[piotr@G3-3590:~]$ env | egrep '^LC'
LC_ADDRESS=pl_PL.UTF-8
LC_NAME=pl_PL.UTF-8
LC_MONETARY=pl_PL.UTF-8
LC_PAPER=pl_PL.UTF-8
LC_IDENTIFICATION=pl_PL.UTF-8
LC_TELEPHONE=pl_PL.UTF-8
LC_MEASUREMENT=pl_PL.UTF-8
LC_TIME=pl_PL.UTF-8
LC_ALL=C
LC_NUMERIC=pl_PL.UTF-8

unsettling the LC_ALL environmental variable solves the problem

[piotr@G3-3590:ecron]$ rebar3 tree
\x{2514}\x{2500} ecron\x{2500}0.6.1 (project app)
   \x{2514}\x{2500} telemetry\x{2500}1.1.0 (hex package)

[piotr@G3-3590:ecron]$ unset LC_ALL
[piotr@G3-3590:ecron]$ rebar3 tree
===> Verifying dependencies...
└─ ecron─0.6.1 (project app)
   └─ telemetry─1.1.0 (hex package)

[ferd]

I wonder if there's really a good way to deal with that aside from showing warnings going LC_ALL does not contain a UTF-8 variant, output may be wrong on some commands like tree

[phiotr]

Well, one of the possible solutions could be printing some simplified ASCII leaders in non-UTF-8 environments, next to the warning message. But to be honest I don't think it is a common problem and worth any further investigation or work. In standard configurations, no one else will find this.

Please feel free to close this issue as a "won't fix"