Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report from testing the Beta Program version 3.5.0.1_B48 #3226

Closed
2peer opened this issue Apr 17, 2024 · 21 comments
Closed

Report from testing the Beta Program version 3.5.0.1_B48 #3226

2peer opened this issue Apr 17, 2024 · 21 comments

Comments

@2peer
Copy link

2peer commented Apr 17, 2024

Few issues I noticed so far while testing the latest Beta release:

  1. It dumps a huge amount of data to stdout or stderr (not sure which, i'm normally piping both into the same file). I got ove 870MB in just a few minute run. What's even more interesting the output doesn't seem to be much line separated. wc - l reports only 22 lines total. Most text editors choke on that kind of input
  2. The output data is not pure text. I actually noticed this before in release builds, but shrugged it of. It normaly outputs a few null characters (zero bytes) before starting it's usual text/debug output. Not sure if any other non-text data is output further.
  3. I can report that the first 1MiB is all 0 bytes (see the point before)
  4. The log file has 873233990 out of which 873232166 is the first line (obtained using head -n 1)
  5. skipping 200MiB in and dumping just 1MiB I can still get an all 0 file
  6. same goes for 820MiB in
  7. I'm not seeing the original messages in the form of "com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1" I was reporting in the original reddit thread, I was seeing them live with tail -f on the output file. Used the strings util to dump all the textual data from the file, it's just not there. Could have BiglyBT spawned a second proccess/thread that overwriten all of those with all 0 s?

strings dump from the log file (what was left of it).:

java.lang.reflect.InaccessibleObjectException: Unable to make field private static transient java.net.InetAddress$NameService java.net.InetAddress.nameService accessible: module java.base does not "opens java.net" to unnamed module @551bca18
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
        at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
        at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
        at com.biglybt.core.util.spi.AENameServiceJava12.init(AENameServiceJava12.java:38)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at com.biglybt.core.config.COConfigurationManager.preInitialise(COConfigurationManager.java:242)
        at com.biglybt.core.config.impl.ConfigurationChecker.setSystemProperties(ConfigurationChecker.java:85)
        at com.biglybt.core.config.impl.ConfigurationManager.initialise(ConfigurationManager.java:177)
        at com.biglybt.core.config.impl.ConfigurationManager.getInstance(ConfigurationManager.java:110)
        at com.biglybt.core.config.COConfigurationManager.getStringParameter(COConfigurationManager.java:303)
        at com.biglybt.platform.unix.ScriptAfterShutdown.main(ScriptAfterShutdown.java:43)
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by com.biglybt.core.security.impl.ClientSecurityManager (file:/home/fedex/usr/bin/biglybt/BiglyBT.jar)
WARNING: Please consider reporting this to the maintainers of com.biglybt.core.security.impl.ClientSecurityManager
WARNING: System::setSecurityManager will be removed in a future release
No shutdown tasks to do
BiglyBT TERMINATED.

It didn't crash. I shut it down manually.

Aside:

  • I noticed that Beta branch automatically enabled the I2P and tor plugins (at least I don't remember having them enabled before)
  • OS and version : Linux - Fedora release 38 (Thirty Eight)
  • BiglyBT Version Number: 3.5.0.1_B48
  • For *nix: KDE Plasma
@parg
Copy link
Contributor

parg commented Apr 17, 2024

Beta versions do automatically enable I2P - disable it in the plugin settings (don't uninstall it, it will re-install)

"Unable to make field private static transient java.net.InetAddress$NameService" is a known error, the latest unix launch script has the required "add opens" command line parameters to remove it

"A terminally deprecated method in java.lang.System has been called" etc is a known issue.

I very much doubt that there is actually 1Mib of all "0 bytes"

@2peer
Copy link
Author

2peer commented Apr 17, 2024

OK. Did another experiment, split the redirects of stdout and stderr:

  • The initial zero bytes seem to be part of the stderr output
  • The non line separated messages reported before are part of some huge array dump to stdout:
DEBUG::Wed Apr 17 11:52:36 CEST 2024  50 listeners added for DM:PeerListenDispatcher
        addPeerListener (DownloadManagerImpl.java:4418), addPeerListener (DownloadManagerImpl.java:4406), <init> (GlobalManagerFileMerger.java:1094), syncFileSets (GlobalManagerFileMerger.java:777), runSupport (GlobalManagerFileMerger.java:259), AERunnable.run, perform (DelayedEvent.java:48), runSupport (TimerEvent.java:133), runSupport (Timer.java:330), AERunna
ble.run, ThreadPool.runIt
        [com.biglybt.core.global.impl.GlobalManagerStatsImpl$1$1@643678a8, com.biglybt.core.tag.impl.TagDownloadWithState$3@3810bdb0, com.biglybt.core.tag.impl.TagDownloadWithState$3@764d279a, com.biglybt.core.tag.impl.TagDownloadWithState$3@648cc73d, com.biglybt.core.tag.impl.TagDownloadWithState$3@33a8103c, com.biglybt.core.global.impl.GlobalManagerFileMerger$
SameSizeFiles$1@e68d6c4, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@11065164, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@1331c956, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@606b02fc, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@28fd36c3, com.biglybt.core.global
.impl.GlobalManagerFileMerger$SameSizeFiles$1@b53dfc9, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@1635cc8c, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@2ef14054, com.biglybt.core.global.impl.GlobalManagerFileMerger$SameSizeFiles$1@6ffb2067,

... it continues on and on.

The message seems to be repeated (there are some newlines at first) but the array gradually grows to larger and larger sizes.

@2peer
Copy link
Author

2peer commented Apr 17, 2024

I very much doubt that there is actually 1Mib of all "0 bytes"

There is, you can check my test procedure:

dd if=biglybt.log.old of=biglybt.log.first1MB bs=1M count=1
dd if=/dev/zero of=nullfile.bin bs=1M count=1
diff biglybt.log.first1MB nullfile.bin

No output produced.
... I know my linux tools

@2peer
Copy link
Author

2peer commented Apr 17, 2024

Also the whole old log file compresses (gzip default opts) to just 848KB. and what I posted is the complete strings biglybt.log.old output so something did output binary data there (mostly zeros).
Could it be string terminating 0 charactes? Maybe, I don't know, but somehow it got there.

@parg
Copy link
Contributor

parg commented Apr 17, 2024

I don't see any zero bytes so no idea. I'll reduce that logging

@2peer
Copy link
Author

2peer commented Apr 17, 2024

"Unable to make field private static transient java.net.InetAddress$NameService" is a known error, the latest unix launch script has the required "add opens" command line parameters to remove it

Looks like startup script (the check_version() function) fails to detect the java version on my system.

On my system java -version outputs:

Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true
openjdk version "17.0.9" 2023-10-17
OpenJDK Runtime Environment (Red_Hat-17.0.9.0.9-4) (build 17.0.9+9)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.9.0.9-4) (build 17.0.9+9, mixed mode, sharing)

...so the head -n 1 command doesn't pick anything useful to determine the java version. Should probably switch to grep or something

Also, may I suggest you start using the excellent shellcheck tool? Shell scripting is quite error prone and I already see a few warnings about unused variables etc.
Thanks

@parg
Copy link
Contributor

parg commented Apr 17, 2024

nah, I don't do linux

@2peer
Copy link
Author

2peer commented Apr 17, 2024

Shellcheck is multiplatform, there are integrations into vsCode etc. and you can also check your script online by just pasting it into "https://www.shellcheck.net/".
When Windows people do shell scripting it gets kind of scary (for example Valves Steam instalation script deleting peoples whole systems by accident a few years back).
But OK. Maybe I can help you adjusting the startup script (to make it more universal)?
For now, maybe just switching the line:

JAVA_HEADER=$($JAVABIN -version 2>&1 | head -n 1)

to

JAVA_HEADER=$($JAVABIN -version 2>&1 | grep version)

... would do the trick? Let me read the whole script and report back.

@parg
Copy link
Contributor

parg commented Apr 17, 2024

wtf is java --version outputting that first line (Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true) for?

Any changes have to deal with all the JRE/JDKs out there...

@2peer
Copy link
Author

2peer commented Apr 17, 2024

wtf is java --version outputting that first line (Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true) for?

Probably have to ask RedHat about that

Any changes have to deal with all the JRE/JDKs out there...

Any official listing of all of those? ... as you might have noticed, I'm not really a Java person

@parg
Copy link
Contributor

parg commented Apr 17, 2024

@2peer
Copy link
Author

2peer commented Apr 17, 2024

Yeah. I think the issue might be the merge of stdout and stderr there.

The Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true is output to error output, but the accepted answer parses both together.

@2peer
Copy link
Author

2peer commented Apr 17, 2024

The third answer there warns about this and uses grep instead.

@parg
Copy link
Contributor

parg commented Apr 17, 2024

so just pick up stdout and ignore stderr and use the existing code?

@2peer
Copy link
Author

2peer commented Apr 17, 2024

I suspect the redirect is there for a reason for some specific JDK versions that just outputs the version to stderr.
Switching JAVA_HEADER line to:

JAVA_HEADER=$($JAVABIN -version 2>&1 | grep -i version | head -n 1)

should do the trick (the sed command bellow assumes the presence of the version keyword anyway so it shouldn't break anything), but

I have a question:
How should we behave if we fail to detect the java version? For now it just ran somehow (at least in my case), but there is also the if [ "$JAVA_IMPL" = "#" ] ; then path that detects the '#' in the first output field and reports it as java crash?
So if we fail to detect java version, should we just report it and exit or "Yolo" it and run anyway?

@parg
Copy link
Contributor

parg commented Apr 17, 2024

Yup, fail -> run it anways

@2peer
Copy link
Author

2peer commented Apr 17, 2024

OK. Less immediate breakage. Would probably like to output a warning / error message to stderr anyway so user can report if it breaks for some future JDK version again.

@2peer
Copy link
Author

2peer commented Apr 17, 2024

OK. So maybe like this:

check_version()
{
  JAVA_ISGCJ=$($JAVABIN -version 2>&1 | grep "gcj")
  if [ ! "$JAVA_ISGCJ x" = " x" ] ; then
    echo "$MSG_ISGCJ"
    return 1
  fi

  JAVA_HEADER=$("$JAVABIN" -version 2>&1 | grep -i version | head -n 1)
  if [ "$JAVA_HEADER" = "" ] ; then
    echo "ERROR: Failed to detect JAVA version" >&2
  fi
  JAVA_IMPL=$(echo "${JAVA_HEADER}" | cut -f1 -d' ')
  if [ "$JAVA_IMPL" = "#" ] ; then
    echo "$MSG_JAVABORK"
    $JAVABIN -version 2>&1
    exit 1
  else
    JAVA_VERSION=$(echo "${JAVA_HEADER}" | sed "s/^.* version \"\(.*\)\".*$/\1/")
    if echo "$JAVA_VERSION" | grep "^1.[0-7]" ; then
      echo "$MSG_JAVA_TOO_OLD" "[$JAVABIN = ${JAVA_VERSION}]" ; echo "$MSG_JAVA_BELOW_MIN"
      return 1
    else
      echo "${MSG5} [$JAVABIN = ${JAVA_IMPL} ${JAVA_VERSION}]"
      return 0
    fi
  fi
}

You can move the new error message into a variable up top maintain the style

@2peer
Copy link
Author

2peer commented Apr 17, 2024

Running on this adjusted script now, successfully getting the '--add-opens' option. Still getting a huge stdout, but at least now I run with options for my particular JDK.

The "Unable to make field private static transient " exception is gone.

@2peer
Copy link
Author

2peer commented Apr 17, 2024

So to summarize:
The only remaining issues I see from this report is the cleanup of stdout, and the weird zero bytes (who knows where they come from).
What is the policy there? Can I safely assume that all useful bug reporting info will come to stderr and just throw away the stdout completely?
It is prefixed with 'DEBUG:' so I assume it is an effect of Beta releases being a debug build by default?
The zero bytes still come to stderr and there is still lot of them (latest run: 500kB total, only 4 warning lines at the end).

@parg
Copy link
Contributor

parg commented Apr 17, 2024

B49 has reduced debug and fixed startup script

@parg parg closed this as completed May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants