Java Heap Space - Memory Issue

[kazzaw]

sdrtrunk Version
nightly build 2024-02-13

Describe the bug
SDRTrunk Crashes with the following error messages

20240408 103149.791 [sdrtrunk channel [69/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 96%] 20240408 103309.759 [sdrtrunk channel [44/DMR] 505287500 thread 1] WARN i.g.dsheirer.map.MapService - Received plottable decode event that does not contain a FROM identifier - cannot plot [2GB/2GB 97%] 20240408 103349.793 [sdrtrunk channel [65/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 95%] 20240408 103549.852 [sdrtrunk channel [69/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 98%] 20240408 103749.795 [sdrtrunk channel [64/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 94%] 20240408 103932.296 [sdrtrunk channel [66/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unrecognized IP Packet Version: 0 - returning unknown packet [2GB/2GB 95%] 20240408 103949.847 [sdrtrunk channel [67/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 98%] 20240408 104043.843 [sdrtrunk channel [45/DMR] 462862500 thread 1] WARN i.g.dsheirer.map.MapService - Received plottable decode event that does not contain a FROM identifier - cannot plot [2GB/2GB 97%] 20240408 104058.445 [sdrtrunk channel [45/DMR] 462862500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UNIFIED DATA TRANSPORT - returning unknown packet [2GB/2GB 96%] 20240408 104149.749 [sdrtrunk channel [65/DMR] 463162500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 96%] 20240408 104549.851 [sdrtrunk channel [69/DMR] 463612500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 99%] 20240408 104932.257 [sdrtrunk channel [64/DMR] 463612500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: RESERVED 12 - returning unknown packet [2GB/2GB 95%] 20240408 104949.951 [sdrtrunk channel [66/DMR] 463612500 thread 1] INFO i.g.d.m.d.d.m.d.p.PacketSequenceMessageFactory - Unknown Packet SAP: UDP/IP HEADER COMPRESSION - returning unknown packet [2GB/2GB 97%] 20240408 105152.737 [sdrtrunk channel [69/DMR] 463612500 thread 1] DEBUG i.g.d.r.AudioRecordingManager - Audio Segment detected with NO TO identifiers [2GB/2GB 97%] 20240408 105222.103 [sdrtrunk USB tuner - bus [2] port [21]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105222.104 [sdrtrunk USB tuner - bus [2] port [19]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105222.423 [sdrtrunk polyphase ifft processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class java.util.ArrayList] to listener [class io.github.dsheirer.dsp.filter.channelizer.ComplexPolyphaseChannelizerM2$IFFTProcessorDispatcher$$Lambda$589/0x00000008006f3e38] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105222.104 [sdrtrunk USB tuner - bus [2] port [18]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105222.424 [sdrtrunk polyphase ifft processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class java.util.ArrayList] to listener [class io.github.dsheirer.dsp.filter.channelizer.ComplexPolyphaseChannelizerM2$IFFTProcessorDispatcher$$Lambda$589/0x00000008006f3e38] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105222.423 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class io.github.dsheirer.buffer.ByteNativeBuffer] to listener [class io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105208.926 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.d.d.f.c.PolyphaseChannelManager - Error [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space at io.github.dsheirer.dsp.filter.channelizer.ComplexPolyphaseChannelizerM2.process(ComplexPolyphaseChannelizerM2.java:340) at io.github.dsheirer.dsp.filter.channelizer.ComplexPolyphaseChannelizerM2.receive(ComplexPolyphaseChannelizerM2.java:222) at io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver.receive(PolyphaseChannelManager.java:516) at io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver.receive(PolyphaseChannelManager.java:467) at io.github.dsheirer.util.Dispatcher.process(Dispatcher.java:184) at io.github.dsheirer.util.Dispatcher$Processor.run(Dispatcher.java:207) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.runWith(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) 20240408 105222.105 [sdrtrunk scheduled thread 4] ERROR i.g.d.a.b.AudioStreamingBroadcaster - Error while processing audio streaming queue [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105229.339 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.d.d.f.c.PolyphaseChannelManager - Error [2GB/2GB 96%] java.lang.OutOfMemoryError: Java heap space 20240408 105229.339 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.d.d.f.c.PolyphaseChannelManager - Error [2GB/2GB 97%] java.lang.OutOfMemoryError: Java heap space 20240408 105229.339 [sdrtrunk USB tuner - bus [2] port [18]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 97%] java.lang.OutOfMemoryError: Java heap space 20240408 105229.350 [sdrtrunk USB tuner - bus [2] port [21]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 98%] java.lang.OutOfMemoryError: Java heap space 20240408 105234.800 [sdrtrunk USB tuner - bus [2] port [16]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.187 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.d.d.f.c.PolyphaseChannelManager - Error [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.187 [sdrtrunk polyphase ifft processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class java.util.ArrayList] to listener [class io.github.dsheirer.dsp.filter.channelizer.ComplexPolyphaseChannelizerM2$IFFTProcessorDispatcher$$Lambda$589/0x00000008006f3e38] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.187 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class io.github.dsheirer.buffer.ByteNativeBuffer] to listener [class io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.187 [sdrtrunk polyphase ifft processor thread 1] ERROR i.g.d.d.f.c.ComplexPolyphaseChannelizerM2 - Error during IFFT and dispatch of processed channel results [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.188 [sdrtrunk scheduled thread 1] ERROR i.g.d.a.b.AudioStreamingBroadcaster - Error while processing audio streaming queue [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.188 [sdrtrunk USB tuner - bus [2] port [16]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105249.188 [sdrtrunk USB tuner - bus [2] port [19]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105259.774 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.d.d.f.c.PolyphaseChannelManager - Error [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105259.775 [sdrtrunk USB tuner - bus [2] port [19]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105259.774 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class io.github.dsheirer.buffer.ByteNativeBuffer] to listener [class io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver] [2GB/2GB 99%] java.lang.OutOfMemoryError: Java heap space 20240408 105302.560 [sdrtrunk polyphase buffer processor thread 1] ERROR i.g.dsheirer.util.Dispatcher - Error while dispatching element [class io.github.dsheirer.buffer.ByteNativeBuffer] to listener [class io.github.dsheirer.dsp.filter.channelizer.PolyphaseChannelManager$NativeBufferReceiver] [2GB/2GB 98%] java.lang.OutOfMemoryError: Java heap space 20240408 105302.560 [sdrtrunk USB tuner - bus [2] port [19]] ERROR i.g.d.s.t.u.USBTunerController - Error while processing LibUsb timeout events [2GB/2GB 98%] java.lang.OutOfMemoryError: Java heap space 20240408 111335.044 [AWT-EventQueue-0] INFO i.github.dsheirer.gui.SDRTrunk - Application shutdown started ... [2GB/2GB 99%] 20240408 111335.046 [AWT-EventQueue-0] INFO i.github.dsheirer.gui.SDRTrunk - Stopping channels ... [2GB/2GB 99%] 20240408 111335.063 [AWT-EventQueue-0] INFO i.github.dsheirer.gui.SDRTrunk - Stopping spectral display ... [2GB/2GB 99%] 20240408 111335.071 [AWT-EventQueue-0] INFO i.github.dsheirer.gui.SDRTrunk - Stopping tuners ... [2GB/2GB 99%] 20240408 111335.077 [AWT-EventQueue-0] INFO i.g.d.s.t.m.DiscoveredTuner - Stopping Tuner: Airspy USB Bus:2 Port:21 [2GB/2GB 99%]

To Reproduce
Steps to reproduce the behavior:
Run the nightly version for ~7 days without reboot/closing of application. Application eventually becomes non-responsive (I get an alert that no calls have been imported to Trunking Recorder which informs me the app has crashed

Expected behavior
Application should continue to run

Application Log
sdrtrunk_app.log

Desktop (optional - complete the following information):

• OS: Server 2019
• CPU Cores: 6 Cores
• RAM: 10GB (Although Java detects 2GB?)

Additional context
Have attached the log as there seems to be a few different error messages through normal decoding.

Decoding 4x NBFM channels and 2x DMR systems. DMR systems have the ability to have up to 4 Traffic Channels each

Using 4x RTL SDR V3 (RTL-2832) and 1x Airspy Mini

[DSheirer]

Are you modifying the sdrtrunk run script and placing any -Xmx or -Xms switches on the command line?

There's something going on that's placing a 2G memory restriction on the application which is not really enough memory for the application.

[kazzaw]

No, I wasn't. But I did some digging around found the switches, so I've added a manual switch for 6GB and will monitor its performance

[DSheirer]

@kazzaw can we close this issue?

[kazzaw]

Yes, setting the switch for 6GB of ram appears to have fixed the issue. Thanks Denny