JVM crash when calling FFmpegFrameRecorder.stop()

[twper-lin]

i try to pull rtsp stream then convert it to flv format for front-end, here is my code:

public void transfer() {
        log.info("connect rtsp：" + rtspUrl + ",start to create grabber");
        boolean isSuccess = createGrabber(rtspUrl);
        if (isSuccess) {
            log.info("create grabber success");
        } else {
            log.info("create grabber failed");
        }
        startCameraPush();
    }

    /**
     *  create video grabber
     * @param rtsp rtsp url
     * @return
     */
    public boolean createGrabber(String rtsp) {
        // get the video source
        try {
//            FFmpegFrameGrabber.createDefault(rtsp)
            grabber = new FFmpegFrameGrabber(rtsp);
            grabber.setOption("rtsp_transport", rtspTransportType);
            grabber.setOption("timeout", "5000000");
            grabber.setOption("buffer_size", "1024000"); // buffer size, unit is byte
            grabber.setOption("analyzeduration", "10000000"); // Maximum duration (in AV_TIME_BASE units) of the data read from input
            grabber.setOption("probesize", "10000000"); // Maximum size of the data read from input
            System.setProperty("org.bytedeco.javacpp.logger", "slf4j");
            System.setProperty("org.bytedeco.javacpp.logger.debug", "true");
            FFmpegLogCallback.set();
            grabber.startUnsafe();
            isStart = true;
            recorder = new FFmpegFrameRecorder(outputStream, 640, 350, grabber.getAudioChannels());
            //avcodec.AV_CODEC_ID_H264  //AV_CODEC_ID_MPEG4
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
            recorder.setFormat("flv");
//            recorder.setFrameRate(grabber.getFrameRate());
            //reduce frame rate to 15
            recorder.setFrameRate(15);
            recorder.setSampleRate(grabber.getSampleRate());
            recorder.setAudioChannels(grabber.getAudioChannels());
            return true;
        }
        catch (FrameGrabber.Exception e) {
            log.error("create rtsp FFmpegFrameGrabber fail");
            log.error("create rtsp FFmpegFrameGrabber exception: ", e);
            stop();
            reset();
            return false;
        }
        catch (Exception e) {
            log.error("createGrabber fail", e);
            return false;
        }
    }

    private void startCameraPush() {
        if (grabber == null) {
            log.info("retry connect rtsp：" + rtspUrl + ",start to create grabber");
            boolean isSuccess = createGrabber(rtspUrl);
            if (isSuccess) {
                log.info("create grabber success");
            } else {
                log.info("create grabber failed");
                try {
                    outputStream.close();
                } catch (IOException e) {
                    log.error("close output stream fail", e);
                }
            }
        }
        try {
            if (grabber != null) {
                recorder.start();
                Frame frame;
                while (isStart && (frame = grabber.grabFrame()) != null) {
                    recorder.setTimestamp(grabber.getTimestamp());
                    recorder.record(frame);
                }
                outputStream.close();
                stop();
                reset();
            }
        } catch (FrameGrabber.Exception | RuntimeException | FrameRecorder.Exception e) {
            log.error(e.getMessage(), e);
            stop();
            reset();
        } catch (IOException e) {
            log.error("close output stream fail", e);
        }
    }

    public void stop() {
        try {
            if (recorder != null) {
                recorder.stop();
                recorder.release();
            }
            if (grabber != null) {
                grabber.stop();
            }
        } catch (Exception e) {
            log.error(e.getMessage(), e);
        }
    }

    public void reset() {
        recorder = null;
        grabber = null;
        isStart = false;
    }

Here are my pom.xml configuration:

<dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv</artifactId>
            <version>${javacv.version}</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>ffmpeg-platform</artifactId>
            <version>6.0-1.5.9</version>
        </dependency>

hs_err_pid.log information as below:

A fatal error has been detected by the Java Runtime Environment:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d9899b3, pid=29788, tid=0x00007534

JRE version: Java(TM) SE Runtime Environment (8.0_221-b11) (build 1.8.0_221-b11)

Java VM: Java HotSpot(TM) Client VM (25.221-b11 mixed mode windows-x86 )

Problematic frame:

C [avformat-60.dll+0x799b3]

Failed to write core dump. Minidumps are not enabled by default on client versions of Windows

If you would like to submit a bug report, please visit:

http://bugreport.java.com/bugreport/crash.jsp

The crash happened outside the Java Virtual Machine in native code.

See problematic frame for where to report the bug.

--------------- T H R E A D ---------------

Current thread (0x16b84c00): JavaThread "Thread-6" daemon [_thread_in_native, id=30004, stack(0x1a770000,0x1a7c0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x1aa27cd4

Registers:
EAX=0x1aa27cc0, EBX=0x00000001, ECX=0x00000000, EDX=0x6c56d7c8
ESP=0x1a7bf510, EBP=0x1adfca80, ESI=0x0003a45a, EDI=0x00000000
EIP=0x6d9899b3, EFLAGS=0x00210206

Top of Stack: (sp=0x1a7bf510)
0x1a7bf510: 1aa27ac0 00000018 6e044bdc 00000000
0x1a7bf520: 1a7bf510 00000001 00000001 00000000
0x1a7bf530: 0000001b 1aa27ac0 1aa27cc0 00000000
0x1a7bf540: 1aa27ac0 00000000 00000042 1aa27ac0
0x1a7bf550: 00000000 1adfab80 00000000 6da410c1
0x1a7bf560: 1aa27ac0 00000000 80000000 00000004
0x1a7bf570: f0bc4d66 16b84d40 1aa27ac0 1a7bf600
0x1a7bf580: 1a7bf570 00000000 1a7bf82c 1aa27ac0

Instructions: (pc=0x6d9899b3)
0x6d989993: 8b 54 24 1c 89 2c 24 89 44 24 04 89 54 24 08 e8
0x6d9899a3: 39 a8 fb ff 8b 44 24 28 c7 44 24 0c 00 00 00 00
0x6d9899b3: 8b 50 14 8b 40 10 89 2c 24 89 44 24 04 89 54 24
0x6d9899c3: 08 e8 87 7b fb ff 85 d2 0f 88 9f 00 00 00 c7 44

Register to memory mapping:

EAX=0x1aa27cc0 is an unknown value
EBX=0x00000001 is an unknown value
ECX=0x00000000 is an unknown value
EDX=0x6c56d7c8 is an unknown value
ESP=0x1a7bf510 is pointing into the stack for thread: 0x16b84c00
EBP=0x1adfca80 is an unknown value
ESI=0x0003a45a is an unknown value
EDI=0x00000000 is an unknown value

Stack: [0x1a770000,0x1a7c0000], sp=0x1a7bf510, free space=317k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [avformat-60.dll+0x799b3]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.bytedeco.ffmpeg.global.avformat.av_write_trailer(Lorg/bytedeco/ffmpeg/avformat/AVFormatContext;)I+0
j org.bytedeco.javacv.FFmpegFrameRecorder.stop()V+15
j com.cmes.cache.utils.MediaVideoTransferUtil.stop()V+11
j com.cmes.cache.utils.MediaVideoTransferUtil.startCameraPush()V+191
j com.cmes.cache.utils.MediaVideoTransferUtil.transfer()V+72
j com.cmes.cache.controller.CameraController.lambda$online$0(Lcom/cmes/cache/utils/MediaVideoTransferUtil;Ljava/io/PipedOutputStream;Lcom/cmes/cache/domain/Camera;)V+20
j com.cmes.cache.controller.CameraController$$Lambda$1337.run()V+12
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub

--------------- P R O C E S S ---------------

Java Threads: ( => current thread )
0x16b83400 JavaThread "JavaCPP Deallocator" daemon [_thread_blocked, id=13592, stack(0x1a900000,0x1a950000)]
=>0x16b84c00 JavaThread "Thread-6" daemon [_thread_in_native, id=30004, stack(0x1a770000,0x1a7c0000)]
0x169e8400 JavaThread "logback-8" daemon [_thread_blocked, id=21952, stack(0x1a6e0000,0x1a730000)]
0x169e8000 JavaThread "logback-7" daemon [_thread_blocked, id=26796, stack(0x1a650000,0x1a6a0000)]
0x169e7800 JavaThread "AsyncResolver-bootstrap-executor-0" daemon [_thread_blocked, id=23132, stack(0x1a5c0000,0x1a610000)]
0x169eb800 JavaThread "logback-6" daemon [_thread_blocked, id=2116, stack(0x1a530000,0x1a580000)]
0x169ebc00 JavaThread "logback-5" daemon [_thread_blocked, id=26128, stack(0x1a3a0000,0x1a3f0000)]
0x169e7000 JavaThread "logback-4" daemon [_thread_blocked, id=8696, stack(0x1a310000,0x1a360000)]
0x169eb000 JavaThread "DestroyJavaVM" [_thread_blocked, id=28952, stack(0x00df0000,0x00e40000)]
0x169ea400 JavaThread "CMES-Async-Service-1" [_thread_blocked, id=11960, stack(0x1a280000,0x1a2d0000)]
0x169e9c00 JavaThread "logback-3" daemon [_thread_blocked, id=6392, stack(0x1a0f0000,0x1a140000)]
0x169ec800 JavaThread "DiscoveryClient-CacheRefreshExecutor-0" daemon [_thread_blocked, id=29928, stack(0x00c10000,0x00c60000)]
0x169ec400 JavaThread "DiscoveryClient-HeartbeatExecutor-0" daemon [_thread_blocked, id=5984, stack(0x00b80000,0x00bd0000)]
0x169e6400 JavaThread "logback-2" daemon [_thread_blocked, id=28116, stack(0x00af0000,0x00b40000)]
0x169e9800 JavaThread "DatebookHikariCP housekeeper" daemon [_thread_blocked, id=5084, stack(0x19e60000,0x19eb0000)]
0x169eac00 JavaThread "PostgreSQL-JDBC-Cleaner" daemon [_thread_blocked, id=21608, stack(0x19dd0000,0x19e20000)]
0x169e6c00 JavaThread "pool-8-thread-3" [_thread_blocked, id=7852, stack(0x197b0000,0x19800000)]
0x169e8c00 JavaThread "pool-8-thread-2" [_thread_blocked, id=29360, stack(0x19720000,0x19770000)]
0x169e6000 JavaThread "pool-8-thread-1" [_thread_blocked, id=23504, stack(0x19690000,0x196e0000)]
0x169e5400 JavaThread "http-nio-8103-Acceptor" daemon [_thread_in_native, id=19308, stack(0x19600000,0x19650000)]
0x169e5800 JavaThread "http-nio-8103-Poller" daemon [_thread_in_native, id=7764, stack(0x19570000,0x195c0000)]
0x169e9000 JavaThread "http-nio-8103-exec-10" daemon [_thread_blocked, id=24892, stack(0x194e0000,0x19530000)]
0x14efec00 JavaThread "http-nio-8103-exec-9" daemon [_thread_blocked, id=1080, stack(0x19450000,0x194a0000)]
0x14efe000 JavaThread "http-nio-8103-exec-8" daemon [_thread_blocked, id=25196, stack(0x193c0000,0x19410000)]
0x14efcc00 JavaThread "http-nio-8103-exec-7" daemon [_thread_blocked, id=28428, stack(0x19330000,0x19380000)]
0x14efe800 JavaThread "http-nio-8103-exec-6" daemon [_thread_blocked, id=29956, stack(0x192a0000,0x192f0000)]
0x14efc000 JavaThread "http-nio-8103-exec-5" daemon [_thread_blocked, id=28452, stack(0x19210000,0x19260000)]
0x14efd800 JavaThread "http-nio-8103-exec-4" daemon [_thread_blocked, id=28684, stack(0x19180000,0x191d0000)]
0x14eff400 JavaThread "http-nio-8103-exec-3" daemon [_thread_in_native, id=25072, stack(0x190f0000,0x19140000)]
0x14efd400 JavaThread "http-nio-8103-exec-2" daemon [_thread_blocked, id=21992, stack(0x19060000,0x190b0000)]
0x14efc800 JavaThread "http-nio-8103-exec-1" daemon [_thread_blocked, id=1256, stack(0x18fd0000,0x19020000)]
0x14eff800 JavaThread "DiscoveryClient-InstanceInfoReplicator-0" daemon [_thread_blocked, id=5316, stack(0x18f40000,0x18f90000)]
0x14269800 JavaThread "DiscoveryClient-1" daemon [_thread_blocked, id=25036, stack(0x18eb0000,0x18f00000)]
0x14270400 JavaThread "DiscoveryClient-0" daemon [_thread_blocked, id=30000, stack(0x18e20000,0x18e70000)]
0x13c19400 JavaThread "AsyncResolver-bootstrap-0" daemon [_thread_blocked, id=29280, stack(0x18b90000,0x18be0000)]
0x13b1b000 JavaThread "spring.cloud.inetutils" daemon [_thread_blocked, id=4760, stack(0x18900000,0x18950000)]
0x16577c00 JavaThread "Statistics Thread-DEFAULT-1" daemon [_thread_blocked, id=548, stack(0x18070000,0x180c0000)]
0x1657dc00 JavaThread "DEFAULT" daemon [_thread_blocked, id=30556, stack(0x17fe0000,0x18030000)]
0x16551c00 JavaThread "container-0" [_thread_blocked, id=24676, stack(0x17d30000,0x17d80000)]
0x16551400 JavaThread "Catalina-utility-2" [_thread_blocked, id=5272, stack(0x158d0000,0x15920000)]
0x16554000 JavaThread "Catalina-utility-1" [_thread_blocked, id=22532, stack(0x14a70000,0x14ac0000)]
0x14b35800 JavaThread "logback-1" daemon [_thread_blocked, id=30712, stack(0x15840000,0x15890000)]
0x13a9e000 JavaThread "Service Thread" daemon [_thread_blocked, id=26100, stack(0x14060000,0x140b0000)]
0x13a9c000 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=13508, stack(0x13fd0000,0x14020000)]
0x13a9a800 JavaThread "Attach Listener" daemon [_thread_blocked, id=30452, stack(0x13f40000,0x13f90000)]
0x13a98800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=28544, stack(0x13eb0000,0x13f00000)]
0x13a77400 JavaThread "Finalizer" daemon [_thread_blocked, id=26800, stack(0x13d30000,0x13d80000)]
0x13a75000 JavaThread "Reference Handler" daemon [_thread_blocked, id=12800, stack(0x13ca0000,0x13cf0000)]

Other Threads:
0x013ce000 VMThread [stack: 0x03580000,0x035d0000] [id=27384]
0x13a96800 WatcherThread [stack: 0x140f0000,0x14140000] [id=27988]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

[sitakanta2006]

Is there a solution for this ?

Even I see a jvm crash at av_write_trailer() while stopping the recorder.

Looks like "oc" is getting nullified in flush() and while using the same in av_write_trailer() it is crashing. We may need to do a null check before calling av_write_trailer() in stop() method.

Do we really need to call both flush() and av_write_trailer() in stop().

Code from FFmpegFrameRecorder.

public synchronized void flush() throws Exception {
synchronized (this.oc) {
while (this.video_st != null && this.ifmt_ctx == null && recordImage(0, 0, 0, 0, 0, -1, (Buffer[])null));
while (this.audio_st != null && this.ifmt_ctx == null && recordSamples(0, 0, (Buffer[])null));
if (this.interleaved && (this.video_st != null || this.audio_st != null)) {
avformat.av_interleaved_write_frame(this.oc, null);
} else {
avformat.av_write_frame(this.oc, null);
}
}
}

public void stop() throws Exception {
if (this.oc != null)
try {
flush();
avformat.av_write_trailer(this.oc);
} finally {
release();
}
}

[saudet]

@sitakanta2006 Please open a pull request with your suggested changes! Thanks