Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Questions about how to receive and use ROS1 image msg #1827

Open
Hogushake opened this issue Apr 15, 2024 · 0 comments
Open

Questions about how to receive and use ROS1 image msg #1827

Hogushake opened this issue Apr 15, 2024 · 0 comments

Comments

@Hogushake
Copy link

@dusty-nv hello
I'm trying to use backgroundnet in a ROS1 environment.
I've asked the same question before.
#1789

And i modified code as you said, a different error occurs.

my modified code is

net = backgroundNet(args.network, sys.argv)
input = videoSource(args.input_URI, argv=sys.argv)
output = videoOutput(args.output_URI, argv=sys.argv)

class backgroundnet():
    def __init__(self):
        self.image_sub = rospy.Subscriber("/camera/color/image_raw", Image, self.callback, queue_size=1)
        self.bridge = CvBridge()

    def callback(self, data):
        try:
            cv_image = self.bridge.imgmsg_to_cv2(data, "rgba8")
            self.detect(cv_image)
        except CvBridgeError as e:
            print(e)
           

    def detect(self, image):
        try:
            bgr_img = cudaFromNumpy(image, isBGR=True)
            rgb_img = cudaAllocMapped(width=bgr_img.width,
                                      height=bgr_img.height,
                                      format='rgba8')

            cudaConvertColor(bgr_img, rgb_img)
            net.Process(rgb_img, filter=args.filter_mode)

            output.Render(rgb_img)
            output.SetStatus("backgroundNet {:s} | Network {:.0f} FPS".format(net.GetNetworkName(), net.GetNetworkFPS()))
            net.PrintProfilerTimes()

        except CvBridgeError as error:
            print(error)

if __name__ == '__main__':
    rospy.init_node('backgroundnet', anonymous=True)
    backgroundnet()
    try:
        rospy.spin()
    except KeyboardInterrupt:
        print("Shutting down!")

when I run that code I get the following error

backgroundNet -- loading background network from:
           -- model       networks/Background-U2Net/u2net.onnx
           -- input_blob  'input_0'
           -- output_blob 'output_0'
           -- batch_size  1

[TRT]    TensorRT version 8.5.2
[TRT]    loading NVIDIA plugins...
[TRT]    Registered plugin creator - ::BatchedNMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::BatchedNMS_TRT version 1
[TRT]    Registered plugin creator - ::BatchTilePlugin_TRT version 1
[TRT]    Registered plugin creator - ::Clip_TRT version 1
[TRT]    Registered plugin creator - ::CoordConvAC version 1
[TRT]    Registered plugin creator - ::CropAndResizeDynamic version 1
[TRT]    Registered plugin creator - ::CropAndResize version 1
[TRT]    Registered plugin creator - ::DecodeBbox3DPlugin version 1
[TRT]    Registered plugin creator - ::DetectionLayer_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Explicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_Implicit_TF_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_ONNX_TRT version 1
[TRT]    Registered plugin creator - ::EfficientNMS_TRT version 1
[TRT]    Could not register plugin creator -  ::FlattenConcat_TRT version 1
[TRT]    Registered plugin creator - ::GenerateDetection_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchor_TRT version 1
[TRT]    Registered plugin creator - ::GridAnchorRect_TRT version 1
[TRT]    Registered plugin creator - ::GroupNorm version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 1
[TRT]    Registered plugin creator - ::InstanceNormalization_TRT version 2
[TRT]    Registered plugin creator - ::LayerNorm version 1
[TRT]    Registered plugin creator - ::LReLU_TRT version 1
[TRT]    Registered plugin creator - ::MultilevelCropAndResize_TRT version 1
[TRT]    Registered plugin creator - ::MultilevelProposeROI_TRT version 1
[TRT]    Registered plugin creator - ::MultiscaleDeformableAttnPlugin_TRT version 1
[TRT]    Registered plugin creator - ::NMSDynamic_TRT version 1
[TRT]    Registered plugin creator - ::NMS_TRT version 1
[TRT]    Registered plugin creator - ::Normalize_TRT version 1
[TRT]    Registered plugin creator - ::PillarScatterPlugin version 1
[TRT]    Registered plugin creator - ::PriorBox_TRT version 1
[TRT]    Registered plugin creator - ::ProposalDynamic version 1
[TRT]    Registered plugin creator - ::ProposalLayer_TRT version 1
[TRT]    Registered plugin creator - ::Proposal version 1
[TRT]    Registered plugin creator - ::PyramidROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::Region_TRT version 1
[TRT]    Registered plugin creator - ::Reorg_TRT version 1
[TRT]    Registered plugin creator - ::ResizeNearest_TRT version 1
[TRT]    Registered plugin creator - ::ROIAlign_TRT version 1
[TRT]    Registered plugin creator - ::RPROI_TRT version 1
[TRT]    Registered plugin creator - ::ScatterND version 1
[TRT]    Registered plugin creator - ::SeqLen2Spatial version 1
[TRT]    Registered plugin creator - ::SpecialSlice_TRT version 1
[TRT]    Registered plugin creator - ::SplitGeLU version 1
[TRT]    Registered plugin creator - ::Split version 1
[TRT]    Registered plugin creator - ::VoxelGeneratorPlugin version 1
[TRT]    completed loading NVIDIA plugins.
[TRT]    detected model format - ONNX  (extension '.onnx')
[TRT]    desired precision specified for GPU: FASTEST
[TRT]    requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]    [MemUsageChange] Init CUDA: CPU +188, GPU +0, now: CPU 232, GPU 10279 (MiB)
[TRT]    Trying to load shared library libnvinfer_builder_resource.so.8.5.2
[TRT]    Loaded shared library libnvinfer_builder_resource.so.8.5.2
[TRT]    [MemUsageChange] Init builder kernel library: CPU +106, GPU +104, now: CPU 360, GPU 10406 (MiB)
[TRT]    native precisions detected for GPU:  FP32, FP16, INT8
[TRT]    selecting fastest native precision for GPU:  FP16
[TRT]    found engine cache file /usr/local/bin/networks/Background-U2Net/u2net.onnx.1.1.8502.GPU.FP16.engine
[TRT]    found model checksum /usr/local/bin/networks/Background-U2Net/u2net.onnx.sha256sum
[TRT]    echo "$(cat /usr/local/bin/networks/Background-U2Net/u2net.onnx.sha256sum) /usr/local/bin/networks/Background-U2Net/u2net.onnx" | sha256sum --check --status
[TRT]    model matched checksum /usr/local/bin/networks/Background-U2Net/u2net.onnx.sha256sum
[TRT]    loading network plan from engine cache... /usr/local/bin/networks/Background-U2Net/u2net.onnx.1.1.8502.GPU.FP16.engine
[TRT]    device GPU, loaded /usr/local/bin/networks/Background-U2Net/u2net.onnx
[TRT]    Loaded engine size: 85 MiB
[TRT]    Trying to load shared library libcudnn.so.8
[TRT]    Loaded shared library libcudnn.so.8
[TRT]    Using cuDNN as plugin tactic source
[TRT]    Using cuDNN as core library tactic source
[TRT]    [MemUsageChange] Init cuDNN: CPU +342, GPU +234, now: CPU 687, GPU 10730 (MiB)
[TRT]    Deserialization required 1417078 microseconds.
[TRT]    [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +84, now: CPU 0, GPU 84 (MiB)
[TRT]    Trying to load shared library libcudnn.so.8
[TRT]    Loaded shared library libcudnn.so.8
[TRT]    Using cuDNN as plugin tactic source
[TRT]    Using cuDNN as core library tactic source
[TRT]    [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 688, GPU 10730 (MiB)
[TRT]    Total per-runner device persistent memory is 653824
[TRT]    Total per-runner host persistent memory is 414880
[TRT]    Allocated activation device memory of size 66172928
[TRT]    [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +64, now: CPU 0, GPU 148 (MiB)
[TRT]    The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[TRT]    
[TRT]    CUDA engine context initialized on device GPU:
[TRT]       -- layers       286
[TRT]       -- maxBatchSize 1
[TRT]       -- deviceMemory 66172928
[TRT]       -- bindings     8
[TRT]       binding 0
                -- index   0
                -- name    'input_0'
                -- type    FP32
                -- in/out  INPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  3
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 1
                -- index   1
                -- name    'output_0'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 2
                -- index   2
                -- name    'output_1'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 3
                -- index   3
                -- name    'output_2'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 4
                -- index   4
                -- name    'output_3'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 5
                -- index   5
                -- name    'output_4'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 6
                -- index   6
                -- name    'output_5'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]       binding 7
                -- index   7
                -- name    'output_6'
                -- type    FP32
                -- in/out  OUTPUT
                -- # dims  4
                -- dim #0  1
                -- dim #1  1
                -- dim #2  320
                -- dim #3  320
[TRT]    
[TRT]    binding to input 0 input_0  binding index:  0
[TRT]    binding to input 0 input_0  dims (b=1 c=3 h=320 w=320) size=1228800
[TRT]    binding to output 0 output_0  binding index:  1
[TRT]    binding to output 0 output_0  dims (b=1 c=1 h=320 w=320) size=409600
[TRT]    allocated 409600 bytes for unused binding 2
[TRT]    allocated 409600 bytes for unused binding 3
[TRT]    allocated 409600 bytes for unused binding 4
[TRT]    allocated 409600 bytes for unused binding 5
[TRT]    allocated 409600 bytes for unused binding 6
[TRT]    allocated 409600 bytes for unused binding 7
[TRT]    
[TRT]    device GPU, /usr/local/bin/networks/Background-U2Net/u2net.onnx initialized.
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] failed to create X11 Window.
[video]  no valid output streams, creating fake null output

[TRT]    ------------------------------------------------
[TRT]    Timing Report /usr/local/bin/networks/Background-U2Net/u2net.onnx
[TRT]    ------------------------------------------------
[TRT]    Pre-Process   CPU   0.03635ms  CUDA   0.17856ms
[TRT]    Network       CPU 271.17477ms  CUDA 270.57571ms
[cuda]   cudaEventElapsedTime(&cuda_time, mEventsGPU[evt], mEventsGPU[evt+1])
[cuda]      device not ready (error 600) (hex 0x258)
[cuda]      /home/frlab/jetson-inference/build/aarch64/include/jetson-inference/tensorNet.h:769
[TRT]    Post-Process  CPU   0.04771ms  CUDA   0.00000ms
[TRT]    Total         CPU 271.25882ms  CUDA 270.75427ms
[TRT]    ------------------------------------------------

[TRT]    note -- when processing a single image, run 'sudo jetson_clocks' before
                to disable DVFS for more accurate profiling/timing measurements


[TRT]    ------------------------------------------------
[TRT]    Timing Report /usr/local/bin/networks/Background-U2Net/u2net.onnx
[TRT]    ------------------------------------------------
[TRT]    Pre-Process   CPU   0.05776ms  CUDA   0.05968ms
[TRT]    Network       CPU  59.59396ms  CUDA  59.20493ms
[cuda]   cudaEventElapsedTime(&cuda_time, mEventsGPU[evt], mEventsGPU[evt+1])
[cuda]      device not ready (error 600) (hex 0x258)
[cuda]      /home/frlab/jetson-inference/build/aarch64/include/jetson-inference/tensorNet.h:769
[TRT]    Post-Process  CPU   0.04883ms  CUDA   0.00000ms
[TRT]    Total         CPU  59.70056ms  CUDA  59.26461ms
[TRT]    ------------------------------------------------


[TRT]    ------------------------------------------------
[TRT]    Timing Report /usr/local/bin/networks/Background-U2Net/u2net.onnx
[TRT]    ------------------------------------------------
[TRT]    Pre-Process   CPU   0.02909ms  CUDA   0.06000ms
[TRT]    Network       CPU  54.75547ms  CUDA  54.34557ms
[cuda]   cudaEventElapsedTime(&cuda_time, mEventsGPU[evt], mEventsGPU[evt+1])
[cuda]      device not ready (error 600) (hex 0x258)
[cuda]      /home/frlab/jetson-inference/build/aarch64/include/jetson-inference/tensorNet.h:769
[TRT]    Post-Process  CPU   0.04512ms  CUDA   0.00000ms
[TRT]    Total         CPU  54.82968ms  CUDA  54.40557ms
[TRT]    ------------------------------------------------

I think it's output error, but I'm not sure.
I would appreciate your help.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Hogushake