How to live graph in python [mjvFigure]?

[rohit-kumar-j]

I am trying to build a graphing tool built on top of mujoco-python-viewer for pid tuning ,etc. Intending to use it like this:

model = mujoco.MjModel.from_xml_path("path_to_file")
data = mujoco.MjData(model)
while True:
    viewer.graph_1(self.data.qpos[0])
    viewer.graph_1(actuation_signal) # actuation_signal is a data stream of float values (ex: from a loop)
    viewer.render()

I have looked at simulate.cc, but was not able to obtain the required result using profiler_update section.

1. How do we populate and call the graphing utilities in python for the live graph of joint values similar to the CPU-time profiler:

How to write to these pixels?

[JeanElsner]

It's probably easier and more pythonic to simply use a Python library to do the plotting. For example we use animated matplotlib widgets within a Qt UI to do live plots, adjust parameters etc.

[nimrod-gileadi]

I agree with @JeanElsner's answer, but if you do want to use mjvFigure, it should be possible.
Please share the code you used to try to replicate profiler_update.

I'd follow what happens in simulate.cc to figtimer, and check that your code does all of it:

1. It's initialized with mjv_defaultFigure - this will be done for you automatically by the Python bindings.
2. Various properties are set on it for axes, colours, etc.
3. figtimer.linepnt and figtimer.linedata are set.
4. On every frame, profilershow is called, which calls mjr_figure with a particular viewport and the render context. This would be mujoco.mjr_figure in python.

[yuvaltassa]

You are absolutely right. While MuJoCo has great documentation for most things, plotting (and UI) rendering is an exception. We intend to fix this very soon with comprehensive documentation, but in the meantime here is the type definition and here is the 2D plot implementation. I apologise that this is the best we can do for now.

[rohit-kumar-j]

@yuvaltassa What is float linedata[mjMAXLINE][2*mjMAXLINEPNT]; // line data (x,y)? Is it [x][y] values as in:

pixel_value = figure.linedata[x][y] # either 1 or 0

With a resolution of 2000*100 (Y*X) in pixels per graph?

[yuvaltassa]

You can draw up to mjMAXLINE lines, each line with at most mjMAXLINEPNT points. The XY values values of the data are in linedata. The memory space is preallocated with the maximum values. The actual lines that you want to plot is given by

int     linepnt[mjMAXLINE];     // number of points in line; (0) disable

Each integer in linepnt[i] is either 0 (don't draw line i) or n>0 (draw n data points from line i).
Makes sense?

[rohit-kumar-j]

Thank you, I was able to figure that out now, I have been working with the following lines. The [i],[i+1],[i-1],[i+2],[i-2] is not clear though.

How do we write to the [2*mjMAXLINEPNT]?

self.fig.linedata[0][i] = sensor_data[0] #sensor 0 data
self.fig.linedata[0][i - 1] = 0.0
self.fig.linedata[0][i + 1] = 0.0
self.fig.linedata[0][i - 2] = 0.0
self.fig.linedata[0][i + 2] = 0.0

[rohit-kumar-j]

Is this layout correct?:
With O as (0,0), and point B addressed something like:

# B is at(2,-1) relative to (0,0)
b_xpos = 2 
b_ypos = -1

# write data
fig.linedata[line_id][4] = sensor_data[line_id]
fig.linedata[line_id][-1] = sensor_data[line_id]

# clear previous data (unclear)
fig.linedata[line_id][b_ypos-2] = 0.0
fig.linedata[line_id][b_ypos-1] = 0.0

[rohit-kumar-j]

Thanks for your answers!
@JeanElsner I did try blitting with matplotlib, but wanted to add more functionality to interact with the simulation. like an autotuning algorithm for PID, so I wanted to use the inbuilt plotter in mujoco.

@nimrod-gileadi This is the code for the graph I have used(I can provide the entire code, but most of it might not be necessary I think):

    def __init__(self, model, data):
        self.fig = mujoco.MjvFigure()
        # Manual
        self.fig.range[0][0] = -5
        self.fig.range[0][1] = 5
        self.fig.range[1][0] = -5
        self.fig.range[1][1] = 5
        self.fig.flg_extend = 0
        self.fig.flg_symmetric = 0
        self.pid_viewport = mujoco.MjrRect(0, 0, 800, 200)


    def sensorupdate(self):
        num_sensors = self.model.nsensor
        sensor_data = self.data.qpos
        adr = self.model.sensor_adr
        print("num_sensors: ", num_sensors, sensor_data, adr)
        p = self.fig.linepnt[0]
        pnt = mujoco.mju_min(201, self.fig.linepnt[0] + 1)
        # print(f"pnt: {pnt}")
        for n in range(0, 3):
            # shift data
            for i in range(1, 0):
                self.fig.linedata[n][2 * i + 1] = self.fig.linedata[n][2 * i - 1]
                print(f"n:{n} i:{i}")

            # assign new
            self.fig.linepnt[n] = pnt
            self.fig.linedata[n][1] = sensor_data[n]
        print(self.fig.linedata)

    # show sensor figure
    def sensorshow(self, rect):
        mujoco.mjr_figure(self.pid_viewport, self.fig, self.callbacks.ctx)

    def render(self): #This is called in each loop
        self.sensorupdate()
        self.sensorshow(rect=self.pid_viewport)
        mujoco.mjr_overlay(
                        mujoco.mjtFontScale.mjFONTSCALE_100,
                        mujoco.mjtGridPos.mjGRID_BOTTOMLEFT,
                        self.pid_viewport,
                        "t1",
                        "t2",
                        self.callbacks.ctx,
                    )

mjvFigure-problem.mp4

[rohit-kumar-j]

@nimrod-gileadi @yuvaltassa Thanks for the support! I was able to figure out how it works.

• I needed to set some of the axes to automatic scaling
• The math according to the simulate.cc #L423

Will update the link to the repo.

Preview:

• The sine wave is a data_stream added with math.sin(x) inside the main loop
• Rest of the lines are joint values

mujoco_graph_working.mp4

[yuvaltassa]

Excellent!

Very happy you figured this out, and apologies again for the lacking documentation.

[rohit-kumar-j]

@yuvaltassa no worries 😅 .Mujoco with python is still new. I'm sure it will take time to fully mature.

This branch has a rudimentary example in examples/graphs.py

[yaoxt3]

Hello, could you please share the entire code? This link has expired. Thanks in advance.

[rohit-kumar-j]

@yaoxt3 my fork of mujoco_python_viewer. Although simulate viewer should have a similar functionality now.

https://github.com/rohit-kumar-j/mujoco-python-viewer