A simulator for the farmbot

[Bracewind]

Hello everyone,

I contacted Rory Aronson some time ago about the potential creation of a simulator for the farmbot. I take advantage of the coronavirus to find some free time at last.

First, I would like to know your opinion on the subject, especially if it will be useful for you, and if yes what do you need most. For instance, a simulation could be used to test/debug the code, or it could be released and used as a marketing tool to let anyone play with the web interface. Depending on your answer, the technology and the way it is put together with the current code could be different.
To me, the simulator would be a way to play with the code and see how it works before eventually buying the real robot in the years to come.

If you want to go further, i will give you more details about which simulator will be used, where to link the simulator to the code… There is also some details that I need to discuss with you. And obviously, all that will depend on what you want.

Finally, I must be honest, I will not work on it a lot of time and it won’t be ready before 5 to 6 months minimum. If you think you will be able to dispatch someone to work on the simulator, please let me know, as it will influence the technologies used.

Best regards,
Grégoire Pichereau

[RickCarlino]

Hi @Bracewind

It's a busy day for me here at FarmBot, so it will be some time before I can provide a full response, but this is a very interesting idea.

I will provide a longer response some time this week.

Thanks for reaching out to us!

[Bracewind]

Ok, thanks for giving me an answer despite your workload,
have a nice day and good luck !

[RickCarlino]

@Bracewind OK- sorry for the delay. Here is a more in-depth response.

I would like to know your opinion on the subject

This is a really fun idea, and we've had quite a few folks request it in the past. I think this would be fun for people who are passively curious about FarmBot.

a simulation could be used to test/debug the codebase

I could see the feature being useful to non-developers, but for a software developer that is not concerned with visualization or aesthetics, it is already possible to simulate a FarmBot by purchasing a RAMPS board on Amazon (~$25 USD), flashing the FarmBot firmware to it, and running FBOS on a host laptop.

This is how I do most local development for FarmBot features. It might also be a good way for you to test things out as you build the simulator.

I will give you more details about which simulator will be used

I am happy to continue the discussion in this Github thread. If it looks like you are very serious about the project, I can hop on a call with you later on.

We also were contacted by some folks this week that had a similar interest and are using IsaacSim, I'm not sure if you've heard of that before (I have not).

From the perspective of a developer, I'd be happy to share advice on the parts of the stack that would be best suited for simulation.

If you think you will be able to dispatch someone to work on the simulator, please let me know

FarmBot has two software developers (including myself) so that's not an option for us. It's also worth mentioning that our ability to change FarmBot source code to meet the needs of this project would be limited as well, since our main focus is meeting the needs of paying customers.

There are ways that a simulator could be built without changes to the FarmBot codebase. Since I don't know how your idea fits in to the roadmap or our coding standards, I can't promise any changes to existing FarmBot code to meet the needs of a simulator. I am fairly certain a simulation could be built unobtrusively, though I would need to know more about your tool chain to provide options, though. I look forward to learning more about the tools you plan on using.

I hope this helps! Speak soon.

[Bracewind]

@RickCarlino No problem, this is a low priority improvement, so take all the time you need

I could see the feature being useful to non-developers, but for a software developer that is not concerned with visualization or aesthetics, it is already possible to simulate a FarmBot by [...] running FBOS on a host laptop.

I think that more than asthetics, it also helps to simulate what would happen when there is physical inconstencies, for instance if there is less friction between two links, and see if the measures taken by the program are enough to deal with that case. A simple stub may do the trick, but having a physical engine doing the job for you is reassuring.

Anyway, considering what you said, it would probably be more useful to have a beautiful simple model than an ugly complex one.

This is how I do most local development for FarmBot features. It might also be a good way for you to test things out as you build the simulator.

Thanks for the tip, I will definitely use this.

We also were contacted by some folks this week that had a similar interest and are using IsaacSim, I'm not sure if you've heard of that before (I have not).

I don’t know the simulators used in the industry, as most of them are expensive. I made a little search about IssacSim, and the version useful for manipulative robot is still at an early stage. It seems it is a good choice when the robot is controlled using machine learning, as the simulator is probably faster than other simulators, since it is developed by Nvidia. But when it comes to real simulation, I think it will be less realistic than others, and allow for less options. Moreover, I’m not sure it will be completely free when it will be released...

There are three simulators I found interesting : V-rep, Gazebo and Webots.
I still need to test them all more to know which one I will use. I will choose the one that works well on windows, is easy to launch, is easy to interact with, and also one which allows me to easily develop the whole thing.

I know well Gazebo, so I’m keen on using it, but it is probably not the right choice for something easy to use.

our ability to change FarmBot source code to meet the needs of this project would be limited

Changing FarmBot source code never crossed my mind. I think a good way to do this would be to get the g-code generated for arduino and transfer it to the computer, then parse and execute it so it does the movements asked on the simulator. However, this won’t truly be the movements of the farmbot if PID controller or other control systems are executed by the firmware and not the OS, which is probably the case. But I think it’s not really relevant for this project.

If you have anything you want to add or correct, feel free to do so. I will come back after analyzing a little more each simulator

[RickCarlino]

I think a good way to do this would be to get the g-code generated for arduino and transfer it to the computer, then parse and execute it so it does the movements asked on the simulator.

This is a good idea. If you need a starting point (or just want to see how GCode interactions occur), you could flash FarmBot OS onto a RaspberyPi. FBOS can handle the Arduino being unplugged (or not having an Arduino at all). This might be the fastest way to get test data.

Please let me know if you have any questions.

[Bracewind]

A summary of the available choices is given after this paragraph if you are in a hurry

I've tested the three simulators and.... it will be more complex than what I thought ;)

• V-rep : I've tried to import the collada files of the farmbot, but it does not work and crash. Only collada files are accepted with the free version of the software and no color or materials can be imported (Only materials coded using Obj format are accepted by V-rep). Everything seems perfect in V-rep, and maybe creating the structure without the textures could work. It would be ugly but it would work.
• Gazebo : Contrary to what is told, Gazebo is only available on Linux. I tried all afternoon to install it on Windows and it's hard and long. But on Ubuntu, the collada files were very well imported !! Well, I didn't try to move the farmbot, but I know Gazebo and I don't think it will be hard, a little long though.
• Webots : Webots does not support importing 3D design files, and it will be added in the next release. Stl and collada files will be supported, so it could be a serious option when the release will be done. Moreover, contrary to V-rep, the code source is available, so I will be able to tweek a little the functionnality if needed.

You are the one that needs to choose :

• No visual but windows support
• No windows but "good" (gazebo is not really good looking) looking visual.
• Wait for Webots to release the functionnality and work on the parser instead. It also delays the simulator selection.

To be honest, I don't really know who will use the simulator (except me) and I think you know that better than me, this is why I cannot make this decision myself.

Since a parser will be needed for the project, I also was wondering if a g-code parser had already been implemented somewhere.

If you want more details or need more information, feel free to ask.

[RickCarlino]

@Bracewind

You are the one that needs to choose

Since it sounds like you are doing this as a side project for fun, I would definitely pick the one that you find most rewarding / enjoyable to work with. With that being said, from a "usefulness" perspective, I think a simulator is best suited for non-technical users (as I mentioned in a previous message, there are already a variety of tools available for software developers). Having something that is visual and Windows based is probably the best fit for a non-technical user base. On the flip side, we only use Linux for software development here at FarmBot.

work on the parser instead.

It sounds like you are familiar with the design of parsers. One thing to keep in mind about this parser is that it will most likely be one of the most simple you have ever written. The official FarmBot G-Code parser is here. Depending on your programming background, there is an old Ruby-based GCode parser that can be found here. The GCode spec has not changed too much in 5 years, aside from some new additions.

FarmBot Gcode is easily parsed using string splitting and case statements. Use of more advanced techniques (such as regex or string scanners) might be overkill. What language are you using for the parser? A good experiment to try would be to flash FarmBot firmware onto an Arduino Mega and view the output in a serial terminal, such as Hyperterm.

[Bracewind]

I’ve seriously tried to use webots, and I could load the model whithout materials using plugins. But I regularly experiment crashes, and I think the cause is huge memory leak in the software… So I will use Gazebo, even though it is clearly not the optimal choice :/
Running Gazebo on Windows is possible using windows subsystems, not sure if it will be performant though.

I’ve started to create the visual, but to add physical properties such as collisions, I would need models with less mesh than what is available on Onshape. I suppose it has not been done so I will try to do it.

One thing to keep in mind about this parser is that it will most likely be one of the most simple you have ever written. [...] A good experiment to try would be to flash FarmBot firmware onto an Arduino Mega and view the output in a serial terminal, such as Hyperterm.

I looked at the parsers you gave me and you’re right it is quite simple. The code will be done in c++ in the whole project because I think it will be easier to interoperate with the arduino, and because it will be easier to optimize the code if needed. To be consistent, the parser will also be in c++. Since it will probably not be very difficult, and since I still don’t have the Arduino (shipping is longer right now), I will do the parser later.

[gueux]

@Bracewind
How about this one as example? http://amapstudio.cirad.fr/soft/amapsim/start
Seems it has already trained models for some kind of plants that can visualise farming process.with some kind of plugins that can simulate farmbot behaviour.

[Bracewind]

@gueux
Could you be a little more specific ? This software seems to simulate the plant growth, and I don't find plugins for simulating harvest.(I looked at this page http://amapstudio.cirad.fr/soft/amapsim/c_tools/d_plug-ins/start). I'm interested however to see how it works behind the scene, and if a plugin could do the harvesting part, which is something currently impossible, or at least difficult to do, on Gazebo.

I worked on other things the last year, and could not really find time to develop the simulator. For now, I only did a very simple graphical control with no physics and no gcode.