Final report for the Grant Program 2020 of Ruby Association
- Name: Renamed "mmruby" 👉 "PicoRuby"
- Syntax: Almost covered which I planned when the RA Grant program began.
- RAM consumption: Remarkably reduced.
- Correspondence to mruby3: Postponed it as the release of mruby3 delayed.
- Future work: Keyboard firmware, Microcontroller programming platform, More syntax and Test.
- A request to mruby and mruby/c
- Conclusion
Monstarlab https://monstar-lab.com
HASUMI Hitoshi https://github.com/hasumikin
March 15th, 2021
Today I renamed mmruby "PicoRuby" to clarify that it is dedicated to one-chip microcontrollers.
PicoRuby is, once again, an integration of "mini pico mruby compiler (mmrbc picorbc)" and "mruby/c virtual machine".
shell.rb is a demo application program which I showed in RubyKaigi Takeout 2020. It must have been compiled by mruby-compiler (mrbc) at that time though, pico mruby compiler came to be able to compile it.
picorbc now covers enough syntax to develop IoT products.
All the commits I merged during the RA Grant development can be seen here:
https://github.com/hasumikin/picoruby/compare/9c776f4...efb4cbe
I compared picorbc with mrbc regarding RAM consumption in the proposal for the RA Grant:
A summary of reexamination as below:
| Ruby script | mrbc(as of proposal) | mmrbc(as of proposal) | mrbc(tag: 3.0.0) | picorbc(as of today) |
|---|---|---|---|---|
| hello_world.rb | 157.8 KiB | 11.17 KiB | 133.6 KiB | 14.63 KiB |
| larger_script.rb | 162.9 KiB | 53.20 KiB | 135.2 KiB | 22.99 KiB |
From the above,
- mrbc
- RAM consumption decreased about 15% on
hello_world.rband 17% onlarger_script.rb
- RAM consumption decreased about 15% on
-
mmrbcpicorbc- Increased 30% on
hello_world.rb - Decreased 56% on
larger_script.rb
- Increased 30% on
We can say it's a good improvement in mrbc. On the other hand, what happened in picorbc?
I have taken three types of measures to reduce RAM consumption.
-
Reusing literal data
-
Pooled allocation
-
Discarding objects in a loop instead of recursion
But in short, no trick out there.
(This is an insignificant topic in terms of programing technique)
I found many wasteful, duplicated memory allocations of literal data.
I refactored my code to keep data common in a compilation cycle as far as I could.
Token data puts on memory should be reused as it is until exactly before generating VM code for example.
mmrbc had allocated each struct of linked list structure whenever it came to be necessary. However, the size of a pointer which is 4 bytes in 32 bit architecture is not ignorable. The sum of pointer size will be about 1 KiB if there are 250 items on the linked list.
I refactored my code so that picorbc allocates a fixed size of an array of struct in advance.
Each struct doesn't have "pointer member to link" but only "pooled array" has a pointer to *next (and/or *prev).
This is the reason that the RAM usage of compiling hello_world.rb increased from 11.17 KiB to 14.63 KiB due to pooled array which is allocated in advance.
However, we can find a remarkable result in compiling larger_script.rb which got a benefit from the pooled allocation.
The larger one script be, the more benefit it gets in general.
Besides, we can suppress a fragmentation of memory by introducing pooled allocation. It means the efficiency of memory management and total performance are improved.
Take a look at this graph
You can find a spike raising at the right most of the graph.
It apparently happened when "freeing" objects in recursion.
"Objects" in this context includes a linked list and tree structure.
An actual free() call will be induced at the end of a function call chain if you take a recursive strategy.
As a result, the function call stack became deep then RAM consumption became big.
I refactored my code to use a loop instead of recursion to free objects. A combination of this strategy and pooled allocation achieved a remarkable decrease in RAM usage as shown above.
As picorbc now can compile shell.rb, a comparison of compiling it also should be illustrated:
| Ruby script | mrbc(tag: 3.0.0) | picorbc(as of today) |
|---|---|---|
| shell.rb | 177.7 KiB | 43.96 KiB |
The footprint of picorbc is 4x smaller than mrbc in this case.
Note that all the examinations are carried out on 64 bit architecture. These values will be much smaller (likely about 30% minus) on 32 bit architecture.
Because the release date of mruby 3.0.0 was unfortunately late than I expected, I postponed upgrading the VM code which picorbc generates. It should be a part of future work.
Keyboard firmware will be a killer content of PicoRuby, I think. I'm now trying to make one with Raspberry Pi Pico. You will be able to configure your keymap and add programmable macros and anything else you want in a Ruby way.
Since PicoRuby is an interpreter, a one-chip microcontroller can be a friendly programming platform without any troublesome setup. I will create a PicoRuby ecosystem that supports newbies to learn programming.
What's more, thanks to mruby/c's runtime, the programming platform and keyboard firmware will run at the same time!
I wish I can give a talk about these topics on RubyKaigi Takeout 2021.
picorbc covered enough syntax regarding microcontroller product though, there are still many syntax unimplemented. I'm considering introducing a build option that validates advanced syntax features in exchange for an additional ROM (code size) consumption.
I've written a simple and minimal testing framework for pico mruby compiler to make it sanity. There are not enough test cases as of yet, I should write more cases. BTW, it is always welcome that anyone helps me to write tests.
Both mruby VM and mruby/c VM accept only VM code whose header section includes MATZ as the compiler name.
I would like to them accept HASU as an alternative compiler.
I think this will be helpful for people to debug if my compiler has bugs.
By courtesy of Ruby Association Grant, I got big progress in implementing pico mruby compiler. It now covers enough syntax of Ruby while running well on a chip that has a small resource like PSoC5LP.
I could also have plans of PicoRuby which are possibly able to extend the RubyWorld.
Finally, I would deeply like to thank Matz for his help as my mentor.