Skip to content

Home

Jump to bottom
Pratim Ugale edited this page Jan 8, 2022 · 27 revisions

PRU User Space API

Get the Code

This is a Google Summer of Code 2019 project under BeagleBoard.org aimed at providing:

  1. An API for different programming languages to load/unload firmware, start/stop the PRUs and communicate with them from the Linux User Space using the RemoteProc, RPMsg drivers.
  2. Sample PRU firmware and User Space software to demonstrate the use of the project.

Project Details

Build Status
Code: https://www.github.com/pratimugale/PRUSS-Bindings
Mentors: Kumar Abhishek, Zubeen Tolani(ZeekHuge), Patryk Mężydło.
GSoC Project Page: https://summerofcode.withgoogle.com/projects/#5163407328673792
Introduction Video explaining about the project: https://www.youtube.com/watch?v=3Z2PxDIoCpE
Final Video: https://www.youtube.com/watch?v=9G-IIQX89J4
The work I have done/changes I have made to the code are covered in the pull requests

Table of Contents

Brief overview of PRU-ICSS

Starting with the PRU, a Programmable Real-Time Unit (PRU) is a low-latency microcontroller subsystem present on BeagleBoard devices, which provides real-time processing capability lacking in Linux. PRUs are part of the PRU-ICSS, “Industrial Communications SubSystem”. They are free from running on an operating system, and thus can be dedicated to a single function.
The PRU cores were created to be non-pipelined, single-cycle execution engines. This means that all instructions (except for memory reads and writes) complete in a single cycle. Since there is no pipeline, the PRU cores are completely deterministic: the PRU cores will not rearrange instructions during execution.
The PRU-ICSS can also read from or write to external pins in a single cycle using a PRU core register (R30). With a PRU core running at 200 MHz, this means that an external signal change can be detected in as little as 5 nanoseconds. This also means that the PRU can toggle external pins with as fast as 5 nanoseconds.

PRU-ICSS

PRU0 and PRU1 can talk to each other through their shared data memory space. 0x0000_0000 through 0x0000_1fff (8 KB) is the PRU Data RAM0 and 0x0000_2000 through 0x0000_3fff is the PRU Data RAM1 memory.
Data RAM0 is intended to be the primary data memory for PRU0, as is Data RAM1 for PRU1. However, both PRU cores can access Data RAM0 and Data RAM1 to pass information between PRUs. Each PRU core accesses their intended Data RAM at address 0x0000_0000 and the other Data RAM at address 0x0000_2000.
PRU0 and PRU1 also have a shared data memory (PRU SRAM) at 0x0001_0000 through 0x0001_2fff (12 KB).
PRU0 and PRU1 have 8 KB of instruction memory, where executable code should be located.

The PRU-ICSS also has an interrupt controller, it does not support asynchronous interrupts as they introduce jitter in execution time and reduce determinism. Instead it supports an efficient polling system.

How to develop firmware for the PRU

Although not required while using this project, before creating general purpose PRU firmware, one must understand the basics of how the RemoteProc and RPMsg frameworks work.

RemoteProc

The RemoteProc Linux driver is responsible for taking the PRU firmware from the Linux filesystem, parsing it for any resources that it must provide for the PRU (for example, interrupts or shared memory), loading the firmware into PRU instruction memory and data memory, and then start running the PRUs and stop them as required. A resource table is necessary for the RemoteProc driver to work, even if it is empty.

RPMsg

User applications can require communication between the PRUs and the ARM core. Linux provides a method for this communication called RemoteProc Messaging (RPMsg). This communication between the PRU and the ARM core using RPMsg can be easily done using the PRUSS-Bindings for which there is a pruss_api driver.

PRU firmware

PRU firmware can be written using C code or assembly language. C code is more approachable to new users, but it may not get the best performance. This is because the number of PRU clock cycles that will be used is unpredictable while using C. While using assembly, we are sure that every instruction will take exactly 5ns i.e. 1 PRU cycle to execute (except memory access).
Inline assembly code can be used while running the C firmware. Otherwise, a whole different .asm file can be called by the C-code. Control can be passed back from .asm to the C-firmware by using R3.w2, which is a special register used by the clpru compiler.

About the project and how it simplifies things

Although the PRUs are extremely useful for offloading deterministic tasks, using these processors has quite a learning curve and is a long process.

The process for working with the PRUs goes like this:-

  1. Write the PRU firmware, compile and link it using clpru and lnkpru.
  2. Copy the output elf object file to /lib/firmware/
  3. Copy the name of this output object file to /sys/class/remoteproc/remoteproc1/firmware so that the RemoteProc driver knows which firmware file to load from /lib/firmware when the PRU is started.
  4. Start the PRU:
    • sudo modprobe pru_rproc (only if the module is not already loaded, in current versions, this module is automatically loaded after rebooting)
    • cd /sys/class/remoteproc/remoteproc{N}/
      • Start the PRU execution of the firmware by echo start | sudo tee state
      • Stop the PRU execution by echo stop | sudo tee state
  5. If RPMsg is being used, sending/receiving messages to/from PRU takes place like this:
    • cd /dev/
    • echo {message} | sudo tee rpmsg_pru{N}.
      It becomes inconvenient if the data to be sent and received from the PRU in raw/binary form. Also if the program depends on these messages, we further have to deal with root permissions.
  6. To run the PRU in single step mode,
    • There is again a need to change the working directory to /sys/kernel/debug/remoteproc/remoteproc{N}
    • echo 1 | sudo tee single_step
    • To view the PRU Control and General Purpose Registers for debugging:
      sudo cat regs

It can be a hassle to do these tasks, especially when working with multiple firmware.
Sometimes, there is a need for the user space program to send commands to PRU over RPMsg, and the next command could depend on the message sent back by the PRU.
While debugging, often it is required to run multiple steps rather than only a single_step. Manually, this would mean a lot of echoing into the single_step file.
This is why there is a need of a better interface to perform all these tasks.

All the above tasks are performed by prussd.py, which runs as a daemon service on the Linux system.
The cpp-bindings interacts with the daemon service and provides an interface to the user to perform those tasks.

Need of a daemon service

The users have to deal with root permissions all the time if they want to perform operations on the sysfs files. Using a daemon service (prussd.py) helps get around this issue and facilitates the read/write through simple functions.
The prussd.py and cpp-bindings interact with each other by using socket communication over /tmp/prussd.sock.
High level scripting languages like python can use the daemon service by calling the cpp-bindings' functions using SWIG. The SWIG interface files are provided here: https://github.com/pratimugale/PRUSS-Bindings/tree/master/swig

The following UML diagram describes pruss.cpp in cpp-bindings, which is finally the interface provided to the user. cpp-bindings-uml
An example of how the API can be used to accurately control stepper motors:
Code: https://github.com/pratimugale/PRUSS-Bindings/tree/master/examples/firmware_examples/example7-stepper-control

Installing and Using the API

There are two ways for installation:-

  1. Download the packaged .deb file from releases.
    Run dpkg -i pruapi_1.0-1_armhf.deb
    1. Clone this repository:
      git clone https://github.com/pratimugale/PRUSS-Bindings
    2. Run the install script
      cd PRUSS-Bindings/
      bash install.sh
    3. The API is now ready to use, run any specific example from PRUSS-Bindings/examples/firmware_examples/ by cd'ing into the directory and running make. The Makefile will compile the PRU-firmware, load them on to the PRU(using /lib/firmware), compile the userspace program and run it.

Make sure that RPMsg is working, here's a guide for it.

How to use the API?

After installation, this is how a simple userspace program looks like:

#include <iostream>
#include <pruss.h>

using namespace std;
int main()
{
	PRUSS& p = PRUSS::get();
	PRU p1 = p.pru1;
	if(!p1.load("/home/debian/gsoc/PRUSS-Bindings/examples/firmware_examples/example2-rpmsg-pru1/PRU1/am335x-pru1-fw"))
		cout << "Firmware loaded\n";
        else
                return -1;
	p1.enable();
	string s;
	cout << "Enter a message   : ";
	getline(cin, s);
	p1.sendMsg_string(s);
	cout << "Message from PRU  : "<< p1.getMsg();
	p1.disable();
	p.shutDown();
	return 0;

}

If installation is done from the debian package, compile using:

g++ userspace.cpp -L/usr/lib -lpruss

If installed from source, the processor directive must be #include "path/to/cpp-bindings/pruss.h". Run this program by:

g++ userspace.cpp /path/to/cpp-bindings/pruss.cpp -o userspace.o
./userspace.o

The above program performs the following tasks in sequence:

  • Probes the pru_rproc driver for exposing the sysfs files provided by remoteproc.
  • Sets the channel number for the PRU object for RPMsg communication.
  • Loads the appropriate firmware into /lib/firmware.
  • Enables(starts) the correct PRU (PRU1 in this case).
  • At this point, the pruss_api driver is probed because the firmware uses RPMsg communication and has pruss-api as CHAN_NAME. This is what probes this particular driver.
  • Takes a message input from the user and passes it to /dev/pruss_api_pru1. This file is created by the pruss_api driver.
  • Reads the same device file again to check if there was any message sent back by the PRU.
  • Disables(stops) the PRU1 and shuts down PRUSS (Unprobes pru_rproc)

Work Summary

The following work I have done/changes I have made to the code are covered in the pull requests.

  • Wrote mem_read(), mem_write() functions that perform read/write 1-byte integer values to the PRU SRAM/DRAM0/DRAM1 using /dev/mem. These are helpful for debugging purposes as demonstrated in example6.
  • Wrote sendMsg_raw() and sendMsg_string() functions to send data to the PRU through RPMsg device file in raw form as well as string format. Sending data in raw form makes it easier for the data to be reconstructed on the PRU side.
  • Wrote language bindings for C
  • Created various firmware examples to demonstrate the use of the API, which demonstrate the benefit of using the API:
    • example1: [Code]: A simple GPIO toggling/blinky example.
    • example2: [Code]: RPMsg Communication - Demonstrate writing of RPMsg firmware by sending messages back and forth.
    • example3: [Code]: A PWM generator maximum frequency of almost 1MHz.
    • example4: [Code]: Generating a single channel analog waveform on P9_31 GPIO using PWM at its root.
    • example5: [Code]: 4-channel PWM written in assembly based on the example given in PRU Cookbook.
    • example6: [Code]: A debugging program to read/write to PRU SRAM/DRAM using /dev/mem; and to demonstrate the showRegs() function which displays all the current values of the PRU Control Registers and PRU General Purpose Registers(only if the PRU is not running).
    • example7: [Code]: A library to accurately control stepper motors which are driven by the PRU using PRU0 to PRU1 interrupts and RPMsg.
    • example8: [Code]: To demonstrate how to link together multiple .asm files and call them from the C-program. Control is passed back to the C-program by using R3.w2
    • example9: [Code]: Expanding Waveform Generation to 2, 4, and 8-channels. As the number of channels increase, the maximum frequency of the waveform is decreased.
  • Implemented a pruss_api loadable kernel module that creates character device files of the form pruss_api_pruN specific for using with this project (derived from rpmsg_pru): https://github.com/pratimugale/PRUSS-Bindings/blob/master/drivers/pruss_api.c
  • Created a .deb package for easy installation of the software using dpkg. The download link is in releases.
  • Resolved and removed small bugs previously present in the project. For example:
    1. pruss.cpp had buffer length of 1kB when the function showRegs() required more data to be transferred through /tmp/prussd.sock
    2. load_firmware() function that loads firmware into the PRUs went inside an unwanted 'if' block if firmware path didn't begin with /lib/firmware.
  • Made a solution in user-space for a kernel oops issue mentioned in the mailing list: https://github.com/pratimugale/PRUSS-Bindings/commit/bc4ba9070c68a3c3d686c8d28a3f07b83f656d99

Currently we are in the process of upstreaming the code into the package distribution system.

Conclusion

I think that main goals that were behind this project have been met, but there are still some features that can be added. The debugger utility 'prudebug' is yet to be integrated, but a general purpose debugging program is demonstrated in example6.

I will continue to support this project in the future, take care of any bugs and issues and will continue implementing additional features whenever required.

It has been an amazing experience working with BeagleBoard.org and I would like to thank all mentors and org admins for the continued support, specially my mentors Abhishek_ and ZeekHuge.

Known Issues

There is a kernel oops NULL pointer dereference error that shows up which is described here: https://groups.google.com/d/msg/beagleboard-gsoc/711XmP7jFMc/sE0PvoScAwAJ
Update: The problem is solved in user-space by waiting until the rpmsg device file appears in the /dev directory.

ToDo (as of 24 May 2020)

  • Update to support the new BeagleBoard.org Debian Buster IoT Image 2020-04-06 - Some changes in pruss_api driver and packaging required.
  • Document the process for how the Debian Packaging was done.
  • Complete and update the ''node-bindings''
  • Are remoteproc/rpmsg sysfs filenames changed? - Check
  • Implement a better message passing mechanism (Single function for messages of different datatypes, something like OpenMPI)
  • Add DMA Support - Using /dev/mem is a very bad way of accessing the PRU's Data Memory, try and make updates to the PRU DMA project's driver
  • Port/Add examples using the pru-gcc/gnu-pru
  • Make a permanent fix in the rpmsg driver that leads to the kernel oops issue referenced in http://pratimugale.me/gsoc/2019/07/30/KernelOOPS.html
  • Support for AI, X15

Additional Details

Clone this wiki locally