Problems pausing Core0 with WIFI.

[frohro]

Hi,
I am trying to collect data frem I2S on core1, and send it on using UDP on core0. It appears that using rp2040.IdleOtherCore(); causes problems on core0. I noticed this in my code and started testing simple programs to verify my theory that it wasn't a good idea to do idle core0 when using WIFI. Here is some simple test program where if I periodically idle core0, I have unexplained consequences. First, the /dev/ttyACM0 is not available on my host machine anymore. Second, I don't see udp data either. So if I do not comment out the rp2040. things in loop1 and all these terrible things happen. If I do, all is well. Is this expected behavior? It looks like I need to carefully think about how not to pause Core0.

/* Basic_Testing_UDP_Data_Stream for speed.  I get 990 kB/s with this code.
 * I am using the Ubuntu System Monitor to measure this.
 * Author: Rob Frohne, KL7NA, with help from Perplexity and Github Copilot.
 * 4/9/2024
 */
#include <Arduino.h>
#include <WiFi.h>
#include <WiFiUdp.h>
#include <si5351.h>

WiFiUDP udp;

void setup()
{

    WiFi.begin("Frohne-2.4GHz");
    while (WiFi.status() != WL_CONNECTED)
    {
        delay(10000);
    }

    udp.begin(12345);
    Serial.begin(115200);
}

const size_t BUFFER_SIZE = 1470; // Assuming the MTU is 1500 bytes...
char buffer[BUFFER_SIZE];
const char *udpAddress = "192.168.1.101"; // Put your laptop IP here.
const unsigned int udpPort = 12345;
uint32_t start_time;
uint32_t total_time = 0;
uint32_t n = 0;

void loop()
{
    // Send the buffer via UDP over and over again.
    // Read the data rate using the Ubuntu System Monitor.
    udp.beginPacket(udpAddress, udpPort);  // Takes about 7 us.

    start_time = micros();
    udp.write((const uint8_t *)buffer, BUFFER_SIZE);  // Takes about 810 us.
  
    udp.endPacket();  // Takes about 670 us
    int time_elapsed = micros() - start_time;
    total_time += time_elapsed;
    if (n++ == 500) 
    {
        Serial.printf("Time per statement: %f\n", (float)total_time / n);
        start_time = micros();
        total_time = 0;
        n = 0;
    }   
}

void setup1()
{
}

void loop1()
{
    rp2040.idleOtherCore();
    delay(100);
    rp2040.resumeOtherCore();
}

[earlephilhower]

Err, core0 runs the WiFi driver and the USB driver. Idiling it for 100ms (1/10th of a second) probably will kill your network and your USB connection. Also, the code as written will idle core0 basically forever. loop1 will be executed over and over meaning that there will be a couple microseconds when core0 won't be idled....

Why are you idling the 1st core, anyway? There's almost no reason for user code to need this (only SDK-native flash erase/write operations, really).

[earlephilhower]

Don't idle the other core, that's not the way to get where you want to go. Use something like a single writer/single reader queue which doesn't need any locking if implemented carefully. There are other methods involving mutexes/semaphores/etc. that you can also look into.

Idling the other core completely freezes it. That means interrupts are not serviced, that means HW timers are ignored, etc. Again, it's only really needed when using the raw SDK flash erase/write calls.

[frohro]

@earlephilhower thanks! I'll look into those methods.

[frohro]

@earlephilhower, can I use the #include <pice/mutex.h" safely with the Arduino Pico tools?

[frohro]

@earlephilhower , so I am still having some funny things happening. The code goes picking daiseys right in the middle of writing a UDP packet on

udp.write((const uint8_t *)&buffer, BUFFER_SIZE);  // It goes picking daiseys here.

I used the pico/mutex mutex.
Here is the whole code if you are interested.

/*
 * Project Name: Multicore_Test_UDP_I2S
 * File Name: Multicore_Test_UDP_I2S.cpp
 * This is a test of the multicore Arudino Pico librory for the Pico W
 * to use both cors and a circular queue of buffers to send data over UDP from the I2S
 * using the PCM1808.
 *
 * This Pico W program will connect to the PCM1808 via I2S and read the data it RATE samples per second.
 * It then sends the dato over WIFI using UDP.
 *
 *
 * Author: Rob Frohne, KL7NA, with help from Github Copilot.
 * 4/9/2024
 */
#include <Arduino.h>
#include <I2S.h>
#include <WiFi.h>
#include <WiFiUdp.h>
#include <si5351.h>
#include "pico/mutex.h"



#ifndef STASSID
#define STASSID "Frohne-2.4GHz"
#endif

// Use #define for common constants to both cores.
#define RATE 48000 // To do 96000, we need to use 24 bits instead of 32.
// For now we will use 32 bits for debugging ease.
#define MCLK_MULT 256 // 384 for 48 BCK per frame,  256 for 64 BCK per frame

static mutex_t my_mutex;
uint32_t mutex_save;

I2S i2s(INPUT);

WiFiUDP udp;
const char *udpAddress = "192.168.1.101"; // Put your laptop IP here.
const unsigned int udpPort = 12345;

#define BUFFER_SIZE 1468 // 183 samples of 4 bytes + 4 bytes for the packet number
#define QUEUE_SIZE 9
class CircularBufferQueue
{
private:
    char buffers[QUEUE_SIZE][BUFFER_SIZE];
    uint32_t fillIndex;
    uint32_t emptyIndex;

public:
    CircularBufferQueue() : fillIndex(1), emptyIndex(0) {}

    char *getNextBufferAndUpdate(bool isFiller)
    {
        uint32_t &currentIndex = isFiller ? fillIndex : emptyIndex;
        uint32_t otherIndex, temp;
        if (rp2040.fifo.available() == 0)
        {
            otherIndex = isFiller ? emptyIndex : fillIndex; // Return nullptr if the other op is not done.
        }
        else
        {
            while (rp2040.fifo.pop_nb(&temp)) // Gets most recent otherIndex, empties FIFO
            {                                 // This does not appear to work if the FIFO is empty.  Need to check if it is empty.
                otherIndex = temp;
            }
        }
        if ((currentIndex + 1) % QUEUE_SIZE != otherIndex)  // If we can move to the next buffer...
        {
            currentIndex = (currentIndex + 1) % QUEUE_SIZE;
            rp2040.fifo.push(currentIndex);
            Serial.printf("Going: fillIndex: %d, emptyIndex: %d, isFiller is %d.\n", fillIndex, emptyIndex, isFiller);
            return buffers[currentIndex];
        }
        else
        {
            Serial.printf("Stopped: fillIndex: %d, emptyIndex: %d, isFiller is %d.\n", fillIndex, emptyIndex, isFiller);
            return nullptr; // Return null if the buffer is full/empty
        }
    }
};

class BufferFiller
{
private:
    CircularBufferQueue &queue;

public:
    BufferFiller(CircularBufferQueue &q) : queue(q) {}

    void fillBuffer()
    {
        while(!mutex_try_enter(&my_mutex, &mutex_save)) {
            // Mutex is locked, so wait here.
            Serial.println("Mutex locked in filler.");
        }
        char *buffer = queue.getNextBufferAndUpdate(true);
        mutex_exit(&my_mutex);

        Serial.printf("Got filler buffer %p\n", buffer);
        if (buffer == nullptr)
        {
            digitalWrite(17, LOW);
            Serial.printf("Pin 17 should be high now.\n");
            digitalWrite(19, HIGH);
        }
        else
        {
            static int32_t r, l, packet_number = 0;
            uint32_t bufferIndex = 4;
            while (bufferIndex < BUFFER_SIZE - 4) // Leave space for the packet number
            {
                i2s.read32(&l, &r);
                l = l << 9;
                r = r << 9;
                memcpy(buffer + bufferIndex, &l, sizeof(int32_t));
                bufferIndex += sizeof(int32_t);
                memcpy(buffer + bufferIndex, &r, sizeof(int32_t));
                bufferIndex += sizeof(int32_t);
            }

            memcpy(buffer, &packet_number, sizeof(int32_t));
            packet_number++;
            Serial.printf("Filled packet %d\n", packet_number);
            digitalWrite(17, HIGH);
            digitalWrite(19, LOW);
        }   
    }
};

class BufferEmptyer
{
private:
    CircularBufferQueue &queue;

public:
    BufferEmptyer(CircularBufferQueue &q) : queue(q) {}

    void emptyBuffer()
    {
        while (!mutex_try_enter(&my_mutex, &mutex_save)) {
            Serial.println("Mutex locked in emptyer.");
        } 
        char *buffer = queue.getNextBufferAndUpdate(false);
        mutex_exit(&my_mutex);
        Serial.printf("Got emptying buffer %p\n", buffer);
        if (buffer != nullptr)
        {
            Serial.println("Sending packet.");
            udp.beginPacket(udpAddress, udpPort);
            Serial.println("after beginPacket.");
            udp.write((const uint8_t *)&buffer, BUFFER_SIZE);  // It goes picking daiseys here.
            Serial.println("after write.");
            udp.endPacket();
            Serial.printf("Sent packet %d\n", *(int32_t *)buffer);
            digitalWrite(16, LOW);
            digitalWrite(18, HIGH);
        }
        else
        {
            digitalWrite(16, HIGH);
            digitalWrite(18, LOW);
        }
    }
};

CircularBufferQueue bufferQueue;

void setup()
{ // This runs on Core0.  It is the UDP setup.
    Serial.begin();
    mutex_init(&my_mutex);  
    if (!mutex_try_enter(&my_mutex, &mutex_save)) {
            Serial.println("Mutex is already locked, better do it the other way around");
        } else {
        WiFi.begin(STASSID);
        while (WiFi.status() != WL_CONNECTED)
        {
            delay(1000);
        }
        udp.begin(udpPort);
        Serial.printf("Connected to %s\n", STASSID);
        pinMode(16, OUTPUT);
        pinMode(17, OUTPUT);
        pinMode(18, OUTPUT);
        pinMode(19, OUTPUT);
        mutex_exit(&my_mutex);
    }
}

void setup1()
{                   // This runs on Core1.  It is the I2S setup.
    i2s.setDATA(2); // These are the pins for the data on the SDR-TRX
    i2s.setBCLK(0);
    i2s.setMCLK(3);
    // Note: LRCK pin is BCK pin plus 1 (1 in this case).
    i2s.setSysClk(RATE);
    i2s.setBitsPerSample(24);
    i2s.setFrequency(RATE);
    i2s.setMCLKmult(MCLK_MULT);
    i2s.setBuffers(32, 0, 0);
    i2s.begin();
    mutex_enter_blocking(&my_mutex);  // This should syschornize the cores, so one doesn't fill the buffer
    // before the other is ready.
    mutex_exit(&my_mutex);
}

void loop()
{ // This should run on Core0.  It is the UDP loop.
    Serial.printf("Core 0\n");
    static BufferEmptyer emptyer(bufferQueue);
    emptyer.emptyBuffer(); // Empty the buffer
}

void loop1()
{ // This should run on Core1.  It is the I2S loop.
    Serial.printf("Core 1\n");
    static BufferFiller filler(bufferQueue);
    filler.fillBuffer(); // Fill the buffer
}

I thought maybe the mutex was interacting with the rp2040.fifo... commands, but it doesn't seem to be. I messed around with this test program and it seemed to work fine.

/* Basic_Testing_UDP_Data_Stream for speed.  I get 990 kB/s with this code.
 * I am using the Ubuntu System Monitor to measure this.
 * Author: Rob Frohne, KL7NA, with help from Perplexity and Github Copilot.
 * 4/9/2024
 */
#include <Arduino.h>
#include <WiFi.h>
#include <WiFiUdp.h>
#include <si5351.h>
#include "pico/mutex.h"

static mutex_t my_mutex;

WiFiUDP udp;
void setup()
{

    WiFi.begin("Frohne-2.4GHz");
    while (WiFi.status() != WL_CONNECTED)
    {
        delay(10000);
    }

    udp.begin(12345);
    Serial.begin(115200);
}

const size_t BUFFER_SIZE = 1470; // Assuming the MTU is 1500 bytes...
char buffer[BUFFER_SIZE];
const char *udpAddress = "192.168.1.101"; // Put your laptop IP here.
const unsigned int udpPort = 12345;
uint32_t start_time;
uint32_t total_time = 0;
uint32_t n = 0;
uint32_t save;
uint32_t junk;

void loop()
{
    // Send the buffer via UDP over and over again.
    // Read the data rate using the Ubuntu System Monitor.
    udp.beginPacket(udpAddress, udpPort); // Takes about 7 us.
    if( rp2040.fifo.available() > 0)
    {
        rp2040.fifo.pop_nb(&junk);
    }
    start_time = micros();
    if (mutex_try_enter(&my_mutex, &save))
    {
        Serial.println("Mutex entered Core0.");
        udp.write((const uint8_t *)buffer, BUFFER_SIZE); // Takes about 810 us.
        udp.endPacket();                                 // Takes about 670 us
        int time_elapsed = micros() - start_time;
        total_time += time_elapsed;
        Serial.println("Sent packet.");
        if (n++ == 5000)
        {
            Serial.printf("Time per statement: %f\n", (float)total_time / n);
            start_time = micros();
            total_time = 0;
            n = 0;
        }
        mutex_exit(&my_mutex);
    }
    else
    {
        // Serial.println("Mutex not entered Core0.");
    }
}

void setup1()
{
    mutex_init(&my_mutex);
}

void loop1()
{
    // delay(100);
    if (!mutex_try_enter(&my_mutex, &save))
    {
        // Serial.println("Mutex not entered Core1.");
    }
    else
    {
        // Serial.println("Mutex entered Core1.");
        rp2040.fifo.push_nb(0);
        mutex_exit(&my_mutex);
    }
}

The output from my problematic code is this:

Connected to Frohne-2.4GHz
Core 1
Core 0
Mutex locked in emptyer.Going: fillIndex: 2, emptyIndex: 0, isFiller is 1.

Got filler buffer 0x200033bc
Going: fillIndex: 2, emptyIndex: 1, isFiller is 0.
Got emptying buffer 0x20002e00
Sending packet.
after beginPacket.
Filled packet 1

After this it is picking daiseys somewhere in Never-Never Land. :-) If anyone has any bright ideas, I'd love to hear them.
Rob

[JoeEarlam]

Hi Rob, I've written some dual core programmes for the the Pico W. I've never used the mutex functions for this, but I use a combination of "busy" flags (volatile variables, like you'd use for an interrupt routine) and the TaskScheduler library, which allows me to call tasks on one core when I know the other core has finished its actions.

The CircularBuffer library is also really useful, I use it for things like queuing log messages from many functions running on multiple cores, this gives me a single output handler that runs when I want, and avoids race conditions where I try write to the same serial or SD card interface from different cores at the same time. It is not explicitly multicore safe but it is interrupt safe-ish, and I haven't had any significant issues with it handling large amounts of serial data from both cores simultaneously.

For your application here you could push blocks of audio data into a circular buffer from core 1, then write a core0 network handler that pushes any new blocks over UDP when they arrive in the buffer. It's a little memory intensive, but it makes it really easy to implement things like TCP (where you don't destroy the data until it's confirmed received) or parse it into other data outputs like logging to SD card etc.

[frohro]

I got it working now, without funny delays, by copying the buffer to a temporary one before doing the udp.write().

/*
 * Project Name: Multicore_Test_UDP_I2S
 * File Name: Multicore_Test_UDP_I2S.cpp
 * This is a test of the multicore Arudino Pico librory for the Pico W
 * to use both cors and a circular queue of buffers to send data over UDP from the I2S
 * using the PCM1808.
 *
 * This Pico W program will connect to the PCM1808 via I2S and read the data it RATE samples per second.
 * It then sends the dato over WIFI using UDP.
 *
 *
 * Author: Rob Frohne, KL7NA, with help from Github Copilot.
 * 4/9/2024
 */
#include <Arduino.h>
#include <I2S.h>
#include <WiFi.h>
#include <WiFiUdp.h>
#include <si5351.h>
#include "pico/mutex.h"

const char *STASSID = "Frohne-2.4GHz";

const int RATE = 96000; // Works with 32 bits per sample at 96 ks/s.  This is 768 kBps.  We cound save by using 24 bits per sample.
const int MCLK_MULT = 256; // 384 for 48 BCK per frame,  256 for 64 BCK per frame

static mutex_t my_mutex;
uint32_t mutex_save;

I2S i2s(INPUT);

WiFiUDP udp;
const char *udpAddress = "192.168.1.101"; // Put your laptop IP here.
const unsigned int udpPort = 12345;

#define BUFFER_SIZE 1468 // 183 samples of 8 bytes + 4 bytes for the packet number
#define QUEUE_SIZE 9  // Maximum latency is 8 packets, which is 15 ms.  Average is 2 packets, or 4 ms.

class CircularBufferQueue
{
private:
    char buffers[QUEUE_SIZE][BUFFER_SIZE];
    uint32_t fillIndex;
    uint32_t emptyIndex;

public:
    CircularBufferQueue() : fillIndex(1), emptyIndex(0) {}

    char *getNextBufferAndUpdate(bool isFiller)
    {
        uint32_t &currentIndex = isFiller ? fillIndex : emptyIndex;
        uint32_t otherIndex, temp;
        if (rp2040.fifo.available() == 0)
        {
            otherIndex = isFiller ? emptyIndex : fillIndex; // Return nullptr if the other op is not done.
        }
        else
        {
            while (rp2040.fifo.pop_nb(&temp)) // Gets most recent otherIndex, empties FIFO
            {                                 // This does not appear to work if the FIFO is empty.  Need to check if it is empty.
                otherIndex = temp;
            }
        }
        if ((currentIndex + 1) % QUEUE_SIZE != otherIndex) // If we can move to the next buffer...
        {
            currentIndex = (currentIndex + 1) % QUEUE_SIZE;
            rp2040.fifo.push(currentIndex);
            return buffers[currentIndex];
        }
        else
        {
            return nullptr; // Return null if the buffer is full/empty
        }
    }
};

class BufferFiller
{
private:
    CircularBufferQueue &queue;
    
public:
    BufferFiller(CircularBufferQueue &q) : queue(q) {}

    void fillBuffer()
    {
        while (!mutex_try_enter(&my_mutex, &mutex_save))
        {
            // Mutex is locked, so wait here.
        }
        char *buffer = queue.getNextBufferAndUpdate(true);
        mutex_exit(&my_mutex);
        if (buffer != nullptr)
        {
            static int32_t r, l, packet_number = 0;
            uint32_t bufferIndex = 4;
            while (bufferIndex < BUFFER_SIZE - 4) // Leave space for the packet number
            {
                i2s.read32(&l, &r);
                l = l << 9;
                r = r << 9;
                memcpy(buffer + bufferIndex, &l, sizeof(int32_t));  // We could speed this up for the network by sending 24 bits instead of 32.
                bufferIndex += sizeof(int32_t);
                memcpy(buffer + bufferIndex, &r, sizeof(int32_t));
                bufferIndex += sizeof(int32_t);
            }
            memcpy(buffer, &packet_number, sizeof(int32_t));
            packet_number++;
            Serial.printf("Filled packet %d\n", packet_number);
        }
    }
};

class BufferEmptyer
{
private:
    CircularBufferQueue &queue;
    char temp_buffer[BUFFER_SIZE];
public:
BufferEmptyer(CircularBufferQueue &q) : queue(q) {
    memset(temp_buffer, 0xff, BUFFER_SIZE);
}

    void emptyBuffer()
    {
        while (!mutex_try_enter(&my_mutex, &mutex_save))
        {
            // Mutex is locked, so wait here.
        }
        char *buffer = queue.getNextBufferAndUpdate(false);
        mutex_exit(&my_mutex);
        if (buffer != nullptr)
        {
            udp.beginPacket(udpAddress, udpPort);  // Needed
            memcpy(temp_buffer, buffer, BUFFER_SIZE);  // If we don't do this, it hangs in the udp.write below.
            udp.write((const uint8_t *)&temp_buffer, BUFFER_SIZE); 
            udp.endPacket();
            Serial.printf("Sent packet %d\n", *(int32_t *)buffer); 
        }
    }
};

CircularBufferQueue bufferQueue;

void setup()
{ // This runs on Core0.  It is the UDP setup.
    mutex_init(&my_mutex);
    if (mutex_try_enter(&my_mutex, &mutex_save))  // Synchronze cores so they start about the same time.
    {
        Serial.begin();
        // delay(10);
        WiFi.begin(STASSID);
        while (WiFi.status() != WL_CONNECTED)
        {
            delay(1000);
        }
        udp.begin(udpPort);
        Serial.printf("Connected to %s\n", STASSID);
        mutex_exit(&my_mutex);
    }
}

void setup1()
{                   // This runs on Core1.  It is the I2S setup.
    i2s.setDATA(2); // These are the pins for the data on the SDR-TRX
    i2s.setBCLK(0);
    i2s.setMCLK(3);
    // Note: LRCK pin is BCK pin plus 1 (1 in this case).
    i2s.setSysClk(RATE);
    i2s.setBitsPerSample(24);
    i2s.setFrequency(RATE);
    i2s.setMCLKmult(MCLK_MULT);
    i2s.setBuffers(32, 0, 0);
    i2s.begin();
    mutex_enter_blocking(&my_mutex); // This should syschornize the cores, so one doesn't fill the buffer
    // before the other is ready.
    mutex_exit(&my_mutex);
}

void loop()
{ // This should run on Core0.  It is the UDP loop.
    static BufferEmptyer emptyer(bufferQueue);
    emptyer.emptyBuffer(); // Empty the buffer
}

void loop1()
{ // This should run on Core1.  It is the I2S loop.
    static BufferFiller filler(bufferQueue);
    filler.fillBuffer(); // Fill the buffer
}