Zephyr: Message Queues for Real-Time Processing with the Notecard

Zephyr RTOS is a powerful operating system designed for embedded real-time applications. It contains a multitude of powerful functionality designed to handle the complex process of scheduling tasks and events. The Zephyr scheduler can be cleverly used to offload non-urgent work tasks while maintaining the time sensitive processing needed for supporting operations like Interrupt Service Routines or ISRs.

In this example, we'll show you how to easily utilize Zephyr's Message Queue API to trigger Notecard requests in response to a button press. While this example will be kept simple for educational purposes, real world use cases for this could include: time sensitive data telemetry from a sensor array, a precision flight controller system that needs to manage the ESC of a drone or perhaps an industrial use case where the temperature of a system must be precisely maintained. Given that communication between a Host and Notecard might take an arbitrary length of time, it's a better practice to handle this outside of your real-time event processing.

This tutorial applies to any Zephyr supported devices but to follow along you'll want the following:

• Any Blues Notecard
• Blues Notecarrier-F
• Zephyr supported Feather MCU, such as the Swan MCU
  • Debugging Tool, such as the STLINK-V3MINIE

What are Message Queues?

According to the Zephyr Docs, a message queue is described as the following:

"A message queue is a kernel object that implements a simple message queue, allowing threads and ISRs to asynchronously send and receive fixed-size data items."

Breaking it down even further, here are some of the benefits you can leverage by using message queues in your application:

• Keep ISRs (Interrupt Service Routines) short and fast
• Can safely access shared resources with proper synchronization
• Separates event detection (ISR) from event handling (work handler)
• Can prioritize or deprioritize deferred work relative to other threads
• Provides predictable execution timing
• No need for additional mutex or semaphore management
• Works with Zephyr's power management subsystem

Zephyr safely abstracts complex aspects of the cross thread management, including potential race conditions, memory safety, buffer overflows as well as synchronization mechanisms. It's a powerful way to move data around inside of your application, in particular to and from a Notecard.

How to use Message Queues

Message queues are a powerful way to offload work from an ISR to a thread. In this example, we'll show how to use an interrupt on a button press to increment a counter and write the value of the counter to a Notefile, using a threaded message queue. We'll skip over some of the Zephyr scaffolding for using buttons and initializing the Notecard to focus on the message queues.

Following the flow diagram above, we'll start by initializing the utilities required for the message queue. We'll create a thread that will process the message queue, set up the interrupt and added a callback to the user button and we'll configure the Notecard. This is important as we want to immediately return from the ISR and not block the main thread, instead performing the work in the message queue thread. The "work" in this case is the handling of a button press, which we'll use to increment a counter and write its value to a Notefile.

Taking a look at main.c

Following along with the example project, let's dive into the main.c file and have a look at how message queues are initialized / utilized.

Initializing Macros

Zephyr provides a number of macros to help with initializing message queues and threads.

#define STACK_SIZE 1024
#define PRIORITY 5
#define MSG_Q_SIZE 10

K_MSGQ_DEFINE(button_msgq, sizeof(uint32_t), MSG_Q_SIZE, 4);
K_THREAD_STACK_DEFINE(process_stack, STACK_SIZE);

To create a message queue, you can call the macro K_MSGQ_DEFINE, which takes the following arguments:

• q_name Name of the message queue.
• q_msg_size Message size (in bytes).
• q_max_msgs Maximum number of messages that can be queued.
• q_align Alignment of the message queue's ring buffer (power of 2).

You'll then want to define a thread to process message queue, using the K_THREAD_STACK_DEFINE macro, you can instruct Zephyr to construct the memory allocation for the thread:

• sym Thread stack symbol name
• size Size of the stack memory region

Button Callback

Let's now set up a callback to handle our button press interrupt:

static void button_pressed(const struct device *dev, struct gpio_callback *cb,
                          uint32_t pins)
{
    ARG_UNUSED(dev);
    ARG_UNUSED(cb);
    ARG_UNUSED(pins);

    // Send message to queue instead of scheduling work
    uint32_t count = 1;
    if (k_msgq_put(&button_msgq, &count, K_NO_WAIT) != 0) {
        LOG_ERR("Failed to queue button press");
    }
}

K_NO_WAIT signifies that the timeout should be set to NULL, limiting the time spent in the button handler.

We're not interested in any of the entry point arguments (in this callback), so we label them with ARG_UNUSED.

Processing the Message Queue (Thread)

The next thing we need to do is create a function to handle the message queue processing:

static void process_button_thread(void *dummy1, void *dummy2, void *dummy3)
{
    uint32_t count;
    J *req = NULL;
    J *body = NULL;
    int ret;

    ARG_UNUSED(dummy1);
    ARG_UNUSED(dummy2);
    ARG_UNUSED(dummy3);

    while (1) {
        // Wait for message with timeout
        ret = k_msgq_get(&button_msgq, &count, K_MSEC(1000));

        if (ret == -EAGAIN) {
            continue;  // Timeout occurred
        }

        if (ret != 0) {
            LOG_ERR("Error receiving message: %d", ret);
            continue;
        }

        // Process the button press
        button_counter += count;
        LOG_INF("Counter value: %d", button_counter);

        // Create the request
        req = NoteNewRequest("note.add");
        if (!req) {
            LOG_ERR("Failed to allocate request");
            continue;
        }

        // Create the body
        body = JCreateObject();
        if (!body) {
            LOG_ERR("Failed to allocate body");
            continue;
        }

        // Build the request
        JAddStringToObject(req, "file", "button.qo");
        JAddNumberToObject(body, "counter", button_counter);
        JAddItemToObject(req, "body", body);

        // Send the request
        if (!NoteRequest(req)) {
            LOG_ERR("Failed to add note.");
        }
    }
}

K_msgq_get retrieves messages on the button_msgq queue using a first in, first out (FIFO) mechanism and writes them to the address of count. Using the note-c API, we can then write a variable named counter in the body of a note called button.qo on the Notecard.

There's an important feature to pay attention to here; this implementation is blocking and pauses the thread to check if there is a message on the queue. This doesn't halt the execution of the main function as it's isolated to this thread but you may consider batch processing (non-blocking) if you expect to generate a high volume of messages.

Initializing the Thread

Diving into the main function, we can take a look at how the process_button_thread is initialized:

// Start processing thread
k_thread_create(&process_thread, process_stack, STACK_SIZE,
                process_button_thread, NULL, NULL, NULL,
                PRIORITY, 0, K_NO_WAIT);
k_thread_name_set(&process_thread, "button_process");

This is a simple thread implementation, so we're only interested in the pointers to the thread struct process_thread and thread function process_button_thread.

• new_thread Pointer to uninitialized struct k_thread
• stack Pointer to the stack space.
• stack_size Stack size in bytes.
• entry Thread entry function.
• p1 1st entry point parameter.
• p2 2nd entry point parameter.
• p3 3rd entry point parameter.
• Prio Thread priority.
• options Thread options.
• Delay Scheduling delay, or K_NO_WAIT (for no delay).

The thread priority PRIORITY indicates the importance to the schedule that this task is given processing time, in this example 5. You can read more about thread priority in the Zephyr Docs. In Zephyr, threads are scheduled based on their priority, so a higher priority thread will be given more processing time than a lower priority thread. The lower the number, the higher the priority.

K_thread_name_set is a helpful function for tracing & debugging, allowing you to quickly see what each thread is doing.

Setting up the Button Interrupt

Finally, we need to set up the button interrupt and attach the button handler (callback) to it:

// Configure button interrupt
ret = gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_TO_ACTIVE);
if (ret != 0) {
    LOG_ERR("Error: failed to configure button interrupt: %d", ret);
    return ret;
}

// Set up GPIO callback
gpio_init_callback(&button_cb_data, button_pressed, BIT(BUTTON_PIN));
ret = gpio_add_callback(button.port, &button_cb_data);
if (ret != 0) {
    LOG_ERR("Error: failed to add button callback: %d", ret);
    return ret;
}

A final word: this tutorial omits a number of important production features relating to Zephyr & Notecard and should be treated as an introduction to message queues and threads, not as a best practices guide. Please see the Zephyr Docs & Notecard Docs for the latest advice on best practices.

Conclusion

This tutorial has been a quick look at how you can use some of the scheduling power of a real-time operating system like Zephyr to handle time sensitive events while still using the Notecard to log events. Message Queues are just one of the many mechanisms available to Zephyr for handling events and you may find that other mechanisms are more suitable for your applications. Following this guide should give you a good basis for building your own real-time event processing application with Zephyr and Notecard.