MSXV New Testing Framework [WIP]

Owned by Jarvis Weng
Last updated: May 11, 2022
5 min read

This is for design testing only. Not for hardware tests.

3 Types of testing

  • hardware tests (only on hardware)

  • design tests (only on simulation)

  • smoke tests (both?)

 

Due to the inclusion of FreeRTOS, tasks and functions must be designed in a way for ease of testability. Testing is super important especially in a real vehicle and it’s much more important that a vehicle is debug-able and understandable instead of optimal.

 

Newest VSCode has great use of the debugger:

 

Inter Task communication

So there are different way tasks can communicate with each other, thus different ways for the tests to snoop data. The different ways in FreeRTOS are as follows

  • Global Variables

  • Queues

  • Messages

  • Stream Buffers

  • Function overloading

Global Variables

Global variables are defined on the top of the .c file and are usually static. This means that the variable is confined within the file its defined in. Since this is for design testing, we will predominately use these to snoop the internal data.

Queues

FreeRTOS queues is a FIFO that can hold a specific number of elements of a fixed size. This is usually used within a project. Tests should not access queues directly as that can create more issues.

Messages

Message buffers are variable length messages that can be sent from an ISR to a task or from task to task. Currently not really using it, but if we did, it would be complicated to test. Thus will not be accessing message buffers directly in tests.

Stream Buffers

Stream buffers are similar to messages except that it sends a stream of bytes one at a time. Since this is dynamically allocated we will not use this, but again should not be used directly in tests.

Function Overloading

Function overloading is a great way to change a function behaviour. Using this, its possible to change inputs and change firmware behaviour in the middle of a run. We can use function overloading to get the data that should be returned to a function, or change the output accordingly. This is what was predominately used in MSXIV and we could use it again, however, not strictly necessary.

Summary:

So in order to separate project functionality vs project testing, it’s important to keep them as far as part as possible. We want to keep projects as “black box” as possible in order to validate inputs and outputs. So as long as functions are black box, it’s possible and useful to do function overloading. However, something like overloading an entire task is not useful since we’re not testing anything useful.

Thus, it’s important to minimize the effect that tests have on the actual project to make sure that what is being tests is project functionality only.

 

Design of Testing Architecture (Steering)

So the design of the testing architecture will be similar to the CAN architecture. See CAN Autogeneration for more details.

Essentially since this is design testing and simulation only, don’t need to worry about the memory usage. So we will defined output and input global variables to evaluate “black box” testing.

I will be using the “steering” project from firmware_xiv for how describing the new architecture.

Steering main():

So steering starts off with initializing all its module, then start processing the event.

Steering has 3 main functionalities:

  • Processing digital inputs

  • Processing analog inputs

  • Processing CAN data

Processing Digital Inputs

This is in steering_digital_input.c and essentially it sets GPIO interrupts to raise an event if an interrupt occurs.

Processing analog inputs

This is in steering_control_stalk.c and essentially it calls an ADC read periodically. Every ADC read, it’ll run the callback and check the value parameters. It will raise an event if the ADC read value satisfies the conditions.

Processing CAN data

This is in steering_can.c and just sends CAN messages based on the event that’s read.

Steering Summary:

Thus there are 3 main functionality of steering which will roughly translate to their own TASKS.

 

Project Design

The project design for steering should be as follows

 

initialize all GPIOs, SPI, etc initialize all TASKS start TASKS in cycle cycle = 0 while (true) CAN receive Task -> Non-concurrent GPIO Tasks -> Concurrent ADC Tasks -> Concurrent CAN Send Task -> Non-concurrent cycle++ #ifdef TESTING post(cycle_finish) mutex(test_mux) #endif

 

Projects should start with initialization of all modules and then start running each tasks. Each tasks should be run in what’s called a cycle.

  • So the CAN receiving tasks starts first. This is because other tasks depend on the CAN messages coming in.

  • Next the GPIO and ADC tasks can run. These don’t depend on each other, thus can be parallel

  • Finally CAN send tasks runs last. This depends on GPIO and ADC to finish before running.

 

In the case of testing, there’s 2 additional functions to be run. A post operation indicating the end of a cycle, and a mutex operation that will block the tasks from running.

 

The test that is run will look something like this

 

void test_setup() { setups all the modules starts the tasks } void run_cycle(int c) { for (int i = 0; i < c; i++) { mutex(cycle_finish); post(test_mux); } } void test_steering() { set steering inputs run_cycle(1); trigger GPIOs run_cycle(1); change ADC values run_cycle(5); .... }

 

And this should be the framework. While the test_steering input is running, all other tasks should be blocked. Once test_steering runs run_cycle then test_steering is block and all other tasks should run.