What is MPXE?

• Multi-Project x86 Emulation, the idea is to run multiple projects at once for integration tests

• Current state: fully testing one project at a time. Future state: fully testing integration between projects.

• More complete testing than unit tests!

High level overview

• A Python script runs each project and can access the state of each driver

• E.g. pedal project, it reads an ADC for throttle and sends CAN messages based on the project

  • We can set readings for the ADS1259 ADC from MPXE tests! Changing the values that the C reads in memory! That way we can fully test pedal

Open

MPXE project (C-side) components:

• unique scripts per project

• per-driver and per-library mpxe versions

• “store” library accessed by drivers and libraries

  • the “store” is the MPXE term for the information needed by the drivers/libs

  • store library provides access to the store accessible from Python and C

  • e.g. for ADC, it’s the 4 voltage values it reads, for GPIO, it’s the state of each GPIO pin

  • store is specified in protobufs (google “protocol buffers”) to provide access from Python and C

• for a driver to update the store: e.g. in gpio, it loads all its state into the store library’s data, then calls store_export to write it to memory

MPXE project manager

• a script, communicates with projects

• project manager maintains a polling thread to listen for information from projects and store it

• Mock scripts:

  • Whenever polling thread receives an update, it’s passed to the mock script for the driver/component to update the store

  • Also exposes methods for functions to call

  • They encapsulate functionality implemented in hardware/ICs

• There’s also a CAN listener that stores the n most recent CAN messages

ADS1015 driver - setting up and reading from the store

• it’s an ADC, so it’ll only read from the store, not set the store, so data only flows Python → C

• currently, MPXE versions of drivers are hidden inside #ifdef MPXE

• Every driver will have a static variable for the store, which will store the hardware state

  • it’s a protobuf generated struct, like MxAds1015Store

• When the driver is initialized, it calls prv_init_store

  • Init flow:

  • Call store_config: sets up listener thread for updates from project manager (Python), open fifo for C code to write to harness (opens communication channels) - call it in every driver, it guards against multiple initialization

  • Set up a StoreFuncsstruct to register generated protobuf functions with store, call store_register - so the store library knows how to deal with the type

    • most of the functions are for working with protobufs

    • GetPackedSizeFunc, PackFunc, UnpackFunc, FreeUnpackedFunc are code-generated by protobuf

    • The only one we write ourselves is UpdateStoreFunc - takes a protobuf message and a protobuf mask, goal is to get Python to update C code by allowing updating some of the state

      • the “mask” binary blob is sort of like a bitmask, it tells us what fields are updated by the message

      • the UpdateStoreFunc sets each item in the store if the mask is nonzero for that item, e.g. in ADS1015

      • it also has to call the free unpacked function since the unpack function uses the heap

      • every driver has to implement this

  • if there are any repeated fields in the protobuf, you have to set the n_* fields of the store with the number, and you also have to malloc the memory for the array (ADS1015 allocates and never frees it since the process will be dead when it’s no longer needed)

  • store_register also takes a key for multiple storages ?, but we can assume it’s always null for now

• So how do we read from the store?

  • Very driver-dependent, but ADS1015 always reads things on a timer, so in this case we read directly from the store

  • Basically whenever you read a value from hardware, read it from the store (don’t just set a default value) so that Python can set the value that you read

Writing to the store - gpio example

• We need to export the state to the store/Python code in gpio whenever a hardware value is update (e.g. gpio_set_state)

• You need to write an export function for any driver that exports to the store

  • It sets the store state to the state of the driver, then call store_export

    • this serializes it as protobufs, adds type information, then sends to python)

  • (some drivers use separate state from the store, so it has to set that to store)

How do you run MPXE?

• Prereq: have protobuf installed (probably already installed from codegen-tooling)

• See mpxe requirements installation for more setup info

• Install protobuf-c: github.com/protobuf-c/protobuf-c

  • Clone that repo to ~/shared, cd into it, run ./autogen.sh && ./configure && make && make install

To run MPXE:

What does/should an MPXE integration test look like?

They go in mpxe/integration_tests.

We’re using the Python unittest library for now. We do need an API wrapping unittest for assertions, setup, teardown.

Action items for improving it:

• figure out how to define test dependencies in makefile like rules.mk to avoid running make from python, lol

• set up an IntegrationTest class inheriting from unittest.TestCase

  • setup would start pm and start projects, teardown would stop the projects

• add a stop all running projects function in project manager for the integration test base class

• we also want a mock superclass for handling updates and logs