Telemetry at FSGP 2019

Owned by Aaron Tan
Jul 13, 2019

For telemetry at FSGP 2019, the telemetry server built in Golang was used, with the changes by Misha Zharov to send/receive data via a REST endpoint. In addition, a quick script to forward all CAN messages was used to log all CAN messages sent over system CAN by the car. The telemetry backend as well as the SocketCAN forwarding script ran both on the Pi in the car and on a Linode VPS in Toronto.

InfluxDB and Grafana were used to visualize the data. A small JavaScript script was used to forward data sent over the websocket by the telemetry backend to push data to InfluxDB, which was then used as a data source by Grafana to create a dashboard.




SocketCAN Forwarding

SocketCAN forwarding consists of two parts:

  • A Python script that both:
    • Ingests data from a specified CAN bus and sends it to a websocket (the sender), and
    • Receives messages over a websocket, then broadcasts that data over a CAN bus (the receiver).
  • A node.js script that receives data over a websocket and forwards this to other websockets (the forwarder).

The sender should run on the Raspberry Pi in the car. The forwarder should run on a server. The receiver should run on any device where you wish to receive the CAN messages from.

The sender and receiver expects a command-line argument pointing to a JSON file. The JSON should contain two fields, channel and server_url, which indicate:

  • Which CAN interface to listen on (in the case of the sender), or which CAN interface to broadcast messages on (in the case of the receiver)
  • What url to send the data to (in the case of the sender), or which url to listen for data on (in the case of the receiver).

Setup

Install requirements for sender/receiver script by with pip: pip install -r requirements.txt.

Install requirements for forwarding script: node install ws.

Usage 

Sender: python3 pi.py config-sender.json

Receiver: python3 pi.py config-receiver.json

Forwarder: node server.js

On the computer running the receiver, you can then do candump -L vcan1 >> some_file.log  to log all CAN messages in a format that can be replayed by canplayer.

See the socketcan-forwarding folder under branch feature/aaronhktan/update_can for source code and more details.

Telemetry Backend

The telemetry backend runs on both the server and the car.

  • In the car, it ingests data from a CAN message source (serial, fake, SocketCAN), and serves the driver display webpage.
  • In the server, it ingests data from the REST endpoint, logs the data to a SQLite database, and serves a driver display webpage.

Setup

See the telemetry repo for setup instructions.

Usage


In the car: The driver display script should handle this: driver_dispay_startup.sh

On the server: ./bin/telemetry start --token="test" --source=r --serverPort="8080" --schema=can_messages.asciipb --dbConnectString test.db

Misc. Notes:

Updating CAN message definitions

  • Use codegen to generate new Go files.
  • Put can_msg_defs.go in canmsgdefs under pkg.
  • Put can.pb.go in protos under pkg.
  • Build with make. If building for Raspberry Pi, use env GOOS=linux GOARCH=arm GOARM=5 make


Driver Display

Driver display is served by the telemetry backend. It can be accessed by Chrome/Chromium at <server_url>:8080/driver_display.html.


Misc. Notes

Updating CAN message definitions

  • Use codegen to generate new Typescript files.
  • Put can_msg_defs.ts in src/ts under client
  • Compile with tsc in client.

InfluxDB and Grafana

Piggybacking off the websocket created by the telemetry backend on the server that provides data, a small Node.js/Javascript script ingests data and puts it in an InfluxDB database. This database is then used as a source by Grafana, which allows us to view historical data and create new graphs or visualizations as necessary.

Setup


InfluxDB

Grafana

Glue script

  • node install ws
  • node install influx

Usage

  • Create a database in Influx:
    • influx
    • create database <database_name>
  • Use web browser to use Grafana:
    • <server_url>:3000
    • Default username/password is admin/admin
    • In the sidebar, add a data source:
      • Configuration (settings gear) → Data Sources → Add a Data Source → InfluxDB
        • URL: localhost:8086
        • Database Name: <your database name>
        • Min time interval: 1ms
    • Then, create a dashboard, and add panels.
  • Send data from telemetry backend to Influx (aka glue script):
    • node send.js
  • Backup/restore
    • To back up data from InfluxDB:
      • influxd backup -portable <path to backed up data>
    • To restore data to InfluxDB:
      • influxd restore -portable <path to backed up data>

Misc. Notes

  • To update CAN message definitions, copy the can_msg_defs.js compiled by Typescript for the driver display and place it in the same directory as the glue script.