Home

The Tactile Board is a mobile Augmentative and Alternative Communication (AAC) device for individuals with deafblindness. The Tactile Board allows text and speech to be translated into vibrotactile signs that are displayed real-time to the user via a haptic wearable.

This repository contains code for the Python scripts that are responsible for listening for messages that arrive from the Tactile Board¹ application through the MQTT message bus.

If you are looking for the repository of the Android application, click here.

The following video demonstrates how the Tactile Board is used.

Requirements

For USB: PyCmdMessenger https://pypi.org/project/PyCmdMessenger/
For BLE: bluepy https://pypi.org/project/bluepy/
For I2C: adafruit-pca9685 https://github.com/adafruit/Adafruit_Python_PCA9685
Paho MQTT Client: paho-mqtt https://pypi.org/project/paho-mqtt/

Setup

Install the above named python packages via pip. To execute, simply run the command from the root directory:

python Main.py

If you have your own MQTT message broker, change the required credentials in the provided config.json file.

By default the program runs with an I2C interface. If you are using a device connected through USB change line 20 in Main.py to:

with UsbVestDevice("YOUR_COM_PORT") as vest_device, MqttMessageService() as mb:

Use the respective COM port (e.g. “COM3”).

Config

A configuration file is provided to adjust certain settings such as the MQTT broker endpoint and the required credentials. See the configuration page here for further details.

Additionally, it is possible to redirect the activation of a pin to another. This can be done by setting the OverridePinLayout value to true. When doing so the program will use the values provide in the Layout property.

Authors

James Gay (james.gay@hs-offenburg.de)

Citing

If you use this code in your research, you must cite:

¹ Arthur Theil, Lea Buchweitz, James Gay, Eva Lindell, Li Guo, Nils-Krister Persson, and Oliver Korn. 2020. Tactile Board: A Multimodal Augmentative and Alternative Communication Device for Individuals with Deafblindness. In 19th International Conference on Mobile and Ubiquitous Multimedia (MUM 2020), November 22–25, 2020, Essen, Germany. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3428361.3428465