Tensorflow lite setup
Setup python environment (Ubuntu 20.04)
Get pip
and virtualenv
installed:
sudo apt-get install python3-pip
sudo apt-get update
sudo apt-get install python3-venv
Create a new environment and activate it:
python3 -m venv tf
source tf/bin/activate
Install the tensorflow
library and, if needed, opencv
:
pip install tensorflow
pip install opencv-contrib-python
Tensorflow lite
If there is no need to train models, a TensorFlow lite environment can be used by itself. One needs to select the correct version from this list and install it. For example on Ubuntu 20.04 64bit:
pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_x86_64.whl
The tensorflow interpreter can now be used as follows:
import tflite_runtime.interpreter as tflite
interpreter = tflite.Interpreter(model_path=args.model_file)
Coral USB accelerator
First, install the coral dependencies:
sudo apt-get install curl
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
Finally, install the coral library. There are two versions available, the standard and the max version, which is faster but the USB device gets hot fast. NB: only one version can be installed in the system. Installing a different one will override the previous.
# standard:
sudo apt-get install libedgetpu1-std
# max:
sudo apt-get install libedgetpu1-max
Compile model for edge hardware
To achieve the best performance from the coral USB accelerator we need to compile our tensorflow model. To do so we need to install the edgetpu-compiler following the instruction of this link. Notice that we can not use the compiler directly on an ARM64 device such as the raspberry pi, but we need a more powerful desktop environment. Once the compiler is installed we can convert our model as follows:
edgetpu_compiler my_model.tflite
The output will be my_model_edge.tflite
.
Notice that the compiler requires a quantized tflite model as input. The quantize model can be created during training or by transforming a trained model. All the instructions can be found here. The post-training quantization procedure is the following:
import tensorflow as tf
def representative_dataset():
for data in tf.data.Dataset.from_tensor_slices((rep_images)).batch(1).take(100):
yield [data]
saved_model_dir = 'my_tensorflow_full_model.model'
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
tflite_quant_model = converter.convert()
with tf.io.gfile.GFile('quantized_model.tflite', 'wb') as f:
f.write(tflite_quant_model)
where my_tensorflow_full_model.model
is the path to the original tensorflow model and rep_images
is a list containing representative samples the model can expect as input (for example out test data).