Pool Pi Guy - AI Driven Alarm System and Pool Monitoring Using Raspberry Pi

In this instructable we will show how to:

1. Setup a Raspberry Pi with OpenCV and TensorFlow
2. Connect a Webcam through a long USB cable
3. Write an OpenCV algorithm to detect motion
4. Use TensorFlow for object detection
5. Set up a web server on the Raspberry Pi to show the interesting images
6. Integrate with IFTTT to trigger mobile alerts in case a person is detected
7. Attach a relay HAT to the Raspberry Pi and connect it to a solenoid valve that would add water to the pool
8. Attach a water level sensor to the Raspberry Pi and interface with it using the Pi's GPIO
9. Write some code to glue it all together

All the components are readily available from Amazon. Feel free to experiment and exchange components - that's half the fun!

1. Raspberry Pi
2. Raspberry Pi Power Supply (don't skimp here)
3. Memory Card (bigger is better)
4. Case (this one is large enough to house both the Pi and the HAT)
5. USB Webcam (any webcam will do, but you want one that gets good images and balances lighting well)
6. USB extension cable (if needed - measure the distance between the Pi and where you would place the camera)
7. Relay board HAT (this one has 3 relays and we only need one, but you will find a use for the other ones soon enough!)
8. Solenoid
9. Solenoid fitting 1 and Fitting 2 (that really depends on what you fit the solenoid to, but these worked for me)
10. Solenoid Power Supply (any 24V AC would do)
11. Cable (again, almost any 2 strand cable would do - the current is minimal)
12. Water Level Float Switch (this is just an example, check what can be easily connected to your pool)
13. Some Jumper wires and Wire connectors

Raspberry Pi is a great little computer. It costs just $35, runs consistently, and has lots of compatible software and hardware. Setting it up is quite easy:

1. Format your SD card. This needs special care - Raspberry Pi can only boot from a FAT formatted SD card. Follow these instructions.
2. Connect the Raspberry Pi to a USB keyboard and mouse, plus an HDMI display, and follow the instructions in the Raspberry Pi NOOBS tutorial. Make sure to setup WiFi and enable SSH access. Don't forget to setup a password for the default pi account.
3. On your home network setup a static IP for the Raspberry Pi - it would make it much easier to SSH into.
4. Make sure you have an ssh client installed on your desktop/laptop. For a PC I would recommend Putty, which you can install from here.
5. Unhook the USB and HDMI from the Raspberry Pi, reboot it, and ssh into it - if it all worked you should see something like this:

Linux raspberrypi 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law. 
Last login: Mon May 13 10:41:40 2019 from 104.36.248.13 
pi@raspberrypi:~ $

OpenCV is an amazing collection of image manipulation functions for computer vision. It will allow us to read images from the Webcam, manipulate them to find areas of motion, save them and more. Setup on the Raspberry Pi is not difficult but requires some care.

Start by installing virtaulenvwrapper: we will use python to do all of our programming, and virtualenv would help us keep dependencies separate for OpenCV and TensorFlow vs. Flask or GPIO:

pi@raspberrypi:~ $ sudo pip install virtualenvwrapper

Now you can execute "mkvirtualenv" to create a new environment, "workon" to work on it, and more.

So, let's create an environment for our image manipulation, with python 3 as the default interpreter (it's 2019, there is no reason to stick with the older python 2):

pi@raspberrypi:~ $ mkvirtualenv cv -p python3

...

(cv) pi@raspberrypi:~

We are now ready to install OpenCV. We will mostly follow the excellent tutorial in Learn OpenCV. Specifically follow their step 1 and 2:

sudo apt -y update<br>sudo apt -y upgrade
## Install dependencies
sudo apt-get -y install build-essential checkinstall cmake pkg-config yasm
sudo apt-get -y install git gfortran
sudo apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev
sudo apt-get -y install libtiff5-dev
sudo apt-get -y install libtiff-dev
sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev
sudo apt-get -y install libxine2-dev libv4l-dev
cd /usr/include/linux
sudo ln -s -f ../libv4l1-videodev.h videodev.h
sudo apt-get -y install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
sudo apt-get -y install libgtk2.0-dev libtbb-dev qt5-default
sudo apt-get -y install libatlas-base-dev
sudo apt-get -y install libmp3lame-dev libtheora-dev
sudo apt-get -y install libvorbis-dev libxvidcore-dev libx264-dev
sudo apt-get -y install libopencore-amrnb-dev libopencore-amrwb-dev
sudo apt-get -y install libavresample-dev
sudo apt-get -y install x264 v4l-utils
sudo apt-get -y install libprotobuf-dev protobuf-compiler
sudo apt-get -y install libgoogle-glog-dev libgflags-dev
sudo apt-get -y install libgphoto2-dev libeigen3-dev libhdf5-dev doxygen
sudo apt-get install libqtgui4
sudo apt-get install libqt4-test

Now we can just install OpenCV with python bindings inside the cv virtualenv (you are still in it, right?) using

pip install opencv-contrib-python

And that's it! We have OpenCV installed on our Raspberry Pi, ready to capture photos and videos, manipulate them and be cool.

Check that by opening a python interpreter and importing opencv and check that there are no errors:

(cv) pi@raspberrypi:~ $ python

Python 3.5.3 (default, Sep 27 2018, 17:25:39) 

[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>>

TensorFlow is a machine learning / AI framework developed and maintained by Google. It has extensive support for deep-learning models for a variety of tasks including object detection in images, and is now fairly simple to install on Raspberry Pi. The performance of its lightweight models on the tiny Pi is around 1 frame per second, which is perfectly adequate for an application like ours.

We will basically follow the excellent tutorial by Edje Electronics, with modifications made possible by more recent TensorFlow distributions:

pi@raspberrypi:~ $ workon cv
(cv) pi@raspberrypi:~ $ pip install tensorflow
(cv) pi@raspberrypi:~ $ sudo apt-get install libxml2-dev libxslt-dev
(cv) pi@raspberrypi:~ $ pip install pillow lxml jupyter matplotlib cython
(cv) pi@raspberrypi:~ $ sudo apt-get install python-tk

Now we need to compile Google's protobuf. Just follow the instructions in step 4 of the same excellent tutorial

Finally, clone and setup TensorFlow's model definitions - follow step 5 in Edje Electronics tutorial

Feel free to follow their example in step 6 as well, it is a great introduction to object detection on the Raspberry Pi.

Let's start by testing that OpenCV can interface with our webcam: ssh into the Raspberry Pi, move to the cv virtualenv (workon cv), open a python interpreter (just type python), and enter the following python commands:

import cv2
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
ret, frame = cap.read()
print('Read frame size: {}x{}'.format(frame.shape[1], frame.shape[0])

With any luck you will see that OpenCV was able to read an HD frame from the camera.

You can use cv2.imwrite(path, frame) to write that frame to disk and sftp it back to take an actual look.

The strategy to detect motion is fairly straight forward:

1. Work on lower resolution frames - there is no need to operate on full HD here
2. Further, blur the images to ensure as little noise as possible.
3. Keep a running average of the last N frames. For this application, where the frame rate is around 1 FPS (just because TensorFlow takes some time per frame), I found that N=60 returns good results. And since a careful implementation does not take more CPU with more frames that is OK (it does take more memory - but that is negligible when we work with the lower resolution frames)
4. Subtract the current image from the running average (just be careful with typing - you need to allow for positive and negative values [-255 .. 255], so the frame needs to be converted to int)
5. You can perform the subtraction on a gray-scale conversion of the frame (and the average), or do it separately for each of the RGB channels and then combine the results (which is the strategy I chose, making it sensitive to color changes)
6. Use a threshold on the delta and remove noise by erosion and dilation
7. Finally look for contours of areas with a delta - these areas are where motion has happened and the current image is different from the average of previous images. We can further find bounding boxes for these contours if needed.

I have encapsulated the code to do this in the DeltaFinder python class that you can find in my github here

If you have followed the TensorFlow installation procedure, you have already tested that you have TensorFlow installed and working.

For the purpose of detecting people in a general outdoor scene, models that are pre-trained on the COCO data set perform quite well - which is exactly the model we have downloaded at the end of the TensorFlow installation. We just need to use it for inference!

Again, I have encapsulated the model loading and inference in the TFClassify python class to make things easier, which you can find here.

The easiest way to access the object detection results is a web browser, so let's set up a web server on the Raspberry Pi. We can then set it up to serve up pictures from a given directory.

There are multiple options for a web server framework. I chose Flask. It is extremely configurable and easy to extend with Python. Since the "scale" we need is trivial, it was more than enough.

I suggest installing it in a new virtualenv, so:

pi@raspberrypi:~ $ mkvirtualenv webserv
(webserv)pi@raspberrypi: ~ $ pip install Flask

Note that with a normal network setup it will only be reachable when your browser is on the same wireless LAN as your Raspberry Pi. You could create a port mapping / NAT configuration on your Internet router to allow external access - but I recommend against that. The code I wrote does not attempt to provide the security you would need when allowing general Internet access to your Raspberry Pi.

Test your installation by following the Flask quick start guide

I really want to get mobile notifications when events occur. In this case, when a person is detected and when the water level goes low. The simplest way I found to do that, without having to write a custom mobile app, is using IFTTT. IFTTT stands for "If This Then That" and enables many types of events to trigger many types of actions. In our case, we are interested in the IFTTT Maker Webhook trigger. This allows us to trigger an IFTTT action by making an HTTP POST request to the IFTTT server with a special key assigned to our account, along with data that specifies what happened. The action we take can be as simple as creating a notification on our mobile device using the IFTTT mobile app, or anything more complex than that.

Here is how to do that:

1. Create an IFTTT account on ifttt.com
2. While logged in, go to the Webhook service settings page and enter the URL in your browser (something like https://maker.ifttt.com/use/ . That web page will show you your key and the URL to use to trigger actions.
3. Create an IFTTT applet that will generate a mobile notification when the Webhook is triggered with the details of the event:
   1. Click "My Applets" and then "New Applet".
   2. Click "+this" and choose "webhooks". Click "Receive a web request" to go on to the details
   3. Give your event a name, e.g. "PoolEvent" and click "Create trigger"
   4. Click "+that" and choose "notifications". Then choose "Send a rich notification from the IFTTT app"
   5. For "title" choose something like "PoolPi"
   6. For "message" write "Pool Pi detected: " and click "add ingredient".."Value1".
   7. Go back to the URL you copied in step 2. It will show the URL to use to invoke your newly created applet. Copy that URL, replacing the placeholder {event} with the event name (in out example PoolEvent)
4. Download, install and login to the IFTTT app for your mobile device
5. Run this python script on your Raspberry Pi to see it working (note it might take a few seconds or minutes to trigger on your mobile device):

import requests
requests.post('https://maker.ifttt.com/trigger/PoolEvent/with/key/<YOUR KEY>', json={"value1":"Hello Notifications"})

Before proceeding with this step TURN OFF your Raspberry Pi: ssh to it and type "sudo shutdown now", then disconnect it from power

Our goal is to turn on and off the power supply to a solenoid valve - a valve that can open or close the water supply based on 24V AC power it gets from a power supply. Relays are the electrical components that can open or close a circuit based on a digital signal that our Raspberry Pi can provide. What we do here is hook up a relay to these digital signal pins of the Raspberry Pi, and have it close the circuit between the 24V AC power supply and the solenoid valve.

The pins on the Raspberry Pi that can act as digital input or output are called GPIO - General Purpose Input/Output and they are the row of 40 pins on the side of the Pi. With the Pi turned off and insert the relay HAT firmly into it. The HAT I chose has 3 relays in it, and we will use just one of them. Imagine all you can do with the other two :)

Now turn the Raspberry Pi back on. The red "power" LED on the relay HAT should turn on, indicating it is getting power from the Pi through the GPIO. Let's test that we can control it: ssh into the Pi again, enter python and type:

import gpiozero
dev = gpiozero.DigitalOutputDevice(26, initial_value = True)
dev.off()

You should hear an audible "click", indicating that the relay is engaged, and see a LED turn on showing that the first relay is in the connected position. You can now type

dev.on()

Which would turn the relay to the "off" position (odd, I know...) and exit() from python.

Now using jumper cables and the longer cable connect the relay between the 24V power supply and the solenoid valve. See the diagram. Finally, connect the solenoid valve to a faucet using the adapters and get ready to test it all by repeating the commands above - they should turn the water on and off.

Attach a hose to the solenoid valve and put the other end deep in the pool. You now have a computer-controlled pool top-off system, and it's time to connect a sensor to tell it when to run.

A water level sensor is simply a float that connects an electrical circuit when the float is down, and breaks it when it floats up. If you insert it in the pool at the right height the float will be up when the water level is adequate but fall down when there aren't enough water.

For the Raspberry Pi to know the status the water level sensor we need the Pi to sense an open or closed circuit. Luckily that is very simple: the same GPIO connectors that we use as digital output to control the relays can act as inputs (hence the I in GPIO). Specifically, if we connect one wire of the sensor to +3.3V on the GPIO connector and the other sensor wire to a pin that we configure as pull-down input (meaning it will be normally at GND voltage level), that pin will measure a digital "high" or "on" voltage only when the water level sensor closes the circuit - when the water level is low. I used GPIO pin 16 as input, which I marked in the image above.

The python code to configure the pin as input and test its current state is:

import gpiozero

level_input = gpiozero.Button(16)
water_low = level_input.is_pressed

One potential challenge is that when the sensor just changes state it would oscillate fast between on and off states. The solution to that is known as "debouncing" and looks for a consistent state change before taking action. The GPIOZERO library has code to do that, but for some reason that code did not work well for me. I wrote a simple loop to trigger IFTTT alerts when a consistent state change is detected, which you can find in my repository here.

That's it. Our setup is complete. You can write your own code to tie things together into a full system, or use the code I provide. To do that just create the directory structure and clone the repository, like so:

mkdir poolpi
cd poolpi 
git clone https://github.com/rafitzadik/PoolPiGuy.git

Next, edit the files named ifttt_url.txt in the motion_alert and water_level directories to have the URL for your own IFTTT web hook with your secret key. You could use two different web hooks for different actions.

Finally, we want this code to run automatically. The easiest way to accomplish that is through the Linux crontab service. We can add some crontab lines for two main tasks:

1. Run our three programs: the object detector, the water level sensor and web server on every reboot
2. Cleanup the output directory, deleting old pictures and old video files (I chose to delete files older than 1 day and pictures older than 7 days - feel free to experiment)

To do that type crontab -e which will open up your nano text editor. Add the following lines to the bottom of the file:

0 1 * * * find /home/pi/poolpi/output -type f -name "*.avi" -mtime +1 -delete
0 2 * * * find /home/pi/poolpi/output -type f -name "*.jpg" -mtime +7 -delete
@reboot python3 /home/pi/poolpi/motion_alert/webserv/webserv.py
@reboot python3 /home/pi/poolpi/motion_alert/motion_obj_alert.py
@reboot python3 /home/pi/poolpi/water_level/test_water_level.py

Finally, reboot your Raspberry Pi. It is now ready to keep your pool full and safe.

Do tinker with the setup, the code, and don't forget to star my github repository and comment on the instructable if you find it useful. I am always looking to learn more.

Happy making!