< Billy Overton >

Smart Meter Dashboard: Collecting the Data

Previous Post: Smart Meter Dashboard: Installing and Configuring InfluxDb

Once I had a place to store my power usage data, I needed to start actually collecting it. I already had my radio and software picked out during my initial testing, but now I needed a more permanent setup. With the idea being to lower my power usage, I couldn’t just leave my desktop running all the time. On my first attempt I tried an older Raspberry PI Model B I had laying around, but it couldn’t keep up with the processor load. While it would catch a few entries, for the most part it would drop the data.

A perfect excuse to get a new PI. I went with Raspberry PI 3, although I expect this will work with a Model 2 B.

Installing RTL-SDR

I installed the requirements for RTL-SDR based on Adafruit’s SDR scanner tutorial. I only needed the requirements for rtl-sdr since I’m not using a python library to interact with the radio directly.

sudo apt-get update
sudo apt-get install cmake build-essential libusb-1.0-0-dev git

cd ~
git clone git://git.osmocom.org/rtl-sdr.git
cd rtl-sdr
mkdir build
cd build
cmake ../ -DINSTALL_UDEV_RULES=ON -DDETACH_KERNEL_DRIVER=ON
make
sudo make install
sudo ldconfig

Setting rtl-tcp To Start At Boot

Rtlamr uses rtl-tcp to control and read data from the radio. For testing purposes you can start it on its own terminal by running sudo rtl-tcp, but I wanted something that would start automatically on boot. To do this, I created a systemd unit file at /etc/systemd/system/rtl-tcp.service that will always restart the service if it fails.

[Unit]
Description=Software Defined Radio TCP Server
Wants=network.target
After=network.target

[Service]
ExecStart=/usr/local/bin/rtl_tcp
Restart=always

[Install]
WantedBy=multi-user.target

Once that is created I set it to start on boot with sudo systemctl enable rtl-tcp.service.

Installing Go and Rtlamr

The version of Go in the Rasbian repository was high enough to allow rtlamr to run, so I used it instead of compiling or installing a later ARM Go version: sudo apt-get install golang

Once that was installed, I went through the standard Go setup procedure to create my Go path folder and adding the Go environment variables to my .profile file.

...
GOPATH=$HOME/go
PATH="$HOME/go/bin:$PATH"
...

You can reload your .profile by running . ~/.profile. With that done, the installation of Rtlamr was as simple as go get github.com/bemasher/rtlamr

For simplicity, I copied the binary to a dedicated location as root.

sudo mkdir /opt/powermon
sudo cp /home/pi/go/bin/rtlamr /opt/powermon

Testing

To make sure everything was working I ran rtlamr at the console to make sure it was receiving data. There are quite a few apartments near mine using the same meter type, so it didn’t take long to see a large number of results coming through.

$ sudo systemctl start rtl_tcp.service
$ /opt/powermon/rtlamr
05:10:44.703399 decode.go:79: CenterFreq: 912600155
05:10:44.703907 decode.go:79: SampleRate: 2359296
05:10:44.704002 decode.go:79: DataRate: 32768
05:10:44.704083 decode.go:79: SymbolLength: 72
05:10:44.704161 decode.go:79: PreambleSymbols: 21
05:10:44.704242 decode.go:79: PreambleLength: 3024
05:10:44.704322 decode.go:79: PacketSymbols: 96
05:10:44.704403 decode.go:79: PacketLength: 13824
05:10:44.704481 decode.go:79: Preamble: 111110010101001100000
05:10:44.704564 recv.go:93: GainCount: 29
<DATA>

You can filter the results to only include your meter using the --filterid switch. My meter had the ID printed on it, but it wouldn’t have been hard to compare my current meter reading to all of the results to find the ID.

Influx Python Upload Script

The final step of the collection was to send the data to InfluxDB. I decided to go with a simple Python script to parse then forward that information on to the database. Rtlmar is capable of outputting its data in JSON format which made reading the data simple line by line. Python also has a nice Influx client library (that I was already planning to use on the API side of things), however the version that is in the Rasbian package is out of date and does not work with the latest Influx version. The one in pip works just fine.

$ sudo apt-get install python-pip
$ sudo pip install influxdb # Do not use the python-influxdb package

The script itself is pretty simple. It starts rtlamr with a filter for my meter’s ID and a JSON output format. The output is parsed line by line, converted into a format for Influx, and then sent onward. Below is the current state of the script (with placeholders for my db info and my meter ID).

#!/usr/bin/env python
import sys
import json
import subprocess
import time
from influxdb import InfluxDBClient


# Delay. Useful for now while I try and figure out systemd targets. I've had
# issues where if both powermon and rtlamr start immediately at boot, no data is
# reported. The delay solves the problem but I don't like it.
time.sleep(2*60)

client = InfluxDBClient('{influx host}', 8086, '{username}', '{password}', '{database}', True)

proc = subprocess.Popen(['/opt/powermon/rtlamr', '-filterid={meter ID}', '-quiet=true', '-format=json'],stdout=subprocess.PIPE)

while True:
    line = proc.stdout.readline()
    if not line:
        break

    data=json.loads(line)
    message = [{"measurement":"consumption", "time": data['Time'], "tags": {"meter": data['Message']['ID'] }, "fields": {"value": data['Message']['Consumption']}}]
    client.write_points(message)

Just like rtl_tcp, I created a systemd unit file to start powermon at boot.

[Unit]
Description=Python PowerMon Influx Client
Wants=network-online.target rtl_tcp.service
After=network-online.target rtl_tcp.service

[Service]
ExecStart=/opt/powermon/powermon.py
Restart=always

[Install]
WantedBy=multi-user.target

I then enabled it sudo systemctl enable powermon.service

Problems

For now the biggest problem I’m having is the clock on the Raspberry Pi. Any time I lose power (a very infrequent event), the clock gets out of sync. I’ve tried some of the recommended fixes, but so far nothing has seemed to work. I don’t lose any data when this occurs, but the it does throw off the data.

I’m also not a fan of having the 2 minute delay in the script, but that’s a relatively minor problem for now.