HipChat bot on AWS

Over the last year or so I’ve spent some time testing out Amazon Web Services. While at first the myriad of options seemed a bit overwhelming, once I figured out what I wanted, it was fairly easy to navigate and manage the setup. In particular I used Elastic Compute Cloud with a Debian Wheezy image and stuck to the AWS Free Tier. This setup allowed me to run a t2.micro instance of Linux continuously (only rebooting it for necessary software updates) with the following basic specs:

  • CPU:  Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
  • RAM: A little over 512 MB
  • Architecture: x86_64

I was a little concerned while I used it that I would push the system out of the free territory, but was pleasantly surprised to not see any charges to my account. The first few months were spent getting familiar with what AWS was all about, and the next few on hosting a few NodeJS ideas.

The best test application I setup for it (and what I finished the free period with) was one based on good ol’ trusty PHP and the HipChat v2 API. As it turns out I haven’t been using an RSS reader in awhile (thanks Google) but still enjoy reading news. Instead of hitting a bunch of different pages to check and see the latest, I figured I would write a simple bot to collect the data for me and message a room in HipChat with the contents.

The project is called NewsToChat and installation was straightfoward:

  • On the HipChat profile page:
    • Create an OAuthAPI token
  • On the AWS instance:

The basic idea is that there are scripts setup to run in cron, and are mapped to the commands available in NewsToChat. These are:

  • pullnews
    • uses a few classes to pull from and format the identified news sources
    • makes a basic attempt to de-duplicate what was found
    • uses a database service to store the data
  • pushnews
    • push one article to the identified chat target
    • marks the article as expired
  • maintenance
    • perform maintenance on the pool of news articles in the database

For the little experiment I was running, I pushed the first article I gathered into the HipChat room on September 6th, 2014 and the last on December 16th, 2014. In all, a little over sixty thousand news items were gathered, providing more than enough content to publish about every five minutes.

It was extremely simplistic, but could have easily been expanded to behave more like an appropriate chat room bot. For instance, had I spent the time on it, it would have been nice for the bot to support user specific preferences and listen to commands from the chat room.

Shortly after the free period expired, I let it run for awhile to see how much it would cost me. Turns out, it was about 0.47 USD per day. Not bad. Less than the cost of a newspaper. :-)

Portable Development Environments

I have been enjoying setting up and utilizing my development environments these days. Though that may sound odd to some, it has been that way as I’ve been able to isolate them from my main system, allow them to be cross-platform, and easily reproduce them on other hosts. The most efficient way I’ve found to do this up until now is by utilizing Vagrant. While most of my work has been based on a mixture of Puppet and Bash shell script provisioning, I’m looking forward to utilizing Docker at some point in the future.

Here are some of the ones I have developed:

By no means should they be considered a complete development environment, or ready for all projects that are based on what is in the title. They should be a starting point for future work and something to build upon. Additionally they’ve been great for learning a little more about automation. Any comments, ideas, or applicable pull requests are much appreciated.

For the most part, the workflow goes like this for initially launching the development environment:

  • Clone the repository and start the virtual machine. Tthe first time will take awhile to boot and subsequent ones will be much quicker. In this example I’m showing the master branch from the vagrant-nodejs repository:
$ git clone https://github.com/kherrick/vagrant-nodejs
$ cd vagrant-nodejs/
$ bin/vm start
  • To turn off the virtual machine:
$ bin/vm stop
  • To login to the virtual machine:
$ bin/vm ssh

2048 on a Touchscreen Raspberry Pi

As I was checking out the latest accessories for the Raspberry Pi, I stumbled across this nice little shield at Adafruit: PiTFT Mini Kit – 320×240 2.8″ TFT+Touchscreen. Several days went by before I was certain I had to have it, and when it eventually arrived, I was not disappointed.


To start, I followed the excellent setup guide, which went from the soldering (hint: use thin solder and a pointed tip) and software setup all the way to instructions on the display calibration. Before, during, and after verifying that everything was working, I kept wondering about all the things I could make with this thing. What I ended up landing on was building a portable 2048 gaming system. :-)

So, I began to do some research on how exactly I was going to control 2048 itself. At first I wasn’t sure whether I wanted to use a keyboard like peripheral to simulate the arrow keys, wire up buttons for use with RPi.GPIO / WiringPi, or to look into using swipe events.

Also I reasoned, since I didn’t plan on writing the game from scratch, running the original seemed ideal as it works in a plain ol’ web browser. And if that didn’t pan out, I would resort to booting it in command line mode, as there are a variety of C, Python, and Bash ports. Obviously there are other ways to approach this, especially if I were to be developing the game for market, but I was looking at it as a minimum viable prototype project (MVPP™). j/k

For the command line forks, the simplest way seemed to be a method involving launching the application, and then sending keystrokes such as ‘w’, ‘a’, ‘s’, or ‘d’ to /dev/tty1 using a small C utility:


#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>

int main(void)
  int hTTY = open("/dev/tty1", O_WRONLY|O_NONBLOCK);
  ioctl(hTTY, TIOCSTI, "b");
  return 0;

Another idea that came up was to start a tmux session and send keystrokes through it:

tmux send-keys -t SESSIONNAME Down
tmux send-keys -t SESSIONNAME Up

However, as much as I like the command line, for this project I wanted to avoid it for the UI. I also wondered how I would control it within X? The original 2048 didn’t seem to support using a mouse, or the touch screen from Adafruit, so I looked at utilizing XAUT or xdotool so that I could script out a solution that would be triggered from a button press.

Didn’t want to do any of those either. I did arrive at this solution though:

  • auto login to Linux without a password by modifying /etc/inittab:
    • look for a line like this: 1:2345:respawn:/sbin/getty --noclear 38400 tty1
    • change it to something like this: 1:2345:respawn:/sbin/getty --autologin {USERNAME} --noclear 38400 tty1
  • auto start LXDE
  • auto start Midori in fullscreen
  • and then make sure the default home page is set to the modified 2048.

The main difference from the original is that I have the app zoomed to 90%, and I swapped out the original touch event setup with one based on Hammer.js. This was done only to work with the touch screen and the events provided by Midori when swiping in various directions on the game board.

The only other functionality provided in the project is a listener script running in the background that is started from /etc/rc.local on bootup:


#!/usr/bin/env bash

#wiring pi pin 4 maps to gpio-23
/usr/local/bin/gpio mode 4 up

while true; do
  sleep .5
  if [ `/usr/local/bin/gpio read 4` -lt 1 ]; then
    sudo poweroff

This is to allow for a poweroff event to be sent to the Raspberry Pi, so that when I am done playing the game, I can get the unit to turn off without using a keyboard. Then, I power off the battery pack using a stylus on the underside of the unit.

Parts used:

Watch a video of it booting, a little game play, and the shutdown sequence here:

Checking out Bolt

Recently I was able to spend some time with Bolt, “a tool for Content Management, which strives to be as simple and straightforward as possible.”

Actually, It was very straightforward, as I only had to:

  • Clone the project from GitHub
  • Get Composer and install
  • Tweak my existing Nginx configuration
  • Export the entries from this installation of WordPress into XML
  • Enable the ImportWXR extension, and import the entries into the new Bolt database
  • And finally, tweak a few configs and Twig templates

Below you can see the entries and menu items from this blog (screenshots are from Firefox Mobile):

Tracking with Tasker

After finding Tasker for Android and realizing all of the potential it has, I had to try it out. To begin, I decided to build a prototype backend service for receiving the location signals sent from the app itself. The possibilities seemed endless. As the website states:

Tasker is an application for Android which performs tasks (sets of actions) based on contexts (application, time, date, location, event, gesture) in user-defined profiles or in clickable or timer home screen widgets.

What I came up with isn’t nearly as robust as something like Android Device Manager or services like Google+ location sharing, but it did allow me to think through the ins and outs of implementing products like these. The project requires:

  • Tasker
  • A web server that can run PHP 5.4 code
  • A terminal that can run PHP CLI 5.4 code

The sending portion involves thoroughly understanding the location intricacies of Tasker, and a basic idea of how it works. I configured it as shown in the following images:

The receiving portion is setup by checking out and building the git repository, as well as configuring hosting for public/index.php.

 git clone https://github.com/kherrick/tracking
 cd tracking/

If everything is in place properly when Tasker posts to the service, the data can be stored in an sqlite database or a simple log file.


The final piece is to generate the output from the data stored. When tracker.php is executed on the command line, the help page is displayed showing the available arguments.

$ ./tracker.php 
Tracker version .001

  [options] command [arguments]

  --help           -h Display this help message.
  --quiet          -q Do not output any message.
  --verbose        -v|vv|vvv Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
  --version        -V Display this application version.
  --ansi              Force ANSI output.
  --no-ansi           Disable ANSI output.
  --no-interaction -n Do not ask any interactive question.

Available commands:
  help   Displays help for a command
  list   Lists commands
  map    Generate a Google static map URL.

In this version, the main command “map” parses through the included demo log file and returns a URL for a static map from Google, with location points marked.

$ ./tracker.php map -l logs/YYYY-MM-DD_post_capture_DOT_log


The Bermuda Triangle
The Bermuda Triangle

Controlling a Lego motor with the Raspberry Pi

Raspberry Pi and Gertboard controlling a 4.5v Lego motor #6216mWhen the Gertboard originally came out, it was a do it yourself kit that required soldering, and it had enough pieces that made me want to wait until one was offered pre-assembled (you can still get one like the original from Tandy called the Multiface Kit). After purchasing the latest version, I tried putting one of the Lego motors we have around the house through its paces. Using one of the older models (4.5v Lego motor #6216m) and learning from the examples in the Python Gertboard Suite made crafting my own setup surprisingly easy.

The main steps were to code a motor controlling script in Python (see lego-motor-control.py), and demo it in Bash (see lego-motor.sh). I find tinkering around with these components immensely rewarding while exploring topics like the Internet of Things. This type of exploration will only get easier as time goes on, for example, I just noticed today that Gert van Loo created something called the Gertduino. It is similar to the original product, but much smaller and easier to use. What kind of fun projects could be implemented with even smaller kit like TinyDuino or Intel Edison? I do like sticking with the Pi for now, so I dug up a list of compatible shields for your perusal (or browse an even larger list of expansion boards @ elinux.org):

Also, check out the video to get an idea on how the Lego motor and scripts ended up working out:


echo -e "Turning the lego motor right for five seconds...\n"
sudo python ~/lego-motor-control.py right &
sleep 5
sudo kill -INT $pid

echo -e "\nSleeping for three seconds to let the motor wind down\n"
sleep 3

echo -e "Turning the lego motor left for five seconds...\n"
sudo python ~/lego-motor-control.py left &
sleep 5
sudo kill -INT $pid


#!/usr/bin/env python
import RPi.GPIO as GPIO
import collections, signal, sys
from time import sleep

# get command line agruments
arg_names = ['command', 'direction']
args      = dict(zip(arg_names, sys.argv))
arg_list  = collections.namedtuple('arg_list', arg_names)
args      = arg_list(*(args.get(arg, None) for arg in arg_names))

# set initial values
step      = 1
mota      = 18
motb      = 17
left      = motb
right     = mota
reps      = 400
hertz     = 2000
freq      = (1 / float(hertz)) - 0.0003
ports     = [mota,motb]
percent   = 100

# function to run the motor
def run_motor(reps, pulse_width, port_num, period):
    for i in range(0, reps):
        GPIO.output(port_num, True)
        GPIO.output(port_num, False)

# trap SIGINT and provide a clean exit path
def signal_handler(signal, frame):
    GPIO.output(direction, False)

signal.signal(signal.SIGINT, signal_handler)


# initialize the ports being used
for port_num in ports:
    GPIO.setup(port_num, GPIO.OUT)
    print "setting up GPIO port:", port_num
    GPIO.output(port_num, False)

# determine direction or set a default
if args[1] == "left":
    direction = left
elif args[1] == "right":
    direction = right
    direction = right

# the main loop
while True:
    pulse_width = percent / float(100) * freq
    period      = freq - (freq * percent / float(100))
    run_motor(reps, pulse_width, direction, period)

Lego Car & Raspberry Pi

Lego Car with Raspberry Pi

For awhile I’ve wanted to connect an RC car to the Raspberry Pi by wiring up to a Tyco Fast Traxx remote control. There are even products like Pi-Cars now, and entire blogs devoted to the construction of autonomous cars. I decided to build out a similar concept with the help of my kids (read about the last time we posted on lego cars).

Recently we acquired the Lego 4×4 Crawler, which includes:

After they completed the Lego build instructions, they added:

This provided an opportunity for me to control the 4×4 Crawler’s infrared remote with a Parallax servo and Lego gear attached to the Raspberry Pi.

The Parallax Standard Servo was attached with Lego bricks built around the outside of it and a larger Lego gear wired on top of it. I then built a platform in the bed of the 4×4 for space to hold this custom part, as well as the connecting gear, and Lego Infrared Remote Control.

To distribute the weight a little bit, the USB battery was enclosed on the underside of the vehicle. As you can see in the video at the end of the post, it still runs lopsided at times. When turning it on, the only thing it directly powers is the USB hub on top. The USB hub then powers the Raspberry Pi, servo, and other peripherals.

Parts used:

Although the servo in the project was wired to the USB hub, the illustration below shows how the USB battery can power the servo, as well as how the control wire is able to be connected up to BCM_GPIO pin 18 for testing.

Servo driven from battery and Raspberry Pi

example wiring showing how to run the servo from external battery

For issuing the commands remotely, I had in mind to hook up a Bluetooth USB adapter and pair the Bluetooth game controller directly to the Pi, but in the interest of simplicity, I paired it with my Nexus 4 instead and ssh’ed into the Pi using Terminal IDE.

Using a wireless keyboard I launched the listener script for the servo and was able to control it using the arrow keys. The Bluetooth game controller worked flawlessly as the buttons mapped to standard keyboard codes I specifically looked out for in the script.


ENTER=$(printf "%b" "\n")
ESCAPE=$(printf "%b" "\e")

while true; do
  read -sn1 a
  case "$a" in
    $ENTER) echo "ENTER";;
      read -sn1 b
      test "$b" == "[" || continue
      read -sn1 b
      case "$b" in
          echo "Turning Right"
          echo "Straigtening"
          echo "Turning Left"
        *) continue;;
      echo "Turning Right"
      echo "Straigtening"
      echo "Turning Left"

      echo "Taking a picture"
      ~/picture.sh &
      echo "Taking a video"
      ~/video.sh &
    *) echo "$a";;

When the various buttons or keys are pressed, the listener script calls out to the other scripts, which provide for turning, picture, and video taking.


dateTime=`date +%Y-%m-%d-%H-%M-%S`

raspistill -w 640 -h 480 -e jpg -t 0 -o $pictureDirectory/$dateTime.jpg


dateTime=`date +%Y-%m-%d-%H-%M-%S`

echo "Taking a video for 10 seconds"
raspivid -t 10000  -o $videoDirectory/$dateTime.h264


sudo ~/control.py 1100 .25


sudo ~/control.py 1300 .25


sudo ~/control.py 1550 .25

The turning scripts all make use of RPIO.PWM which I was able to use after installing RPIO and building off of the provided examples.


import time
import sys
from RPIO import PWM

servo = PWM.Servo()

#appropriate range of values for BCM_GPIO pin 18 and the servo: 370 - 2330

print servo.set_servo(18, float(sys.argv[1]))



Though the Raspberry Pi camera module is rated
5 megapixels with a native resolution of 2592×1944 I have it statically set to 640×480 in the picture taking script above. With it, I was able to capture a picture of my cat ignoring everything but the treat we were bribing her with.

First cat photo from Raspberry Pi camera module

And another one, when she had realized something was going on.

Second cat photo from Raspberry Pi camera module

Lastly, I put together a short video of the Lego 4×4 Crawler being driven from the outside as well as a first person perspective clip from the Raspberry Pi camera.

PHP one-liners on the CLI

PHP Code

After checking out a page called Perl one-liners and reading how you could execute Perl inline on the CLI, I decided to try and replicate some grep functionality using PHP one-liners for fun.

Interestingly, PHP seems quite slow compared to other tools for my particular task, so I probably won’t be using these approaches outlined below in daily practice. :-)

Here are some stats for a search through all files in the current directory for words with ‘foo’ in them:

time to search using PHP:

username@host:~$ time php -r '$s="foo";$fs=scandir(".");foreach($fs as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo "$f:$l";}}'
filename.txt:football season is approaching

real	0m0.013s
user	0m0.012s
sys	0m0.000s

time to search using Perl:

username@host:~$ time perl -ne 'print "$ARGV:$_" if /foo/' *
filename.txt:football season is approaching

real	0m0.003s
user	0m0.000s
sys	0m0.000s

time to search using grep:

username@host:~$ time grep -H foo *
filename.txt:football season is approaching

real	0m0.002s
user	0m0.000s
sys	0m0.000s

As you can see, to use the one liner technique it appears to be as simple as “compressing” your scripts down to a single line, making sure they are shell friendly, and executing them with the “-r” option. Some other ones that I worked out with PHP are below:

to search for ‘foo’ and print matching lines in a single file:

php -r '$s="foo";$ls=file("filename");foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}'

to search for ‘foo’ and print matching lines in all files in the current directory:

php -r '$s="foo";$fs=scandir(".");foreach($fs as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}}'

to search for ‘foo’ and print matching lines recursively from the current directory:

php -r '$s="foo";$t=new RecursiveDirectoryIterator(".");foreach(new RecursiveIteratorIterator($t) as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}}'

Observations on HTML

In December of 2012 HTML5 became HTML5 Logofeature complete according to the W3C, “meaning businesses and developers have a stable target for implementation and planning.” They continued to describe HTML5 as, “the cornerstone of the Open Web Platform, a full programming environment for cross-platform applications with access to device capabilities; video and animations; graphics; style, typography, and other tools for digital publishing; extensive network capabilities; and more.”

What a long road. Think back to the turn of the century. Shortly after HTML 4.01 was published as a W3C recommendation and XHTML 1.0 had it’s turn, it was said to be “developed to make HTML more extensible and increase interoperability with other data formats.” I remember the routine. Best practices at the time were separating form from content, moving to CSS, utilizing unobtrusive JavaScript, and practicing graceful degradation. Don’t forget to close your tags. Make it XML. Oh, and the machines are coming! :-)

As XHTML 2 approached it became clear that it would be an entirely new way of doing things, and not just an incremental approach that preserved compatibility. In 2004, Mozilla and Opera published the “Position Paper for the W3C Workshop on Web Applications and Compound Documents.” Some key sections included headings like, “Backwards compatibility, clear migration path”, “Users should not be exposed to authoring errors”, and “Scripting is here to stay.” Ultimately the initiatives were voted down and in response the WHATWG was formed. Between the years of 2007 and 2009, not only did the W3C accept WHATWG’s “Proposal to Adopt HTML5“, they allowed the XHTML 2 Working Group’s charter to expire, even going on to acknowledge that HTML5 would be the standard rather than the XML variant XHTML5. Regarding the two formats they wrote, “The first such concrete syntax is the HTML syntax. This is the format suggested for most authors.”

Since then, the whole web has been marching toward HTML5 domination, steadily learning best practice and implementation. In the earlier days I recall it not being as rapid as more recent, with people discussing the semantics, along with calls to prepare, but there has been a ton of solid information on the topic for awhile now and the momentum has shifted. Not only has the WHATWG decided that HTML is a living standard while the W3C publishes regular snapshots, the working draft of HTML 5.1 has been issued.

Lastly, I find it interesting to see the various web development strategies work themselves out as the craft changes. Graceful degradation (and desktop centric) has steadily given way to a solid progressive enhancement (and mobile first) approach as the web continues to gain in mobile traffic. In addition there are quite a few ideas going around on how to best accommodate all of the client browsers, especially in the comments. Should one start with an adaptive web design, and how is that related to responsive web design? Is one really a part of the other and should we have a strategy utilizing both? Maybe that’s the future… I guess it depends.

Time flies when you are having fun. Certainly I can be sure about one thing, I still like to close my tags. :-)

HTML5 Logo by W3C.

Compiling TrueCrypt on Raspberry Pi

These steps were gleaned from the work completed by Reinhard Seiler.

Other than the manual download and placement of the TrueCrypt source, the rest should be fairly hands off. So first, get the “TrueCrypt 7.1a Source.tar.gz” package from http://www.truecrypt.org/downloads2 and copy it to /usr/local/src/ on the Raspberry Pi.

Secondly, run the commands below, oh, and at your own risk of course… :-)

#get source files other than the TrueCrypt source
sudo wget -P /usr/local/src http://prdownloads.sourceforge.net/wxwindows/wxWidgets-2.8.11.tar.gz
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11.h
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11f.h
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11t.h

#get and install dependent packages
sudo apt-get -y install libgtk2.0-dev libfuse-dev nasm libwxgtk2.8-dev

#extract, configure, and make wxWidgets
sudo tar -xzvf /usr/local/src/wxWidgets-2.8.11.tar.gz -C /usr/local/src
cd /usr/local/src/wxWidgets-2.8.11/

#setup, extract
export PKCS11_INC=/usr/local/src/pkcs11
sudo tar -xzvf /usr/local/src/TrueCrypt\ 7.1a\ Source.tar.gz -C /usr/local/src
cd /usr/local/src/truecrypt-7.1a-source

#comment out some lines that prevented building
sed -i 's#TC_TOKEN_ERR (CKR_NEW_PIN_MODE)#/*TC_TOKEN_ERR (CKR_NEW_PIN_MODE)*/#g' Common/SecurityToken.cpp 
sed -i 's#TC_TOKEN_ERR (CKR_NEXT_OTP)#/*TC_TOKEN_ERR (CKR_NEXT_OTP)*/#g' Common/SecurityToken.cpp

#compile, build, make!
sudo make WX_ROOT=/usr/local/src/wxWidgets-2.8.12/ wxbuild
sudo -E make WXSTATIC=1

echo TrueCrypt should be found in /usr/local/src/truecrypt-7.1a-source/Main/

Finally, copy the compiled ‘truecrypt’ binary from /usr/local/src/truecrypt-7.1a-source/Main/ to /usr/local/bin/.

Afterwards, I was able to run it from the command line and view the help. There are two items in particular that may be of interest for those going through these steps. One is, when attempting to mount an encrypted volume with the Raspberry Pi that I have, I received the following:

Error: device-mapper: reload ioctl on truecrypt1_1 failed: No such 
file or directory
Command failed

It appears as though it may be due to a particular kernel module not being compiled in. So I added the “-m=nokernelcrypto” command line option and was successful.

TrueCrypt on the Raspbeery Pi

For more command line usage, see their website @ http://www.truecrypt.org/docs/?s=command-line-usage. I haven’t tested it yet, but it should work graphically as well.

Also, though some may not want to edit the source directly, I ended up commenting out lines 660, 661, and 662 of /usr/local/src/truecrypt-7.1a-source/Common/SecurityToken.cpp in order for the compile to work (see the sed lines in the script I provided above). Here’s the difference between the original and the changed source:

diff ./SecurityToken.cpp \

While there are some compromises in the process described above, it was the only way I could get it compiled in the time I allotted to the task.