PHP one-liners on the CLI

PHP Code

After checking out a page called Perl one-liners and reading how you could execute Perl inline on the CLI, I decided to try and replicate some grep functionality using PHP one-liners for fun.

Interestingly, PHP seems quite slow compared to other tools for my particular task, so I probably won’t be using these approaches outlined below in daily practice. :-)

Here are some stats for a search through all files in the current directory for words with ‘foo’ in them:

time to search using PHP:

username@host:~$ time php -r '$s="foo";$fs=scandir(".");foreach($fs as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo "$f:$l";}}'
filename.txt:football season is approaching
 
real	0m0.013s
user	0m0.012s
sys	0m0.000s

time to search using Perl:

username@host:~$ time perl -ne 'print "$ARGV:$_" if /foo/' *
filename.txt:football season is approaching
 
real	0m0.003s
user	0m0.000s
sys	0m0.000s

time to search using grep:

username@host:~$ time grep -H foo *
filename.txt:football season is approaching
 
real	0m0.002s
user	0m0.000s
sys	0m0.000s

As you can see, to use the one liner technique it appears to be as simple as “compressing” your scripts down to a single line, making sure they are shell friendly, and executing them with the “-r” option. Some other ones that I worked out with PHP are below:

to search for ‘foo’ and print matching lines in a single file

php -r '$s="foo";$ls=file("filename");foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}'

to search for ‘foo’ and print matching lines in all files in the current directory

php -r '$s="foo";$fs=scandir(".");foreach($fs as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}}'

to search for ‘foo’ and print matching lines recursively from the current directory

php -r '$s="foo";$t=new RecursiveDirectoryIterator(".");foreach(new RecursiveIteratorIterator($t) as $f){$ls=file($f);foreach($ls as $l){if(strpos($l,$s)!==false)echo$l;}}'

Observations on HTML

In December of 2012 HTML5 became HTML5 Logofeature complete according to the W3C, “meaning businesses and developers have a stable target for implementation and planning.” They continued to describe HTML5 as, “the cornerstone of the Open Web Platform, a full programming environment for cross-platform applications with access to device capabilities; video and animations; graphics; style, typography, and other tools for digital publishing; extensive network capabilities; and more.”

What a long road. Think back to the turn of the century. Shortly after HTML 4.01 was published as a W3C recommendation and XHTML 1.0 had it’s turn, it was said to be “developed to make HTML more extensible and increase interoperability with other data formats.” I remember the routine. Best practice at the time was separating form from content, moving to CSS, utilizing unobtrusive JavaScript, and practicing graceful degradation. Don’t forget to close your tags. Make it XML. Oh, and the machines are coming! :-)

As XHTML 2 approached it became clear that it would be an entirely new way of doing things, and not just an incremental approach that preserved compatibility. In 2004, Mozilla and Opera published the “Position Paper for the W3C Workshop on Web Applications and Compound Documents.” Some key sections included headings like, “Backwards compatibility, clear migration path”, “Users should not be exposed to authoring errors”, and “Scripting is here to stay.” Ultimately the initiatives were voted down and in response the WHATWG was formed. Between the years of 2007 and 2009, not only did the W3C accept WHATWG’s “Proposal to Adopt HTML5“, they allowed the XHTML 2 Working Group’s charter to expire, even going on to acknowledge that HTML5 would be the standard rather than the XML variant XHTML5. Regarding the two formats they wrote, “The first such concrete syntax is the HTML syntax. This is the format suggested for most authors.”

Since then, the whole web has been marching toward HTML5 domination, steadily learning best practice and implementation. In the earlier days I recall it not being as rapid as more recent, with people discussing the semantics, along with calls to prepare, but there has been a ton of solid information on the topic for awhile now and the momentum has shifted. Not only has the WHATWG decided that HTML is a living standard while the W3C publishes regular snapshots, the working draft of HTML 5.1 has been issued.

Lastly, I find it interesting to see the various web development strategies work themselves out as the craft changes. Graceful degradation (desktop centric) has steadily given way to a solid progressive enhancement (mobile first) approach as the web continues to gain in mobile traffic. In addition there are quite a few ideas going around on how to best accommodate all of the client browsers, especially in the comments. Should one start with an adaptive web design, and how is that related to responsive web design? Is one really a part of the other and should we have a strategy utilizing both? Maybe that’s the future… I guess it depends.

Time flies when you are having fun. Certainly I can be sure about one thing, I still like to close my tags. :-)

HTML5 Logo by W3C.

Compiling TrueCrypt on Raspberry Pi

These steps were gleaned from the work completed by Reinhard Seiler.

Other than the manual download and placement of the TrueCrypt source, the rest should be fairly hands off. So first, get the “TrueCrypt 7.1a Source.tar.gz” package from http://www.truecrypt.org/downloads2 and copy it to /usr/local/src/ on the Raspberry Pi.

Secondly, run the commands below, oh, and at your own risk of course… :-)

#!/bin/bash
#get source files other than the TrueCrypt source
sudo wget -P /usr/local/src http://prdownloads.sourceforge.net/wxwindows/wxWidgets-2.8.11.tar.gz
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11.h
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11f.h
sudo wget -P /usr/local/src/pkcs11 ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v211/pkcs11t.h
 
#get and install dependent packages
sudo apt-get -y install libgtk2.0-dev libfuse-dev nasm libwxgtk2.8-dev
 
#extract, configure, and make wxWidgets
sudo tar -xzvf /usr/local/src/wxWidgets-2.8.11.tar.gz -C /usr/local/src
cd /usr/local/src/wxWidgets-2.8.11/
./configure
make
 
#setup, extract
export PKCS11_INC=/usr/local/src/pkcs11
sudo tar -xzvf /usr/local/src/TrueCrypt\ 7.1a\ Source.tar.gz -C /usr/local/src
cd /usr/local/src/truecrypt-7.1a-source
 
#comment out some lines that prevented building
sed -i 's#TC_TOKEN_ERR (CKR_NEW_PIN_MODE)#/*TC_TOKEN_ERR (CKR_NEW_PIN_MODE)*/#g' Common/SecurityToken.cpp 
sed -i 's#TC_TOKEN_ERR (CKR_NEXT_OTP)#/*TC_TOKEN_ERR (CKR_NEXT_OTP)*/#g' Common/SecurityToken.cpp
sed -i 's#TC_TOKEN_ERR (CKR_FUNCTION_REJECTED)#/*TC_TOKEN_ERR (CKR_FUNCTION_REJECTED)*/#g' Common/SecurityToken.cpp
 
#compile, build, make!
sudo make WX_ROOT=/usr/local/src/wxWidgets-2.8.12/ wxbuild
sudo -E make WXSTATIC=1
 
echo
echo TrueCrypt should be found in /usr/local/src/truecrypt-7.1a-source/Main/

 
Finally, copy the compiled ‘truecrypt’ binary from /usr/local/src/truecrypt-7.1a-source/Main/ to /usr/local/bin/.

Afterwards, I was able to run it from the command line and view the help. There are two items in particular that may be of interest for those going through these steps. One is, when attempting to mount an encrypted volume with the Raspberry Pi that I have, I received the following:

Error: device-mapper: reload ioctl on truecrypt1_1 failed: No such 
file or directory
Command failed

 
It appears as though it may be due to a particular kernel module not being compiled in. So I added the “-m=nokernelcrypto” command line option and was successful.

TrueCrypt on the Raspbeery Pi

For more command line usage, see their website @ http://www.truecrypt.org/docs/?s=command-line-usage. I haven’t tested it yet, but it should work graphically as well.

Also, though some may not want to edit the source directly, I ended up commenting out lines 660, 661, and 662 of /usr/local/src/truecrypt-7.1a-source/Common/SecurityToken.cpp in order for the compile to work (see the sed lines in the script I provided above). Here’s the difference between the original and the changed source:

diff ./SecurityToken.cpp \
/usr/local/src/truecrypt-7.1a-source/Common/SecurityToken.cpp 
660,662c660,662
< 			TC_TOKEN_ERR (CKR_NEW_PIN_MODE)
< 			TC_TOKEN_ERR (CKR_NEXT_OTP)
< 			TC_TOKEN_ERR (CKR_FUNCTION_REJECTED)
---
> 			/*TC_TOKEN_ERR (CKR_NEW_PIN_MODE)*/
> 			/*TC_TOKEN_ERR (CKR_NEXT_OTP)*/
> 			/*TC_TOKEN_ERR (CKR_FUNCTION_REJECTED)*/

 
While there are some compromises in the process described above, it was the only way I could get it compiled in the time I allotted to the task.

Jigging with Debian

I found a package manager for Debian that I’ve never heard of: Wajig.

  • History: Motivations for Wajig

If you’ve tried to remember all the different commands to get different information about different aspects of Debian package management and then used other commands to install and remove packages then you’ll know that it can become a little too much.

Swapping between dselect, deity, deity-gtk, aptitude, apt-get, dpkg, gnome-apt, apt-cache, and so on is interesting but cumbersome. Plus personally I find dselect, deity, and aptitude confusing and even though I’ve spent hours understanding each of them, I don’t think the time was particularly well spent.

This Python script simply collects together what I have learnt over the years about various commands! Clearly I have yet to learn all there is.

As Andrew Tanenbaum once said: “The nice thing about standards is that you have so many to choose from.” Even Debian documentation seems to advocate one tool, aptitude, or the older, apt-get, depending on where you look (at least at this time)…

  • Source one

aptitude – CLI and ncurses front end for Apt (recommended)

  • Source Two

The recommended way to upgrade from previous Debian GNU/Linux releases is to use the package management tool apt-get. In previous releases, aptitude was recommended for this purpose, but recent versions of apt-get provide equivalent functionality and also have shown to more consistently give the desired upgrade results

As for myself I seem to default to apt-get. In addition, the auto removal tools that are offered in any package management tools make me want to proceed with caution. Though, I have been told I worry too much. :-)

Visualizing Motion

The programmability of the Raspberry Pi comes in handy when you want to change the behavior of a circuit without moving a single wire. In this case, I decided the data I was logging with my former motion detection script was largely useless because it only ever recorded when a motion event occurred, but didn’t hint to how long it had happened for.

So, while I admit the new method probably isn’t the -best- way there is, I believe it to be incrementally better. :-) The main difference being that “sleep” is called for half a second in the mix to allow for the line to start at bottom, progress to top, and then back down after the event occurs. I suppose it is inaccurate in that motion didn’t actually happen exactly in this manner, but it does allow for a nicer graph. Google Chart Tools is used along with the PHP built-in web server effectively piping the “data.js” log file to Google for displaying. I know the Pi has to be online…

Finally every evening at 23:59, cron runs a maintenance script and moves the data around for archiving (I love this little linux box). My thoughts on further improvements have been pointing me toward PHPlot instead of the Annotated Time Line from Google or maybe even utilizing kst. Also I would like to avoid that half second delay in future revisions… oh, and I haven’t tested what happens if I dance around in front of the thing right at midnight. :-)

source for motion.sh

#
#!/bin/bash
function setup {
  gpio export 17 out
  gpio -g write 17 1
  gpio export 18 in
  start=0
  echo The PIR sensor is initializing and calibration has begun.
  i=0; while [ $i -lt 40 ]; do i=$(($i+1)); echo $i; sleep 1; done
}
 
function loop {
  while true
  do
    if [ `gpio -g read 18` -eq 1 ]; then #PIR sensor activated
      if [ $start -eq 0 ]; then
        echo '[new Date('`date +"%Y, "`$((`date +%m`-1))`date \
          +", %d, %H, %M, %S"`'), 0],' | tee -a \
          /opt/GoogleVisualization/app/data.js
        sleep .5
        echo '[new Date('`date +"%Y, "`$((`date +%m`-1))`date \
          +", %d, %H, %M, %S"`'), 1],' | tee -a \
          /opt/GoogleVisualization/app/data.js
        start=1;
      fi
    else #PIR sensor de-activated
      if [ $start -eq 1 ]; then
        echo '[new Date('`date +"%Y, "`$((`date +%m`-1))`date \
          +", %d, %H, %M, %S"`'), 1],' | tee -a \
          /opt/GoogleVisualization/app/data.js
        sleep .5
        echo '[new Date('`date +"%Y, "`$((`date +%m`-1))`date \
          +", %d, %H, %M, %S"`'), 0],' | tee -a \
          /opt/GoogleVisualization/app/data.js
        start=0;
      fi
    fi
  done
}
 
setup; loop

 
excerpt of data.js

[new Date(2012, 10, 28, 08, 06, 30), 0],
[new Date(2012, 10, 28, 08, 06, 31), 1],
[new Date(2012, 10, 28, 08, 07, 10), 1],
[new Date(2012, 10, 28, 08, 07, 11), 0],
[new Date(2012, 10, 28, 08, 07, 16), 0],
[new Date(2012, 10, 28, 08, 07, 17), 1],
[new Date(2012, 10, 28, 08, 07, 22), 1],
[new Date(2012, 10, 28, 08, 07, 23), 0],

 
source for index.php

<?php
$hostname='bb.local';
$directory='/opt/GoogleVisualization/app/';
?>
 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="content-type"
    content="text/html; charset=utf-8" />
  <title>Karl's Passive Infrared Sensor Graph</title>
  <script type="text/javascript" 
    src="http://www.google.com/jsapi"></script>
  <script type="text/javascript">
    google.load('visualization', '1', {packages: 
      ['annotatedtimeline']});
    function drawVisualization() {
      var data = 
        new google.visualization.DataTable();
 
      data.addColumn('datetime', 'Date');
      data.addColumn('number', 'Status');
 
      data.addRows([
        <?php include ('data.js'); ?>
      ]);
 
      var annotatedtimeline = new
        google.visualization.AnnotatedTimeLine(
        document.getElementById('visualization'));
        annotatedtimeline.draw(data, 
        {'displayAnnotations': true});
    }
 
    google.setOnLoadCallback(drawVisualization);
 
  </script>
</head>
<body style="font-family: Arial;border: 0 none;">
  <h1>Karl's Passive Infrared Sensor Graph</h1>
  <div id="visualization" style="width: 900px; height: 300px;">
  </div>
  <br />
  <a href="http://<?php echo $hostname ?>/archive">Archive</a>
</body>
</html>

 
source for the maintenance script

#!/bin/bash
directory=$(date +"%Y-%m-%d")
sleep 60 #wait until midnight, cron is set for 23:59 as this keeps the directory names and dates aligned
mkdir /opt/GoogleVisualization/app/archive/$directory
mv /opt/GoogleVisualization/app/data.js /opt/GoogleVisualization/app/archive/$directory/
cp -pr /opt/GoogleVisualization/app/index.php /opt/GoogleVisualization/app/archive/$directory/

Responsive Design View in Firefox

With the faster release cycle and updating of Firefox, it’s been interesting to see features show up in the application like they do in web sites. They’re there the next time you load it, and are discovered almost by accident if you do not go searching for them on purpose.

The other day, while testing various display sizes on this very site, I noticed a new developer tool that was released called Responsive Design View. This special viewing mode was released in Firefox 15, and allows for various device sizes to be represented using the Gecko layout engine without too much hassle.

How nice. :-) The last feature I found out by accident that made me think, “wow cool,” was the 3D view that was released in Firefox 11. It would have been wonderful to have while developing some animated tabs I had running on here back in the day. Particularly because they would pop up from behind the main container of content on the site… so I had to visualize them behind there waiting for an event to happen to send them shooting up via some jQuery effects.

Logging motion with the Raspberry Pi

Since purchasing the Pi Cobbler Breakout Kit I wanted to digitally plan out the project before connecting a single wire. There seems to be a lot of choice for circuit design software (open source even), but the one I chose is called Fritzing.

Fritzing on OS X

After I began to download it, I found out that Adafruit has a library of parts that they update for this very software. I realize it isn’t an enormous leap from lighting an LED with a Raspberry Pi to controlling a PIR sensor, but the amount of fun I’ve been having with the ecosystem surrounding this device (and devices like it, whether single-board microcontrollers or single-board computers) compels me to announce my findings. Even the more minutely incremental.

A Raspberry Pi that logs motion it detects:

While viewing using Fritzing, you can see where positive and negative flow throughout the circuit, move wires around, add more parts, etc. I didn’t connect the breakout kit to the Raspberry Pi in software as I was unsure of how to represent a twenty-six pin ribbon cable, but the rest is there.

Currently, the button turning on the motion detector, and the motion detector doing the logging runs the following scripts below. To install, test and build on, copy/paste them into two files placed in the same directory, then run button.sh. From there, motion.sh is called when pressing the button and stopped when pressing it again. As you can see, I tried to model the scripts similar to the way the Arduino programming language has a setup and loop function.

There’s obvious room for improvement. Play a sound when detecting motion. Log to a Google spreadsheet. You get the idea.

source for button.sh

#!/bin/bash
 
function setup {
	button=0 #sda0 pin where the button is connected
	state=0 #the initialized state value of the pir sensor
	oldstate=0 #the initialized old state value of the pir sensor
	val=0 #the initialized pin value
	oldval=0 #the initialized old pin value
}
 
function loop {
	while true
	do
		val=`gpio -g read $button`
		if [[ $val -eq 1 &amp;&amp; $oldval -eq 0 ]]; then
			state=$((1-$state))
			sleep .1
		fi
 
		oldval=$val
 
		if [ $state -eq 1 ]; then
			if [ $oldstate -lt $state ]; then
				#echo disabled
				if [ "$motionPID" != "" ]; then
					clear
					echo attempting to kill motion.sh and disable the PIR sensor
					kill $motionPID
					gpio -g write 17 0
				fi
				clear
				echo Press the button to enable the PIR sensor and begin logging. Press the button again to disable. CTRL-C exits button.sh.
			fi
		else
			if [ $oldstate -gt $state ]; then
				#echo enabled
				`dirname $0`/motion.sh &amp;
				motionPID=`echo $!`
			fi
		fi
 
		oldstate=$state
	done
}
 
setup; loop


source for motion.sh

#!/bin/bash
function setup {
	gpio export 17 out
	gpio -g write 17 1
	gpio export 18 in
	motion=0
	alert=0
	init=0
}
 
function loop {
	while true
	do
		if [ `gpio -g read 18` -eq 0 ]; then #no motion, no alert
			motion=0
			alert=0
			clear
		else #motion, no alert
			if [ $init -eq 0 ]; then
				echo The PIR sensor is initializing and calibration has begun... Sleeping for forty seconds. Please be as still as possible.
				init=1
				i=1; while [ $i -lt 41 ]; do echo $i; sleep 1; i=$(($i+1)); done
			fi
			if [ $alert -eq 0 ] &amp;&amp; [ $init -eq 1 ]; then #motion, alert
				#aplay `dirname $0`/motion.wav &gt; /dev/null 2&gt;&amp;1
				echo 'Motion has been detected: ' `date +"%Y-%m-%d_%H-%M-%S"` | tee -a `dirname $0`/motion.log
				alert=1
			fi
		fi
	done
}
 
setup; loop

UPDATE (2012-12-03): A better script for logging motion (and visualizing it) has been published at Visualizing Motion

Lighting an LED on the Raspberry Pi

Since getting a Raspberry Pi I’ve wanted to use the GPIO pins, but have tried to heed the various warnings regarding static and other issues one might face like using 3.3v vs 5v devices or how much power peripherals draw. I could have started clipping onto the pins directly, but decided to progress cautiously and get them in a more test friendly environment. Not that I wont fry anything now, I’m just trying to be methodical about it.

So I went out and purchased a breakout kit for it and followed the guide on soldering it together. Since, I’ve completed a similar experiment on the Arduino, I thought my first one on the Pi should be something along the lines of lighting an LED. It sort of seems like the “Hello, World!” of software development. First I needed a refresher on resistors (especially attempting to use the right ones) and how the breadboard works. Then I needed to find a way to get data down the pipes. I chose WiringPI.

WiringPi is a Wiring library written in C and should be usable from C++ and many other languages with suitable wrappers

I also found a tutorial on the WiringPi Author’s site that explained exactly what I wanted to do: GPIO Examples 1 – A single LED.

Arduino Motion Detector

As I have been traveling through Getting Started with Arduino I’ve been trying to take on the “tinkering” mentality described in chapter two titled, “The Arduino Way“. And as the text in the book states, “You should now experiment by looking at all the possible devices that have two contacts that close.” So I made a purposeful visit to Radio Shack to look for parts to hook up to it. What I came up with was a Parallax PIR Sensor.

The package didn’t include documentation for the sensor, but I was able to locate it on their website. This gave me enough confidence to try and hook it up… particularly after deciding to modify the “Using a Pushbutton to Control the LED” example and corresponding code from the Getting Started with Arduino book.

The result was a little motion detector, that turned the LED on while it detected motion, and then turned off shortly after the motion stopped. The sky is the limit I suppose (or time and money, right?). The next stop for me seems to be to hook up the Raspberry Pi to the Arduino in some manner.