That's all very nice but someone has already implemented this with a lot of other nice features. We are going to look at the SimpleTest framework of PHP.
I am looking at the currently latest 1.0.1 version that already includes autorun.php.
Ubuntu packages the PHP Simpletest package but sudo aptitude install php-simpletest only gets you version 1.0 which is relatively old.
So the best course of action is to download the simpletest_1.0.1.tar.gz file from the Simpletest web site and unzip it in a place where your web server can reach it.