Check Your Site With Tarantula
by Reuven Lerner - Jun. 05, 2008Comments (2)- Related Blog Posts
- OStatic Buffer Overflow.....
- Django Gets Its Own Foundation
- SproutCore Raises the Bar for Client-Side Programming
- Book Review: Deploying Rails Applications
- Create Rich Reports With Ruport
If you're running a Web site, then the last thing you want is to have a broken link. Broken links look bad, frustrate users, and confuse search engines. Even when links aren't broken, you can have pages that contain bad HTML, or server-side programs that fail when you enter data into them. These problems, while not fatal to your site's functionality, can still cause you a great deal of trouble.
There are a few partial solutions. If you're using Firefox, then you can download and install the "Web developer" add-on, which adds a number of new functions, not least of which is an easy and automatic connection to the W3C's HTML and CSS tests. But that only allows you to run tests manually. It would be much better if we could test all of the pages automatically, checking for broken links, bad HTML, and even seriously broken programs on a regular basis.
If this sounds good to you, and if you use Ruby on Rails, then you should take a look at Tarantula, a Rails plugin that executes a number of simple tasks against your Web site, producing a detailed report (in HTML, of course) describing the URLs that it crawled, and the responses it received from each URL.
Tarantula's functionality overlaps somewhat with the testing that you might do on your site, particularly if you are practicing test-driven development (TDD). However, because Tarantula runs against your actual Web site, it can help to identify problems with your production server, with user-generated data, and with HTML compliance.
One important thing to keep in mind is the fact that Tarantula crawls your Web site, using real HTTP requests, and getting real HTTP responses. If your site has 1,000 pages, then running Tarantula four times each day will increase the number of requests, and apparent users, by 4,000. This is not necessarily a bad thing, but you should keep these numbers in mind when you look at your Web statistics reports. Alternately, you can configure your statistics package, such as Webalizer, to ignore requests from Tarantula, such that these hits will not be incorporated into the final statistics.
Tarantula isn't the only program for Rails, or any other system, that can be useful in analyzing your site. What else can you recommend?
