brian d foy

Web servers aren't very friendly. When a CGI program fails, you're typically greeted with a brusque and uninformative server error, with no constructive criticism about why the program failed. The detective work is left to the person maintaining the program. Sometimes the program's author is unavailable, making diagnosis all the more difficult.

I present four methods for dealing with uncooperative CGI programs. The first two, "Fiddling with die()" and "Redirecting STDERR", demonstrate basic techniques in error handling that you can apply to more than just CGI programs. The third method illustrates the CGI::Carp module, which adapts those two methods to the Web. The last method should be part of every CGI developer's arsenal - a custom built error handling routine that not only outputs error messages to the right places, but can also send mail alerts or take other action to fix the problem.

A respected physicist once told me that an expert is someone who has made every mistake. I'll defer the expert moniker and simply share what I've learned from my mistakes.

1. Fiddling With die()

What happens when you need to fix a CGI program written by someone else? I've had plenty of clients ask me to patch their existing program so they'd have something in production while I created a long-term solution. Along the way, I've encountered many CGI programs that - unfortunately - use die(). Usually the problem is a failed open() that triggers the die() and terminates the program before the HTTP header is printed. The result is an invalid Web page and an infuriating error message from the server.

The first time I was given a CGI program that used die(), I simply replaced all occurrences of die with something else:

     s/die/$another_thing/g;

That was a big mistake, since the three letters 'die' aren't always a function call. Luckily, no one was around to see this. I needed a more intelligent approach.

Rather than changing every instance of 'die', I now simply redefine what die() does. Instead of printing to STDERR, I have die() print to STDOUT, starting with the HTTP header.

I change what die() does with $SIG{_ _DIE_ _}, which is triggered whenever any die() is called. I set it to an anonymous subroutine to redirect die()'s message from STDERR to STDOUT. The same trick works for warn() with $SIG{_ _WARN_ _}.

#!/usr/bin/perl 

# use an anonymous subroutine to replace die()'s 
# default behavior 

$SIG{_ _DIE_ _} = sub { 
    my $message = shift; 

    print STDOUT "Script died with this message:\n";
    print STDOUT "$message\n"; 
}; 

print STDOUT "Content-type: text/plain\n\n";
print STDOUT "This is from STDOUT\n"; 

# If /etc/pass doesn't exist, the die() triggers.
open FILE, '/etc/pass' or die "Can't open /etc/pass: $!\n"; 

This is a great tool if you inherit a program from a colleague, but I don't recommend it if you're creating your own program from scratch. It's probably fine for short scripts, but someone reviewing your code might not remember that you're trapping die() several hundred lines later. (Not that I know this from experience.)

2. Redirecting STDERR To STDOUT

What happens if your program redefines die(), but still prints to STDERR? If STDERR flushes before the server receives a proper HTTP header, you still lose.

I once had to debug a complicated mess of CGI programs whose documentation seemed to be in a seven-bit encoding of Cyrillic. The tangle of require()s and files was such a mess that I couldn't figure out what was printing to STDERR. I wanted to see the error message so I could infer where it came from, but I was on a VT100 terminal and watching the server log was annoying and time-consuming. Running the script from the command line interwove STDOUT and STDERR into an indecipherable pile of HTML.

I decided to redirect STDERR to STDOUT so that I could see the error message in Lynx, which would also nicely format the HTML. There is an example under "How do I capture STDERR from an external command?" in the Perl FAQ, but my situation was a bit trickier since I needed to print things in a certain order to avoid errors from Web server:

#!/usr/bin/perl

BEGIN {
    # we want STDOUT to flush right away 
    $| = 1; 
    print "Content-type: text/plain\n\n";
}

open STDERR, ">&STDOUT";

print STDOUT "This is from STDOUT\n"; 
print STDERR "This is from STDERR\n";

die 'This is from die';

Notice that I set STDOUT to autoflush. If I don't do that, STDERR flushes first and I get a "Malformed header from script" error, even though I printed the HTTP header in a BEGIN block.

3. Using CGI::Carp

Now that I've shown you the basics of managing fatal errors and STDERR, you can forget about them. Use Lincoln Stein's CGI::Carp module, bundled with the standard Perl distribution. CGI::Carp traps fatal errors with ease and sends them elsewhere. It contains two functions providing simplistic error handling - carpout() and fatalsToBrowser().

The carpout() function lets you redirect the output from die(), warn(), croak(), confess(), and carp() to another filehandle. The CGI::Carp documentation suggests setting up the redirection in a BEGIN block to catch compile-time errors:

  BEGIN {
      use CGI::Carp qw(carpout); 
      open(ERROR_LOG, ">>my_error_log") 
                      or die("my_error_log: $!\n");
      carpout(\*ERROR_LOG); 
  }

Beware! We've now redirected STDERR to a file. STDERR is where Perl prints the carpout.cgi syntax OK message after you test it from the command line with perl -cw carpout.cgi. I forgot this when I tried it the first time, and here's what I saw:

    dog[32] perl -cw carpout.cgi 
    dog[33]

I couldn't figure out what I'd done wrong. The -c switch (which checks a program for syntax errors without running it) always prints something. Running the script was equally fruitless; since its whole job was to die() that output disappeared into the cornfields too.

    dog[36] ./carpout.cgi 
    dog[37]

When I play with redirection, I usually need to draw a picture of what's redirected where to prevent such memory lapses.

I still needed to send the HTTP header to the server, since the carpout() function doesn't do that. The second method, fatalsToBrowser(), redirects fatal messages from die(), croak(), or confess() to the browser along with a minimal set of HTTP headers so that the server won't complain. It uses $SIG{_ _DIE_ _} to trap die() statements:

#!/usr/bin/perl 

use CGI::Carp qw(fatalsToBrowser); 

open FILE, "quotes.txt" or die "Can't open: $!\n";

    ...your code here...

close FILE;

Some HTML is then sent to my browser:

   <H1>Software error:</H1> 
   <CODE>No such file or directory 
   </CODE> 
   <P> 
   Please send mail to this site's webmaster for help.

and the error message is neatly recorded in the error log along with the time and filename.

[Wed Mar 18 04:20:48 1998] test.cgi: No such file or directory

4. Rolling Your Own

By far the best solution for large projects is to make a custom error handling routine. Instead of using die(), I have my own routine which I export from a module: cgi_error(). I can do various janitorial tasks while making sure that I get an intelligent message in the browser if something went wrong. I can even send myself nasty little mail messages:

sub cgi_error { 
    my $message = shift; 

    print <<"HTTP";
Content-type: text/plain

there was an error:

$message 
HTTP

    open MAIL, '| /usr/lib/sendmail -t -odq -oi';
    print MAIL <<"MESSAGE"; 
	
To: brian@sri.net
From: jiminy_cricket\@sri.net 
Subject: Your groovy CGI, baby.

something horrible has happened: 
$message 
MESSAGE

    close MAIL; 
}

I use cgi_error() wherever I'd normally use die().

I can add even more to cgi_error() to collect any other data that I need to diagnose the problem, and perhaps even to fix it. This was especially handy with one script that needed to be setuid. If it wasn't setuid, it lacked the permissions it needed to lock a database, and looped forever.

The program itself didn't break. There were no server errors. It simply ate as much processing time as it could, making the Web site seem very slow. A well-tuned cgi_error() diagnosed the problem - if the script couldn't get the database lock after several tries, it called cgi_error(), which made a few inquiries. Was the process with the lock still running? If not, cgi_error() launched a program to release the zombie lock. Did the program have the right permissions? If not, cgi_error() called a script that gave proper permissions to the program.

Whenever my cgi_error() attempts to fix a problem, it sends me a message saying what it tried to do - and what actually happened. I have found this to be a far superior solution to listening to clients complain "I don't know what's wrong with it - it's just broken!" or waking up before three in the afternoon to diagnose the problem manually. Once I developed a satisfactory cgi_error() function, I spent much less time debugging.

---

brian d foy is the Senior Programmer at Smith Renaud, Inc. (https://www.sri.net), and can be reached at comdog@computerdog.com.

---

References

The Perl FAQ: Section 8, System Interaction: https://www.perl.com/CPAN/doc/FAQs.

Randal Schwartz's WebTechniques columns: https://www.stonehenge.com/merlyn/WebTechniques/.

The CGI::LogCarp module: https://www.perl.com/CPAN/modules/by-module/CGI.

Documentation for $SIG{_ _DIE_ _}: https://www.perl.com/CPAN/doc/manual/html/perlvar.html.