The Cobalt RaQ CGI RAQ
Installing, running and debugging Perl CGI scripts on a Cobalt RaQ
Version 1.0 * 15-06-2000, last updated : 27-07-2000
Newest version at http://users.iol.it/hpstr/
© 2000 H.P. Stroebel
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1
or any later version published by the Free Software Foundation;
with no Invariant Sections being LIST THEIR TITLES, with no
Front-Cover Texts being LIST, and with no Back-Cover Texts being LIST.
A copy of the license is included here.
Leave the
copyright notices unchanged; if you modify and redistribute this document, you have
to preserve the copyright notices and add - according to section 4 of the license - add
your own copyright for your modifications and list the author responsible for the modifications.
Introduction
This FAQ is intended as a
practical CGI introduction for novice Webmasters using
a Cobalt RaQ Server. It is intended for and approved on RaQ 3i, so it
cannot give advice for RaQ, RaQ 2 or RaQ 4, but nearly all of the
procedures should apply for them, too. As there is poor stuff specific
to a Cobalt
RaQ, it should be suitable as a normal, simple CGI Troubleshooting FAQ
for Linux
newbies. Some things are not described in the 100% correct matter (I
beg the pardon of
the Unix/Linux freaks), but it's easier to understand, and the
difference, at
least for the novice, is trivial.
Any comments (not questions, see below), error reports or corrections of bad english are
welcome. If you would like to provide further stuff, contact me.
Please note that changing the configuration of your server outside the GUI could
void your warranty. I provide this FAQ "as is" and take no responsibility for
damages, misconfigurations etc. Do all procedures and actions described in this
text on your own risk !
What is a RAQ ?
A RAQ is a document answering Rarely Asked Questions; the word "rarely" is to be considered
as ironic... More widely it is known as a cute Server Appliance by Cobalt Networks :-)
What is a CGI script ?
CGI is the 'common gateway interface'. It can exchange data with a program running
on the server to provide mail forms, guestbooks, databases etc. The programs that
run on the server are often scripts, plain text files that contain several commands
to be executed. A CGI program can be written in most languages, like C, C++, Perl,
Python, or even as shell scripts. One of the most common scripting languages to
write CGI scripts is Perl (http://www.perl.com).
As most of the CGI programs available on the web are written in Perl, this RAQ
is Perl orientated, too.
What is cgiwrap ?
CGIWrap is a gateway program that allows general users to use CGI scripts and HTML forms without compromising
the security of the HTTP server. Scripts are run with the permissions of the user
who owns the script. In addition, several security checks are performed on the
script, which will not be executed if any checks fail. CGIWrap is used via a URL
in an HTML document. As distributed, cgiwrap is configured to run user scripts
which are located in the ~/public_html/cgi-bin/ directory.
(Taken from http://www.unixtools.org/cgiwrap/)
The Cobalt RaQs seem to run a customized version of CGIWrap.
How do I run CGI/Perl scripts?
If necessary, edit some variables within the script,
upload it to your web directory, give it the right permissions, and run it.
How to edit a Perl script ?
If you have to edit the script, it is important to choose the right editor (that
concerns Windoze users), as some editors can cripple the script. Do NOT
use Notepad.
You could use Wordpad (disable word wrap), but there are two of the best HTML
and general purpose editors on the web available for free :
Arachnophilia
Care-Ware (check the Readme for that) by Paul Lutus, contains even a small FTP
upload tool (but unfortunately it`s not suitable for CGI programs, as you can`t
set permissions)
HTML Editor Phase 52
Freeware by Uli Meybohm, only in German, one of the best editor`s available.
If you use it, be sure to embed the excellent free Selfhtml
documentation (in German). It contains even a basic Perl and Javascript reference, very useful.
Linux / RaQ
On your Raq, the two included editors are vi (excellent, but hard to learn; try with some dummy files first),
and pico, an editor that behaves similar to DOS Edit; it is a fine editor for novice Linux users.
If you use pico, you should become used to call it with the -w option (type : pico -w filename) to disable
word-wrap. You could mess up scripts that don`t tolerate word wrap in command sequences.
Perl Syntax
Be sure to avoid the most common syntax errors when editing the script :
If you enter an email adress, be sure to mask the '@' with a leading backslash.
It has to look like that : me\@domain.tld
You should not fool around in a script when you don`t know exactly what you`re
doing. If the script complains about compilation or syntax errors, check first
if you have not deleted one of the following characters :
Each command line must end with a ';'
(error message : compilation error)
All brackets must be closed, unintendendly you could have deleted a '}'
(error message : missing right bracket)
What are the required paths ?
The following variables are the most common ones that you have to insert in a
cgi script
Perl 5 Location (that is always the first line in every perl script):
#!/usr/bin/perl
Sendmail Location :
/usr/sbin/sendmail
Script Url :
http://domain.tld/scriptname
or, if you have created a /cgi-bin :
http://domain.tld/cgi-bin/scriptname
Full path to main web directory of a website :
/home/sites/sitename/web
(it isn`t really, but it`s a link to the real path)
You can insert the standard CGI environment variable instead (note the double
quotes):
$ENV{'DOCUMENT_ROOT'}
The full path to your cgi-bin directory - if you have one - is:
"$ENV{'DOCUMENT_ROOT'}/cgi-bin"
To find out more about your environment variables, try this little script :
#!/usr/bin/perl
# test.cgi : prints out the CGI environment variables
# H.P. Stroebel, hpstr@operamail.com, 15-06-2000
print "Content-type: text/html\n\n"; # Tell the browser that
# it`s text/html
print "Hello World !\n\n"; # Say hello
foreach $item (keys %ENV) # a loop for every env. variable
{ print "$item = $ENV{$item} \n"; } # print the variable
$dir=`pwd`; # get the working directory (shell command)
print "Working Directory : $dir \n"; # print it
print "\nSo long, and thanks for the fish !"; # Say goodbye
exit 0; # Done :-)
Normally, this script would be a single line (that`s Perl... :-). I commented
it a bit, so you see how a (very basic) Perl CGI script works.
Save it as "test.cgi", upload it to your server and let it run, following the
instructions on this site. It is a good idea to print out the information
provided by the script for further reference.
Remember to remove it later, as everyone can run this script !
How to upload
Be sure to upload the script and any text files in ASCII mode. If the program
contains graphics (*.gif, *.jpg, etc.) be sure to upload them in BINARY mode.
It is good server practice to use only lower case letters for script and related
filenames, that may not matter on certain operation systems, on the better ones
it DOES matter.
Normally you upload the script into the web directory of the site, or into the
created /cgi-bin. Connecting to the RaQ by FTP your initial directory is your
user directory, so move up to the web directory of the site. In most Windoze FTP
clients you can set another starting path; insert /web as initial directory.
You should upload the file as a site admin or a site user, NOT as admin
or even root (you can`t ftp in as root by default, and there are good reasons for that).
cgiwrap runs the script, as described above, as a site user. Under certain circumstances
it cannot be run as admin anymore (as admin can belong to max. 32 groups, and each site
has it`s own group). And, it`s a security issue : the admin password is transmitted in plain
text via ftp; it can be easily sniffed...
What is a /cgi-bin directory ? Where is it ? Do i need one ?
On most servers, cgi scripts are allowed to be run only from a special directory
beneath the web directory called /cgi-bin, or /cgi-local.
On the Cobalt RaQ`s cgi scripts can be run from everywhere in the web directory,
so you don`t need one.
It has - even not changing the above mentioned behavior of the RaQ - some sense
to create a directory /cgi-bin and run the scripts from there.
CGI File Permissions
Under Unix, permissions are set by a number. A higher number means more permissions.
The first number defines the rights for the owner of the file, the second his
group, and the third all others. To learn more about permissions, read the chmod
manpage. Use Telnet or SSH to access your server and type man chmod; navigate
with cursor keys and the space bar, exit the man page by typing q.
If you don`t already have it, you should consider to buy at least a basic book
on Linux.
Instructions for CGI programs available on the net tell you often to set the permissions
of the executable program to 755.
For security reasons, you should first try to give the file the most minimal permissions
as possible, so set them first to 700 (RaQs not running CGIWrap may require
705 or 755).
The cgi-bin directory itself (and directories beneath it) should have 701 or,
if that doesn`t work, 711.
Data or configuration files used by the CGI program may require 600, 604
or 644 (try it in that order).
How do I set permissions for my cgi script?
There are two (main) ways to change permissions of a file :
Get on the shell of your cobalt RaQ, go to the directory where the script is,
and type, for example, chmod 700 scriptname to give the file scriptname the permissions
700 (=read, write, execute only for the owner of the file).If you get the answer
"Permission denied", type su to get root status.
In your FTP client (e.g. CuteFTP or WS-FTP) mark the file(s) or directory,
right click on it and select chmod or change attributes. Please read the instructions
of your FTP client. For changing the file`s permissions for example to 700, click
on the check boxes 'read, write, execute' under owner.
How do i run it ?
CGI programs are mostly called by a hyperlink from within a HTML page. Another
way are Server Side Includes (SSI) , mostly used for non-interactive programs
(e.g. hit counters).
Interactive programs may require some values (e.g. a form mail processor). The
values may be created by a HTML form, containing hidden fields, input fields or
both of them. There are two ways to pass values to a CGI program, the GET
and the POST method.
Read the instructions provided with the cgi script. Good scripts mention the
required method in the documentation or as a comment in the script itself. Better
scripts accept both of them.
If your script is not a good one, ask the author,
throw it away, or try to find it out by yourself :
If the script contains a line like
$data = $ENV{'QUERY_STRING'};, it uses the GET method.
If it contains a line like
read(STDIN, $data, $ENV{'CONTENT_LENGTH'});, it uses the POST method.
If it contains none of the above, most likely the script doesn`t expect values.
If you design a HTML form to call the script, you have to pass the method. A sample
form header for the POST method looks like this :
<form action="http://sitename/cgi-bin/scriptname" method="POST">
Another way to pass values to a CGI program is to attach the values directly in
the hyperlink without using a form :
http://sitename/cgi-bin/scriptname?&name=arthur&surname=dent&question=what?
Here '?' is the delimiter to define the start of the values, '&' is the delimiter
for the value pairs, the value itself is passed by the string 'value name'='value'.
Nice to read, but my script doesn`t run. I am getting '500 Internal Server Error',
'File not found' or 'CGIWrap error' when running CGI scripts ?
First, consider the error messages for cgi scripts of the RaQ like a 'general
protection fault' in another widely known os : the reason can be anything, but
- in most cases - not the one that the error message tries to make you believe.
Double check all the following most common errors :
Is cgi enabled for the virtual site ? Check it in the admin panel.
Did you call it by typing the right URL ? Check both the URL and the script name
for upper/lower case letters.
Did you upload it to the right directory ?
If the script expects values : did you use the right method (POST/GET) ?
In the first line of the script, does the path point to perl ?
If you edited path variables in the script, are they correct ? Normally there
is no trailing slash.
If the script requires sendmail, is the path to sendmail correct ?
Did you upload the script in ASCII mode ?
Does the script have the right permissions ?
Did you upload it as a site user or as a site admin ?
It still doesn`t work ? Check the above points again.
I checked all the items above even three times, and it still doesn`t work.
OK, we`ll do some more debugging (and in 90% of all cases you`ll note that it`s
one of the above errors, so did you check them really ?)
First, we`ll use the debug function of CGIWrapd, which by default is enabled on
the RaQ.
Type the following URL in your browser:
http://sitename/cgiwrapDir/cgiwrapd/scriptname
You should see some environment variables, and at the bottom of the displayed
site, the output of the cgi script. Consult that for error messages. Read the
output carefully, in most cases it indicates that a required file could not be
found or accessed, or a syntax error in the script.
If you see something like
Content-type: text/html
<html>...
the script has been compiled and works. Most likely the URL calling it normally
is incorrect. Check the URL, and call it directly in the browser.
If you see nothing, or a 'File not found' message, most likely the script name
or the URL is not correct, the script has the wrong permissions set, you uploaded
it to the wrong directory, or CGI execution is not allowed for that site.
If that doesn`t help, access the Raq using SSH or Telnet (if you use Telnet to
connect over the Internet, run and install SSH)
Go to the /web directory or the /cgi-bin you have created beneath, and run the
script on the shell by typing
perl -w scriptname | more
and examine the output.
The '-w' parameter tells Perl to create additional warnings about errors or strange
constructs. For interpreting the output, refer to the instructions mentioned above.
The '| more' makes the screen stop after each page, if the output is greater
than one page.
Another way to debug a perl script is to put the following command in the second
line of the perl script after #!/usr/bin/perl
use CGI::Carp qw(fatalsToBrowser);
This line tells the browser to output error messages to the browser. After debugging,
you should comment out this line by adding a '#' in front of it.
It seems that this function does not work on the (my) RaQ.
Where can i get CGI/Perl scripts ?
A very good starting point is http://www.cgi-resources.com,
where you can find zillions of (user rated) CGI programs, documentation and more.
What about security ?
Only a few basic things :
DO :
Strip out at least HTML tags or verify user entries to prevent malicious code,
if you let the script store user entries as HTML pages (guestbooks, classified
ads, etc.)
If your users load up CGI scripts, you should check them for
security issues (system commands, sendmail, open/write/unlink files etc.)
Let your users use the /cgi-bin, or better, allow CGI execution only in the
/cgi-bin (on the RaQ this requires a workaround; i `ll try to add this in the
future).
DON`T :
Don`t upload CGI scripts as admin or root (they run under your user-id, and
have your rights)
Don`t allow directory listing of your /cgi-bin (give it the right permissions,
or/and put a blank html file in it)
Don`t use scripts accepting unverified user input !
Don`t put data files or configuration files in a directory that can be listed
by others; prefer paths beneath your /cgi-bin, or even better, outside your /web
directory, and check the permissions !)
Don`t let the CGI program store or mail private user data or credit card information
to you without encryption ! (This is not easy to implement for beginners, though...)
Where do i learn more ?
- about CGI ?
- about security ?
- about Perl ?
- about Cobalt ?
I have more questions.
If the instructions above don`t solve your problem, it`s rather specific. Post them to the Cobalt mailing list, to
a CGI or a Perl newsgroup.
Please don`t mail me further questions. I wrote this FAQ in my VERY limited free
time, and i want to use some of it for myself. I will not give any personal support, and i will not debug Perl scripts or
server configurations (at least not for free). I frequent the Cobalt mailing lists, and
if I have the time and some further advice for you, i`ll try to answer your question there.
H.P. Stroebel
Germany
Contact me