Python CGI Programming – Shishir Kant Singh

The Concept of CGI

CGI is an abbreviation for Common Gateway Interface. It is not a type of language but a set of rules (specification) that establishes a dynamic interaction between a web application and the client application (or the browser). The programs based on CGI helps in communicating between the web servers and the client. Whenever the client browser makes a request, it sends it to the webserver, and the CGI programs return output to the webserver based on the input that the client-server provides.

Common Gateway Interface (CGI) provides a standard for peripheral gateway programs to interface with the data servers like an HTTP server.

The programming with CGI is written dynamically, which generates web-pages responding to the input from the user or the web-pages interacting with the software on the server.

The Concept of Web Browsing

Have you ever wondered how these blue-colored underlined texts, commonly known as hyperlinks, able to take you from one web-page or Uniform Resource Locator (URL) to another? What exactly happens when some user clicks on a hyperlink?

Let’s understand the very concept behind web browsing. Web browsing consists of some steps that are as follows:

STEP 1: Firstly, the browser communicates with the data server, say HTTP server, to demand the URL.

STEP 2: Once it is done, then it parses the URL.

STEP 3: After then, it checks for the specified filename.

STEP 4: Once it finds that file, a request is made and sent it back.

STEP 5: The Web browser accepts a response from the webserver.

STEP 6: As the server’s response, it can either display the requested file or a message showing error.

However, it may be possible to set up an HTTP server because whenever a file in a specific directory is requested, it is processed as a program rather than sending that file back. The output of that program is shown back to the browser. This function is also known as the Common Gateway Interface or abbreviated as CGI. These processed programs are known as CGI scripts, and they can be a C or C++ program, Shell Script, PERL Script, Python Script, etc.

The working of CGI

Whenever the client-server requests the webserver, the Common Gateway Interface (CGI) handles these requests using external script files. These script files can be written in any language, but the chief idea is to recover the data more efficiently and quickly. These scripts are then used to convert the recovered data into an HTML format that can send data to these web servers in HTML formatted page.

An architectural diagram representing the working of CGI is shown below:

Usage of cgi module

Python provides the cgi module consisting of numerous useful core properties. These properties and functions can be used by importing the cgi module, in current working program as shown below:

import cgi

Now, We will use cgitb.enable() in our script to stimulate an exception handler in the web browser to display the detailed report for the errors that occurred. The save will look as shown below:

import cgi

cgitb.enable()

Now, we can save the report with the help of the following script.

import cgitb 
cgitb.enable(display = 0, logdir = “/path/to/logdir” )

The function of the cgi module stated above would help throughout the script development. These reports help the user for debugging the script efficiently. Whenever the users get the expected result, they can eliminate this.

As we have discussed earlier, we can save information with the help of the form. But the problem is, how can we obtain that information? To answer this question, let’s understand the FieldStorage class of Python. If the form contains the non-ASCII characters, we can apply the encoding keyword parameter to the document. We will find the content <META> tag inside the <HEAD> section of the HTML file.

The FieldStorage class is used to read the form data from the environment or the standard input.

A FieldStorage instance is similar to the Python dictionary. The user can utilize the len() and all the dictionary functions as the FieldStorage instance. It is used to overlook the fields that have values as an empty string. The users can also consider the void values with the optional keyword parameter keep_blank_values by setting it to True.

Let’s see an example:

form = cgi.FieldStorage()   if ("name" not in form or "add" not in form):       
    print("<H1>Input Error!!</H1>")
   print("Please enter the details in the Name and Address fields!")    return 
print("<p>Name: file_item = form["userfile"]   
if (fileitem.file):      
    # It represents the uploaded file     
    count_line = 0       
    while(True):           
        line = fileitem.file.readline()   
        if not line: break           
        count_line = count_line + 1   # The execution of next lines of code will start here...

In the above snippet of code, we have utilized the form [“name”], where name is key, for extracting the value which the user enters.

To promptly fetch the string value, we can utilize the getvalue() method. This method also takes a second argument by default. And if the key is not available, it will return the value as default.

Moreover, if the information in the submitted form has multiple fields with the same name, we should take the help of the form.getlist() function. This function helps in returning the list of strings. Now let’s look at the following snippet of code; we have added some username fields and separate them by commas.

first_value = form.getlist("username")   f_username = ",".join(value)

If we want to access the field where a file is uploaded and read that in bytes, we can use the value attribute or the getvalue() method. Let’s see the following snippet of code if the user uploads the file.

file_item = form["userfile"]   
if (fileitem.file):      
    # It represents the uploaded file     
    count_line = 0       
    while(True):           
        line = fileitem.file.readline()           if not line: break           
        count_line = count_line + 1

An error can often interrupt the program while reading the content of the file that was uploaded. It may happen when a user clicks on the Back Button or the Cancel Button. However, to set the value – 1, the FieldStorage class provides the done attribute.

Furthermore, the item will be objects of the MiniFieldStorage class if the submitted form is in the “old” format. The attributes like list, filename, and file are always None in this class.

Usually, the form is submitted with the POST method’s help and contains a query string with both the MiniFieldStorage and FieldStorage items.

Let’s see a list of the FieldStorage attribute in the following table.

FieldStorage Attributes:

S. No.	Attributes	Description
1	*Name*	The Name attribute is used to represent the field name.
2	*File*	The File attribute is used as a file(-like) instance to read data as bytes.
3	*Filename*	The Filename attribute is used to represent the filename at the Client-side.
4	*Type*	The Type attribute is used to show the type of content.
5	*Value*	The Value attribute is used to upload files, read the files and return byte. It is a string type value.
6	*Header*	The Header attribute is used as a dictionary type instance containing all headers.

In addition to the above, the FieldStorage instance uses various core methods for manipulating users’ data. Some of them are listed below:

FieldStorage Methods:

S. No.	Methods	Description
1	*getfirst()*	The getfirst() method is used to return the received first value.
2	*getvalue()*	The getvalue() method is used as a dictionary get() method
3	*getlist()*	The getlist() method is used to return the list of values received.
4	*keys()*	The keys() method is used as the dictionary keys() method
5	*make_file()*	The make_file() method is used to return a writable and readable file.

CGI Program Structure in Python

Let’s understand the structure of a Python CGI Program:

There must be two sections that separate the output of the Python CGI script by a blank line.
The first section consists of the number of headers describing the client about the type of data used, and the other section consists of the data that will be displayed during the execution of the script.

Let’s have a look at the Python code given below:

print ("Content-type : text/html") 
# now enter the rest html document print ("<html>") 
print ("<head>") 
print ("<title> Welcome to CGI program </title>") 
print ("<head>") 
print ("<body>") 
print ("<h2> Hello World! This is my first CGI program. </h2>") print ("</body>") 
print ("</html>")

Now, let’s save the above file as cgi.py. Once we execute the file, we should see an output, as shown below:

Hello World! This is my first CGI program.

The above program is a simple Python script that writes the output to STDOUT file that is on-screen.

Understanding the HTTP Header

There are various HTTP headers defined that are frequently used in the CGI programs. Some of them are listed below:

S. No.	HTTP Header	Description
1	*Content-type*	The Content-type is a MIME string used for defining the file format that is being returned.
2	*Content-length: N*	The Content-length works as the information used for reporting the estimated time for downloading a file.
3	*Expires: Date*	The Expires: Date is used for displaying the valid date information
4	*Last-modified: Date*	The Last-modified: Date is used to show the resource’s last modification date
5	*Location: URL*	The Location: URL is used to display the URL returned by the server.
6	*Set-Cookies: String*	The Set-Cookies: String is used for setting the cooking with help of a string

The CGI Environment Variables

There are some variables predefined in the CGI environment alongside the HTML syntax. Some of them are listed in the following table:

S. No.	Environment Variables	Description
1	*CONTENT_TYPE*	The CONTENT_TYPE variable is used to describe the type and data of the content.
2	*CONTENT_LENGTH*	The CONTENT_LENGTH variable is used to define the query or information length.
3	*HTTP_COOKIE*	The HTTP_COOKIE variable is used to return the cookie set by the user in the current session.
4	*HTTP_USER_AGENT*	The HTTP_USER_AGENT variable is used for displaying the browser’s type currently being used by the user.
5	*REMOTE_HOST*	The REMOTE_HOST variable is used for describing the Host-name of the user.
6	*PATH_INFO*	The PATH_INFO variable is used for describing the CGI script path.
7	*REMOTE_ADDR*	The REMOTE_ADDR variable is used for defining the IP address of the visitor.
8	*REQUEST_METHOD*	The REQUEST_METHOD variable is used for requests with the help of the GET or POST method.

How to Debug CGI Scripts?

Before we start debugging CGI Scripts, we need to check the error in trivial installation. It often occurs while installing the CGI script. It is recommended to follow the installation instructions and install a replica of the module file – cgi.py as a CGI script.

Then, the test() function can be used from the script. We can write the following code using a single statement

cgi.test()

Pros and Cons of CGI Programming

Some Pros of CGI Programming:

There are numerous pros of using CGI programming. Some of them are as follows:

The CGI programs are multi-lingual. These programs can be used with any programming language.
The CGI programs are portable and can work on almost any web-server.
The CGI programs are quite scalable and can perform any task, whether it’s simple or complex.
The CGIs take lesser time in processing requests.
The CGIs can be used in development; they can reduce the cost of developments and maintenance, making it profitable.
The CGIs can be used for increasing the dynamic communication in web applications.

Some Cons of CGI Programming:

There are a few cons of using CGI programming. Some of them are as follows:

The CGI programs are pretty much complex, making it harder to debug.
While initiating the program, the interpreter has to appraise a CGI script in every initiation. As an output, it creates a lot of traffic because of multiple requests from the client-server’s side.
The CGI programs are fairly susceptible, as they are mostly free and easily available with no server security.
CGI utilizes a lot of time in processing.
The data doesn’t store in the cache memory during the loading of the page.
CGIs have huge extensive codebases, mostly in Perl.