Trial exam: Program your own HTTP server

Introduction

In this assignment you are supposed to program you own HTTP server, also known as a web server.

Your will implement an HTTP server which implements a down scaled version of the original HTTP/1.0 protocol http://www.faqs.org/rfcs/rfc1945.html

However, your HTTP server should (although simple) work with ordinary browsers like Internet Explorer, Google Chrome, Mozilla Firefox, Opera, etc.

The assignment is divided into a number of small steps. It is important that you try to complete step N before proceeding to step N + 1 - at least for the first steps.

Note that you probably solved some of steps already in the socket programming exercises. These projects could be used as a starting point.

Deliverables

• Your C# solution
• No Word processing documents, please!

Testing and documentation

All code must be properly tested. You are free to write the test before or after the code. However, it is important that you write the test code during each step.

All code must be properly documented with /// comments.

Version control

You can use Git with a remote repository for version control.

The repository must be hosted on GitHub https://github.com/

Rules

1. Commit often
2. Push to GitHub at least after completing each of the steps below + every day, at the end of the day

Step 1: Single threaded server doing GET

The first version of your HTTP server will be a very simple single threaded server: There will only be one thread taking care of everything:

• accepting request
• handling requests
• sending responses

The server must listen on some port like 80, 8080, 8888 or the like.

All TCP servers have the same basic structure. This means that you can start looking at a simple TCP server (like Echo server) that you made in the SODP course.

When sending the response, this must follow the format of a HTTP-response see 'Computer networks' pp. 131-133.

Step 1a: Static content in the response

The first version of the server should send back some static content (always return the same text, like "Hello world"). This version should no attempt to open a file, etc. In this version you do not need to read the request.

Your server must send a HTTP/1.0 response, not HTTP/1.1.

Useful C# classes

• System.Net.Sockets.TcpListener
• System.Net.Sockets.TcpClient
• System.IO.StreamWriter
• The property Encoding.UTF8, Encoding.ASCII, or a similar encoding

Make sure all network connections, etc. are closed properly: Use using or finally statements.

Run your server and use it from an ordinary browser.

Use Microsoft Internet Explorer to send the request. Press F12 to open a monitoring window: Find the network like icon, and press the Green record button. Send some requests, and they will be recorded, so that you can check the details later. Very useful for debugging.

Step 1b: Dynamic content in the response

In the next version the server should send some dynamic content back to the client, like "You requested /someFile.html". This version should no attempt to open a file, etc.

In this version you must read the request send from the browser to the server - but reading the first line is enough.

The first line should look like GET /someFile.html HTTP/1.1. You must extract the URI (middle) part of the request line (/someFile.html in this case) and include the URI in the response.

Useful C# API

• System.Net.Sockets.NetworkStream
• System.IO.StreamReader
• The method String.split(...), the first and simplest example is most important.

Step 1c: Sending file contents in the response

In this version the server should send the contents of the resource (read "file") from the requests URI.

In the previous version of the server you extracted the URI from the request line. Now you must define where on the servers disk to look for this file. The place to start looking is called the RootCatalog.

Some examples

• When you request a file like http://www.someserver.com/file.html the file is found in rootCatalog/file.html
• When you request a file like http://www.someserver.com/directory/file.html is found in rootCatalog/directory/file.html

Declare your root catalog like private static readonly string RootCatalog = "c:/temp";

Useful C# API

• System.IO.FileStream
• The method Stream.CopyTo(...)

Make sure the file stream is closed properly in a using or finally statement.

Step 2: Testing the server

You should unit test your server (and its individual parts) before and after each of the steps in this exercise.

Testing a server is different from the unit testing you done in the ordinary socket exercises. To test at server you send a request to the server, and then you assert something about the response:

1. Opens a client socket, connecting to the server.
2. Sends a legal HTTP request, for a file which exists on the web server
3. Reads the HTTP response
4. Make assertions on the parts of the response.

Step 3: Cleaning and documentation

Now it's time to clean and document your code:

1. Remove empty lines that does not contribute to human readability.
2. Remove lines made into comments, like // (2 slashes)....
3. Run Resharper -> Inspect -> Code Issues in Solution. Handle the problems you find relevant.
4. Long methods should be broken up into more shorter (private) methods.
   A method is considered long if it has more lines than can be shown on your screen - without scrolling.
5. Duplicate code should be factored out in private (parameterized) methods.
   
6. Write some /// (3 slashes) XML style comments for your public methods, etc.

Step 4: Multi threaded server

The next version of your HTTP server will spin off multiple threads: One request generates one new thread.

The server still accepts request and then creates a new thread to handle the request and send the response.

Advantages: If one client is slow in sending the request it does not block other clients.

Use a thread pool

Useful C# API:

• The method Task.Run(Action action)

Run the unit test to see that the server still works. VERY IMPORTANT

Cleaning + documentaiton, again

Do the cleaning and documentation again

Step 5: Tracing

Insert Trace statements to monitor important events in the "life" of the server:

• server startup
• client connection
• request
• response
• exceptions
• etc.

Step 6: Proper status codes and reason phrase

The HTTP client (browser) needs to be informed about the status of the HTTP response. This is done using the status response header.

http://www.faqs.org/rfcs/rfc1945.html section 6.1 shows the status response headers.

Check the response status codes with your test. You may have to add more test cases.

Step 7: Content types

All browsers can show plain HTML files, but most browsers can show other kinds of files as well. Examples: GIF and JPEG files.

Some browsers can call external viewers like Adobe Acrobat Readers for PDF files.

If the browser does not know about the file type it will show a "Where to save this file?" dialog.

The server must inform the client (in the HTTP response) about the content type to make all this work. Content-Type is a header in the HTTP response. http://www.faqs.org/rfcs/rfc1945.html section 10.5

The content type can be inferred from the extension of the file name. Some examples

The default content type is application/octet-stream. Use it if no other content types are applicable, i.e. if you have a filename without extension or with an "unknown" extension.

More content types (sometimes called media types) can be found on http://www.iana.org/assignments/media-types/

• System.Collections.Generic.IDictionary 
• System.Collections.Generic.Dictionary 
• The method String.LastIndexOf(char), or the method Path.GetExtension(String)
• The method String.SubString(...)
• A static constructor might be handy ...

Make the content type related methods in a separate class (not inside the general HttpServer class). Make a method like

public static String GetContentType(String filename)

When it works you can copy the method to your server.

Check the response status code with unit testing.

Step 8: Other response headers

You might also include other headers in the response. Take a look at RFC1945, section 10.

The Content-Length header should be fairly easy ... inspiration.

Maybe some of the other response headers are just as easy?

Step 9: Graceful shutdown

Until now we've shutdown the HTTP server by typing Ctrl-C in the console window running the server. That is not a very nice way to end a program, especially not if the program needs to do some clean-up before closing.

Ideas for graceful shutdown: The client sends a special stop request like STOP or QUIT (not part of the HTTP/1.0 protocol). When the server receives this request the server must stop.

The C# keyword volatile might be handy.

Step 10: Refactoring: Make the unit test start and stop the server

In the present version the server must be started before you execute the unit test.

It would be better if the unit test started the server (in a separate thread), ran the test cases, and finally stopped the server.

Helpful Unit test fragment:

        [ClassInitialize]
        public static void StartServer(TestContext context)
        {
           ...
        }

        [ClassCleanup]
        public static void StopServer()
        {
            ...
        }         

More steps

Choose what you want to do next:

• URL encoding and decoding
  URL's need encoding and decoding, since lot of characters are not allowed in URL's: space, æ, ø, å, etc
  A client (like a browser) automatically encodes the URL before sending it. When your server receives an URL it must be decoded.
  Useful C# API: The method HttpServerUtility.UrlDecode
• Configuration file
  The configuration file could contain following information:
  • Server port
  • Root catalog
  • Shutdown port
  • Default content type
  • The default logging level
  • Show catalog content (if no welcome file)
  • Welcome files (filenames) - see some of the next step
• Welcome files
  If the request refers to a folder (not a file), the server should look in the folder for welcome files, like index.html, etc.
• Show catalog contents
  If the request refers to a folder (not a file) - and welcome file exists, the server should respond with a list of file in the folder
• Other methods than GET
  The HTTP protocol methods like GET, PUT, POST, HEAD etc. Implement some of them.