Project 1B
Compressed Network Communication

INTRODUCTION:

When an application (e.g., shell) is to be used remotely, it is usually not sufficient to simply replace local device I/O (e.g., between the user's terminal and the shell) with network I/O. Remote sessions and network protocols add options and behaviors that did not exist when the application was being used locally. To handle this additional processing without making any changes to the application, it is common to create client-side and server-side agents that handle the network communication and shield the application from the complexities of the remote access protocols. Such intermediary agents can implement valuable features (e.g., encrypting traffic for enhanced security or compressing traffic to improve the performance and reduce the cost of WAN-scale communication), completely transparently to the user and application.

In this project, you will build a multi-process telnet-like client and server. This project is a continuation of Project 1A. It can be broken up into two major steps:

RELATION TO READING AND LECTURES:

This lab will build on the process and exception discussions, but it is really about researching and exploiting APIs.

PROJECT OBJECTIVES:

DELIVERABLES:

A single tarball (.tar.gz) containing:

PROJECT DESCRIPTION:

Study the following documentation:

Passing input and output over a TCP socket

Using Project 1A's --shell option as a starting point, create a client program (lab1b-client) and a server program (lab1b-server), both of which support a mandatory --port=port# switch. (Note: you can remove the --shell option)

Your overall program design will look like the diagram above. You will have a client process, a server process, and a shell process. The first two are connected via TCP connection, and the last two are connected via pipes.

The client program

For all of this testing, the client and server will be running on the same host (localhost), but if you would like to add a --host=name parameter you should be able to access a shell session from other computers.

The server program

To send commands to the shell, you will need to start both the client and the server. On the client side, the behavior should be very similar to that of Project 1A, in that you will see your commands echoed back to the terminal as well as the output from the shell. Do not display anything (other than error messages) on the server side.

If you are not familiar with socket programming, you may find this tutorial to be useful. (Note that we are not expecting you to become expert in how compression in general or this particular form of encryption works. Your task is to understand it well enough to use it.)

Choose a random port number (greater than 1024) for your testing. Port numbers below 1024 are reserved. Be advised that listens may remain active for several seconds after the server exits. This means that you may get occasional socket-in-use errors if you restart the server before the old listen is taken down.

Because the server is started independently from the client, it will have its own stderr and stdout which go the terminal session from which it was started. If you want to implement debug output from the server, you can send it to the server's stdout (file descriptor 1).

Shut-down Order

If the client initiates the closing of the socket, it may not receive the last output from the shell. To ensure that no output is lost, shut-downs should be initiated from the server side:

  1. an exit(1) command is sent to the shell, or the server closes the write pipe to the shell.
  2. the shell exits, causing the server to receive an EOF on the read pipe from the shell.
  3. the server collects and reports the shell's termination status.
  4. the server closes the network socket to the client, and exits.
  5. the client continues to process output from the server until itreceives an error on the network socket from the server.
  6. the client restores terminal modes and exits.

After reporting the shell's termination status and closing the network socket, the server should exit. Otherwise it would tie up the socket and prevent testing new server versions. After your test is complete, the client, server, and shell should all be gone. There should be no remaining orphan processes.

The --log option

To ensure that compression is being correctly done, we will ask you to add a --log=filename option to your client. If this option is specified, all data written to or read from the server should be logged to the specified file. Prefix each log entry with SENT # bytes: or RECEIVED # bytes: as appropriate. (Note the colon and space between the word bytes and the start of the data). Each of these lines should be followed by a newline character (even if the last character of the reported string was a newline).

Before running your program in a mode that creates a log file, please use the ulimit command to ensure that your log file does not get too large, which could happen if you have a bug in your program. ulimit 10000 should be sufficient, but check out the ulimit man page for further details. Failing to limit the size of log files has filled up the HSSEAS Linux servers' /tmp directories in the past, which makes the machines hard to use for everybody, including you.

Sample log format output:

Compressed communication

The purpose of compression is to reduce the amount of space data takes up. In communications scenarios, a compressed version of data will use less bandwidth to transmit the same data than its uncompressed form would use. On the other hand, compression will require processing at the sending and receiving end before the data can be effectively used at the receiving process. Sometimes the benefits in reduced bandwidth are worth the costs of extra processing and sometimes they are not, so compression is not used by default in network communications. While compression makes data difficult to read, unlike encryption it does not provide any privacy for the data, since anyone who obtains the compressed version of the data can use standard algorithms to decompress it. Thus, compression is used for efficiency, not for privacy. When both efficiency and privacy are desired, data is first compressed, then encrypted.

In this project, you will only perform compression, without encryption. You will use a standard compression library to improve efficiency of your socket communications.

The compression algorithm you are using is a progressive one, which becomes more effective as it processes more data. As a result you may note very little reduction in the character-at-a-time commands being sent to the shell. Larger output coming back from the shell should, however, compress quite effectively.

Summary of exit codes

SUBMISSION:

Your README file must include lines of the form:

And, if slip days are allowed on this project, and you want to use some, this too must be included in the README file: If, for instance, you wanted to use two slip-days, you would add the following line: Your name, student ID, and email address should also appear as comments at the top of your Makefile and each source file.

Your tarball should have a name of the form lab1b-studentID.tar.gz. You can sanity check your submission with this test script.

We will test it on a departmental Linux server. You would be well advised to test your submission on that platform before submitting it.

GRADING:

Points for this project will be awarded:

value feature
Packaging and build (15% total)
5% un-tars expected contents
5% clean build of correct programs w/default action (no warnings)
3% Makefile has working clean and dist targets
2% reasonableness of README contents
Communication (50% total)
5% input properly echos (character at a time) from client
5% client passes (character at a time) data between server and termimal
5% server passes (character at a time) data between client and shell
5% proper translation of input CR (from keyboard) into NL for shell input
5% proper translation of NL (from) into CR NL to display
5% ^D from client closes connection, restores terminal modes, exits RC=0
5% ^C from client sends a SIGINT to shell
5% server properly logs shell termination status
5% --log= option properly reports sent data
5% --log option properly reports received data
Compression (25% total)
5% --compress reduces number of bytes sent over the wire in both directions
5% data is compressed before sending (tested with --log option)
5% data is decompressed after receiving (tested with --log option)
5% compressed commands sent to shell are properly decompressed before execution
5% compressed output sent by shell are properly decompressed before display
Code Review (10%)
10% correct library use