UCLA CS 111 Operating System Principles, Lecture 5 Scribe Notes
Instructor:
Professor Paul Eggert
Notes:Ashwin Ramesh,Lohith Nagaraja
Table Of contents

  1. Overview
  2. Orthoganality
  3. Orthoganality in process level system calls
  4. Access to devices Input and output
  5. Process Table And File descriptors


  6. 1. Overview

    In this class, the instructor provided an overview of process and file access APIs ,while stressing the need and limitations with respect to orthoganality.The lecture also covered concepts about Process table and working with file descriptors.

    2. Orthoganality

    Consider a 3D geometric space which would be modeled using 3 coordinates. This can be represented using the following diagram.
    3D Coordinate system
    As we could figure out, choice of a coordinate value in any axis is independent of how we choose values in other coordinates. In this way we limit the propagation of effects in either dimensions due to our choice in a particular dimension. We would next see how this concept can be applied to applying the concept of orthogonality in operating system design.

    3. Orthoganality in process level system calls.

    Consider the system call to create a process fork(). A fork system call is adjudged to be successful, if the properties of the created child process is same as the properties of the parent process except the following properties:
    1. Process id A child process gets a new process id.
    2. Parent process id - Parent and child process have different values of parent process ids.
    3. Accumulated execution time The execution time differs with respect to the two processes.
    4. Pending locks.
    5. File locks Only one process parent or child can own a file at a given point of time.
    6. File descriptors Parent and a child process do share file descriptions, but they have own file descriptors.

    The overall aim of the fork() system call is to reduce the number of properties which differ between a child and a parent process. This is to maintain the orthogonality to minimize changes.
    Continuing our discussion, let us see another important system call used execvp(). On analysis of the working of this system call, we could adjudge it as an inverse to the fork() system call. This judgement is rough and not exact.
    This could be supported based on the following facts:
    • Execvp retains the all the parameters which differentiated parent and child process in the fork() system call.
    • It changes the other things, which were maintained the same during the course of the fork() system call.This includes
      1. program
      2. stack and heap memory sections.
      3. Instruction pointer.
    Based on these observations, we could once again visit the concept of independent axes phenomenon in the orthoganality graph with respect to fork and exec system call. Graphically this can be projected as follows
    3D Coordinate system
    The independence concept can be briefly described as follows. We could choose any values of the 2 points and still make it work. From the point of an operating system, this would mean choose any no of system calls with respect to fork and execvp and still make these calls work independent of the no of times the other system call has been executed.

    At this point it would be logical to wonder why not we have a single system call to perform both the operations of fork and exec system calls. As an extension of the discussion, a point was also made to the fact that Microsoft operating systems do not have fork and exec system calls. Rather they have a set of process waiting to be woken up.

    The 2 system calls can indeed be combined into a single system call posix_spawnvp. (fork+exec operation is often referred to as spawn).The function definition is as follows:

    int posix_spawn_vp(
            pid_t *restrict pid,        //pointer to the process id of the process it spawns for.
            char const *restrict file,        //character pointer for name of the file used to perform the execvp function.
            posix_spawnattr_t const *att,       //specify set of actions which define the process start up behavior.
            char * const restrict argv,       //Arguments to be passed to the subsidiary program execvp.
           char * const restrict envp        // Environment variables.
    );

    Note: The “restrict” keyword represents annotation of c level. The caller when it uses the restrict keyword,it “promises” not to point to other storage location which are already pointed by other pointers.

    In a linux operating system posix_spawn_vp system call is provided as a library function.
    In a windows operating system posix_spawn_vp system call is provided as a low level windows primitive.

    Though the posix_spawn_vp combines the features of fork and execvp system calls, it has inherent disadvantages.Combining 2 orthogonal system calls into a single system call abolishes the advantage of orthogonality. This would mess up the system design and is a classic example of nonorthogonality. The fact that this system call is capable of doing so many things causes its downfall!

    4. Access to devices Input and output.

    Before we dive into the details, let us see the qualitative difference accessing an I/O device and the CPU.

    The I/O devices are generally much slower compared to the CPU. Due to this, it is acceptable to have some overhead in the CPU operation.Also access to I/O devices tends to cause more problems compared to Access to CPU. Some of the factors which drive this issue are as follows:
    • Robustness of the I/O device is major issue.
    • It is less likely to have requirement for applications, which would want to talk to I/O devices directly.
    • The I/O devices differ to great extent with respect to properties such as speed and storage.
    • The I/O devices are all over the map. This would demand more abstraction in the operating system.
    We also have to consider the major dichotomy of these I/O devices. We have to deal with evanescent devices such as network cards, key boards and also random access devices such as disks.On a broader level, they can be categorized as data stream devices(Sequential access) and random access devices.

    The difference between these 2 categories can be summarized as follows:

    Sequential Access devices
    Random Access devices
    Facilitates spontaneous data generation only. Facilitates request response protocol based data generation.
    Virtually can support infinite amount of storage. Limited storage capacity.

    Ideally we would want have a set of system call API’s to access all I/O devices. However this is a tough ask, since we have to take into consideration the inconsistent behavioral patterns of devices as described just now.
    The following set of APIs can be used with both sets of devices:


    ssize_t read(         // Opens file with descriptor fd, reads count no of bytes in to the buffer specified by the character pointer.
            int fd,        //file descriptor.
            void *buffer,        // address of a buffer to read into.
            size_t count,       //No of bytes to read.Buffers minimum size.
    );




    ssize_t write(         // write count no of bytes from the input buffer to the file with descriptor fd
            int fd,        //file descriptor.
            void *buffer,        // address of a buffer to write from.
            size_t count,       //No of bytes to write.
    );



    The following API works only random access devices.

    ssize_t lseek(         // move the read/write file offset for the file specified using the descriptor
            int fd,        //file descriptor.
            off_t offset,        // offset value.
            int whence,       // If SEEK_SET,then file offset = offset bytes.If SEEK_CUR, then file offset = current location + offset. If SEEK_END,then file offset = file size + offset
    );



    Since this API works only on random devices, the principle of orthogonality is violated.There is an API available to combine the read and lseek system call. It is pread which is defined as follows:

    ssize_t pread(         //
            int fd,        //file descriptor.
            void *buf,        // input buffer pointer.
            size_t nbyte,       // Number of bytes
            off_t offset,        // offset value.
    );


    Now let us compare the set of APIs used to work on processes (fork,exec,exit) to the set of APIs used to work on I/O devices(read,write,lseek).

    With respect to the process related APIs fork and exec provide the createdestroy paradigm with fork() system call creating a process, whereas exec acts as a destructor.So far, we have not seen such APIs in IO related system call set. We now see the system calls which provide this feature.

    The open system call is used to create a file descriptor, which would be returned on successful creation. A negative value is returned in case of a failure.


    int open(         //
            char const *file,        //Character pointer to the file name.
            int flags,        // Specifies open mode.(O_RDONLY,O_RDWR,O_WRONLY,O_EXEC,O_SEARCH)
            ...       // variable number of arguments based on the value for flags
    );



    The close system call is used to destroy a file descriptor, which was created by the open system call.


    int close(int fd) //Specifies file descriptor to be closed


    The close system call has to be used carefully to ensure proper inputs are passed.Else it would return negative values indicating an error code.For ex, if we call a close on an invalid file descriptor such as close(-1) , the system call would return 1 which corresponds to ErrorCodeFlag EBADF.If close system call is executed multiple no of times on the same file descriptor, duplicate call would return EBADF(-1).

    5. Process Table And File descriptors.
    Process Table

    The ptable contains an array of fds. Each entry points to a file description which in turn points to a file. Each process keeps track of these files.
    Initially all the fds are closed. Among the fds 0,1,2 are special ones.(These are user conventions. It is not the responsibility of the kernel to take care of this).
    • 0->stdin
    • 1->stdout
    • 2->stderr
    When an OPEN() systemcall is issued, the first unused fd in the program is returned.If all the fds are in use, 1 is returned with errno = EMFILE (too many files open / no more fds available).

    Example Command
    cat foo >file 2>&1 //stdout points to a file and stderr points to the clone


    This can be done using the following commands:

    int fd1 = open("file1",0_WRONLY);
    int fd2 = open("file2",0_WRONLY);

    The open() system call is expensive with respect to performance.Instead we can use another system call to duplicate this file descriptor

    int fd2 = dup(fd1);

    Based on our observations we can suggest that What fork() is to process, dup() is to fd. (dup:fd :: fork : process)

    Now we have 2 file descriptors pointing to the same file. Here there are duplicate fds which share a pointer, but they dont allow overwriting.Note that both duplicated file descriptor points to the same file description as the one from which it was duplicated.This is undesirable.

    To fix this issue: Execute the following commands.
    close(0);
    close(1);
    close(2);
    open("/deb/null",O_RDONLY); //returns 0
    fd2=dup(fd) // we get 1

    But on the downside, this works if we are doing only this and nothing else. With respect to performance keep in mind, open("/dev/null") slower than open("/").
    Hence use dup2(fd,fd2) If fd2 is open, close it.(fd2 is cloned to fd).

    int fd=open("file",O_WRONLY)
    dup2(fd,1)
    dup2(fd,2)
    close(fd)

    We still have a couple of problems with this approach:
    1. check return status for dup2,which can fail if no fd
    2. if fd=1, dup2(1,1) no operation. So close(fd) will fail.To overcome this we have to add an if condition if(fd!=1 && fd!=2)
    There are several things which could go wrong with file descriptors.This are listed below:
    1. Using fd that is closed fails returns errno = EBADF
    2. I/o error due to disk error/network card error return errno = EIO
    3. Failure to close a file leads to file descriptor leak similar to memory leak
    4. Trying to read from Keyboard, but the user has not yet typed anything and no data yet.Read hangs OR returns an error
    5. Trying to write and the device is not yet ready
    6. Trying to read from file but End Of File is reached.It returns zero(EOF)
    7. Reading a file during which someone deletes a file.In Unix,If you have a file open and it is removed.But it is unaffected. I/O is still possible.
    Example Program Sort huge data which exceeds the capacity of RAM.

    int fd = open("/tmp/sort",O_RDWR|O_CREATE|O_TRUNC,0666);//Create a temp tile
    if(fd<0) error();
    if(unlink("/tmp/sort")!=0) error();//if opened again, it wont affect our file
    char name[100];
    long pid = getpid();
    sprintf(name,"/tmp/sort %ld",pid);//no collision between processes
    if (fd<0) error();
    compute(fd);
    if(close(fd)!=0) error();
    if(unlink(name)!=0) error();

    There is a problem associated with last 2 lines of this program.If the program is not reliable it dumps core and crashes.It leaves junk file in /tmp.Suppose if multiple users use files with similar names.This would cause an error.To solve, check if file already exists. If so, use a different name.

    char buf[100];
    generate_random_file_name(buf);
    while{if(stat(buf,&st) == 0) } ;
    int fd = open(buf,O_RDWR | O_CREAT | O_TRUNC,0666);

    This might lead to Race condition.Hence to solve this, a lock mechanism is used by using O_EXCL flag.

    do {
    generate_random_file_name(buf);
    } while((fd = open(buf,O_RDWR | O_CREAT | O_TRUNC | O_EXCL,0666)<0 && errno == EEXIST);

    Here 0600 permission can be used to allow only read access or 0400 can be used to allow only the user to access.