What is Unix ?
UNIX is one of the most important operating system in use today, perhaps even the most important. Since its invention around the beginning of the 1970s it has been an object of continual research and development. UNIX is not popular because it is the best operating system one could imagine, but because it is an extremely flexible system which is easy to extend and modify. It is an ideal platform for developing new ideas.
Much of the success of UNIX may be attributed to the rapid pace of its development (a development to which all of its users have been able to contribute) its efficiency at running programs and the many powerful tools which have been written for it over the years, such as the C programming language, make, shell, lex and yacc and many others. UNIX was written by programmers for programmers. It is popular in situations where a lot of computing power is required and for database applications, where timesharing is critical. In contrast to some operating systems, UNIX performs equally well on large scale computers (with many processors) and small computers which fit in your suitcase!
All of the basic mechanisms required of a multi-user operating system are present in UNIX. During the last few years it has become ever more popular and has formed the basis of newer, though less mature, systems like NT. One reason for this that now computers have now become powerful enough to run UNIX effectively. UNIX places burdens on the resources of a computer, since it expects to be able to run potentially many programs simultaneously.
If you are coming to UNIX from Windows or DOS you may well be used to using applications software or helpful interactive utilities to solve every problem. UNIX is not usually like this: the operating system has much greater functionality and provides the possibilities for making your own, so it is less common to find applications software which implements the same things. In UNIX you are usually asked to learn a language in order to express exactly what you want. This is much more powerful than menu systems, but it is harder to learn
UNIX has long been in the hands of academics who are used to making their own applications or writing their own programs, whereas as the Windows world has been driven by businesses who are willing to spend money on software. For that reason commercial UNIX software is often very expensive and therefore not available at this college. On the other hand, the flexibility of UNIX means that it is easy to write programs and it is possible to fetch gigabytes of free software from the Internet to suit your needs. It may not look exactly like what you are used to on your PC, but then you have to remember that UNIX users are a different kind of animal altogether
Like all operating systems, UNIX has many faults. The biggest problem for any operating system is that it evolves without being redesigned. Operating systems evolve as more and more patches and hacks are applied to solve day-to-day problems. The result is either a mess which works somehow (like UNIX) or a blank refusal to change (like DOS or the MacIntosh, prior to MacOS X, which is based on BSD UNIX). From a practical perspective, UNIX is important and successful because it is a multi-process system which
“ has an enormous functionality built in, and the capacity to adapt itself to changing technologies,
“ is relatively portable,
“ is good at sharing resources (but not so good at security),
“ has tools which are each developed to do one thing well,
“ allows these tools to be combined in every imaginable way, using pipes and channeling of data streams,
“ incorporates networking almost trivially, because all the right mechanisms are already there for providing services and sharing, building client-server pairs etc,.
“ it is very adaptable and is often used to develop new ideas because of the rich variety of tools it possesses.
UNIX has some problems: it is old, it contains a lot of rubbish which no one ever bothered to throw away. Although it develops quickly (at light speed compared to either DOS/Windows or MacIntosh) the user interface has been the slowest thing to change. UNIX is not user-friendly for beginners, it is user-friendly for advanced users: it is made for users who know about computing. It sometimes makes simple things difficult, but above all it makes things possible!
The aim of this introduction is to
“ introduce the unix system basics and user interface,
“ develop the unix philosophy of using and combining tools,
“ learn how to make new tools and write software,
“ learn how to understand existing software.
To accomplish this task, we must first learn something about the shell language (the way in which UNIX starts programs). Later we shall learn how to solve more complex problems using Perl and C. Each of these is a language which can be used to put UNIX to work. We must also learn when to use which tool, so that we do not waste time and effort. Typical uses for these different interfaces are
shell
Command line interaction, making scripts which performs simple jobs such as running programs, installing new software, simple system configuration and administration.
perl
Text interpretation, text formatting, output filters, mail robots, WWW cgi (common gateway interface) scripts in forms, password testing, simple database manipulation, simple client-server applications.
C
Nearly all of UNIX is written in C. Any program which cannot be solved quickly using shell or perl can be written in C. One advantage is that C is a compiled language and many simple errors can be caught at compile time.
Much of UNIX’s recent popularity has been a result of its networking abilities: UNIX is the backbone of the Internet. No other widely available system could keep the Internet alive today. GNU/Linux is a free/open source re-write of the UNIX operating system, which many enhancements. While GNU/Linux is not “rocket science” to computer experts, it has distilled the essence of UNIX and placed it in the hands of everyone. It runs on wrist watches and mainframe computers. Like it or loathe it, GNU/Linux is probably the most important single development in computer operating systems for many years.
Once you have mastered the UNIX interface and philosophy you will find that i) the PC and MacIntosh window environments might seem to be easy to use, but are simplistic and primitive by comparison; ii) UNIX is far from being the perfect operating system–it has a whole different set of problems and flaws.
The operating system of the future will not be UNIX or GNU/Linux as we see it today (hopefully), nor will is be DOS or MacIntosh, but one thing is for certain: it will owe a lot to the UNIX operating system and will contain many of the tools and mechanisms we shall describe below.
Flavours of Unix
UNIX is not a single operating system. It has branched out in many different directions since it was introduced by AT&T. The most important `fork()’ in its history happened early on when the university of Berkeley, California created the BSD (Berkeley Software Distribution), adding network support and the C-shell.
Here are some of the most common implementations of unix.
BSD:
Berkeley, BSD
SunOS:
Sun Microsystems, BSD/sys 5
Solaris:
Sun Microsystems, Sys 5/BSD
Ultrix:
Digital Equipment Corporation, BSD
OSF 1:
Digital Equipment Corporation, BSD/sys 5
HPUX:
Hewlett-Packard, Sys 5
AIX:
IBM, Sys 5 / BSD
IRIX:
Silicon Graphics, Sys 5
GNU/Linux:
GNU, BSD/Posix
How to use this reference
This programming guide is something between a user manual and a tutorial. The information contained here should be sufficient to get you started with the unix system, but it is far from complete.
To use this programming guide, you will need to work through the basics from each chapter. You will find that there is much more information here than you need straight away, so try not to be overwhelmed by the amount of material. Use the contents and the indices at the back to find the information you need. If you are following a one-semester UNIX course, you should probably concentrate on the following:
“ The remainder of this introduction
“ The detailed knowledge of the Bash shell
“ A detailed knowledge of Perl, guided by chapter 6. This chapter provides pointers on how to get started in perl. It is not a substitute for the perl book.
“ A sound appreciation of chapter 8 on network programming.
The only way to learn UNIX is to sit down and try it. As with any new thing, it is a pain to get started, but once you are started, you will probably come to agree that UNIX contains a wealth of possibilities, perhaps more than you had ever though was possible or useful!
One of the advantages of the UNIX system is that the entire UNIX manual is available on-line. You should get used to looking for information in the online manual pages. For instance, suppose you do not remember how to create a new directory, you could do the following:
nexus% man -k dir

dir ls (1) – list contents of directories
dirname dirname (1) – strip non-directory suffix from file name
dirs bash (1) – bash built-in commands, see bash(1)
find find (1) – search for files in a directory hierarchy
ls ls (1) – list contents of directories
mkdir mkdir (1) – make directories
pwd pwd (1) – print name of current/working directory
rmdir rmdir (1) – remove empty directories
The `man -k’ command looks for a keyword in the manual and lists all the references it finds. The command `apropos’ is completely equivalent to `man -k’. Having discovered that the command to create a directory is `mkdir’ you can now look up the specific manual page on `mkdir’ to find out how to use it:
man mkdir
Some but no all of the UNIX commands also have a help option which is activated with the `-h’ or `–help’ command-line option.
dax% mkdir –help
Usage: mkdir [OPTION] DIRECTORY…

-p, –parents no error if existing, make parent directories as needed
-m, –mode=MODE set permission mode (as in chmod), not 0777 – umask
–help display this help and exit
–version output version information and exit
dax%

Never -Do’s in UNIX
There are some things that you should never do in UNIX. Some of these will cause you more serious problems than others. You can make your own list as you discover more.
“ You should NEVER EVER switch off the power on a UNIX computer unless you know what you are doing. A UNIX machine is not like a PC running DOS. Even when you are not doing anything, the system is working in the background. If you switch off the power, you could interrupt the system while it is writing to the disk drive and destroy your disk. You must also remember that several users might be using the system even though you cannot see them: they do not have to be sitting at the machine, they could be logged in over the network. If you switch off the power, you might ruin their valuable work.
“ Once you have deleted a UNIX file using rm it is impossible to recover it! Don’t use wildcards with rm without thinking quite carefully about what you are doing! It has happened to very many users throughout the history of UNIX that one tries to type
“ rm *~
but instead, by a slip of the hand, one writes
rm * ~
UNIX then takes these wildcards in turn, so that the first command is rm * which deletes all of your files! BE CAREFUL!
“ Don’t ever call a program or an important file `core’. Many scripts go around deleting files called `core’ because the, when a program crashes, UNIX dumps the entire kernel image to a file called `core’ and these files use up a lot of disk space. If you call a file `core’ it might get deleted!
“ Don’t call test programs test. There is a UNIX command which is already called test and chances are that when you try to run your program you will start the UNIX command instead. This can cause a lot of confusion because the UNIX command doesn’t seem to do very much at all!
What you should know before starting
One library: several interfaces
The core of unix is the library of functions (written in C) which access the system. Everything you do on a unix system goes through this set of functions. However, you can choose your own interface to these library functions. UNIX has very many different interfaces to its libraries in the form of languages and command interpreters.
You can use the functions directly in C, or you can use command programs like `ls’, `cd’ etc. These functions just provide a simple user interface to the C calls. You can also use a variety of `script’ languages: C-shell, Bourne shell, Perl, Tcl, scheme. You choose the interface which solves your problem most easily.
UNIX commands are files
With the exception of a few simple commands which are built into the command interpreter (shell), all unix commands and programs consist of executable files. In other words, there is a separate executable file for each command. This makes it extremely simple to add new commands to the system. One simply makes a program with the desired name and places it in the appropriate directory.
UNIX commands live in special directories (usually called bin for binary files). The location of these directories is recorded in a variable called path or PATH which is used by the system to search for binaries. We shall return to this in more detail in later chapters.
Kernel and Shell
Since users cannot command the kernel directly, UNIX has a command language known as the shell. The word shell implies a layer around the kernel. A shell is a user interface, or command interpreter.
There are two main versions of the shell, plus a number of enhancements.
/bin/sh
The Bourne Shell. The shell is most often used for writing system scripts. It is part of the original unix system.
/bin/csh
The C-shell. This was added to unix by the Berkeley workers. The commands and syntax resemble C code. C-shell is better suited for interactive work than the Bourne shell.
The program tcsh is a public-domain enhancement of the csh and is in common use. Two improved versions of the Bourne shell also exist: ksh, the Korn shell and bash, the Bourne-again shell.
Although the shells are mainly tools for typing in commands (which are executable files to be loaded and run), they contain features such as aliases, a command history, wildcard-expansions and job control functions which provide a comfortable user environment.
The role of C
Most of the unix kernel and daemons are written in the C programming language (1). Calls to the kernel and to services are made through functions in the standard C library. The commands like chmod, mkdir and cd are all C functions. The binary files of the same name /bin/chmod, /bin/mkdir etc. are just trivial “wrapper” programs for these C functions.
Until Solaris 2, the C compiler was a standard part of the UNIX operating system, thus C is the most natural language to program in in a UNIX environment. Some tools are provided for C programmers:
dbx
A symbolic debugger. Also gdb, xxgdb ddd.
make
A development tool for compiling large programs.
lex
A `lexer’. A program which generates C code to recognize words of text.
yacc
A `parser’. This is a tool which generates C code for checking the syntax of groups of textual words.
rpcgen
A protocol compiler which generates C code from a higher level language, for programming RPC applications.
Stdin, stdout, stderr
UNIX has three logical streams or files which are always open and are available to any program.
stdin
The standard input – file descriptor 0.
stdout
The standard output – file descriptor 1.
stderr
The standard error – file descriptor 2.
The names are a part of the C language and are defined as pointers of type FILE.
#include

/* FILE *stdin, *stdout, *stderr; */

fprintf(stderr,”This is an error message!\n”);

The names are `logical’ in the sense that they do not refer to a particular device, or a particular place for information to come from or go. Their role is analogous to the `.’ and `..’ directories in the filesystem. Programs can write to these files without worrying about where the information comes from or goes to. The user can personally define these places by redirecting standard I/O. This is discussed in the next chapter.
A separate stream is kept for error messages so that error output does not get mixed up with a program’s intended output.
The superuser (root)
When logged onto a UNIX system directly, the user whose name is root has unlimited access to the files on the system. root can also become any other user without having to give a password. root is reserved for the system administrator or trusted users.
Certain commands are forbidden to normal users. For example, a regular user should not be able to halt the system, or change the ownership of files (see next paragraph). These things are reserved for the root or superuser.
In a networked environment, root has no automatic authority on remote machines. This is to prevent the system administrator of one machine in Canada from being able to edit files on another in China. He or she must log in directly and supply a password in order to gain access privileges. On a network where files are often accessible in principle to anyone, the username root gets mapped to the user nobody, who has no rights at all.
The file hierarchy
UNIX has a hierarchical filesystem, which makes use of directories and sub-directories to form a tree. The root of the tree is called the root filesystem or `/’. Although the details of where every file is located differ for different versions of unix, some basic features are the same. The main sub-directories of the root directory together with the most important file are shown in the figure. Their contents are as follows.
`/bin’
Executable (binary) programs. On most systems this is a separate directory to /usr/bin. In SunOS, this is a pointer (link) to /usr/bin.
`/etc’
Miscellaneous programs and configuration files. This directory has become very messy over the history of UNIX and has become a dumping ground for almost anything. Recent versions of unix have begun to tidy up this directory by creating subdirectories `/etc/mail’, `/etc/services’ etc!
`/usr’
This contains the main meat of UNIX. This is where application software lives, together with all of the basic libraries used by the OS.
`/usr/bin’
More executables from the OS.
`/usr/local’
This is where users’ custom software is normally added.
`/sbin’
A special area for statically linked system binaries. They are placed here to distinguish commands used solely by the system administrator from user commands and so that they lie on the system root partition where they are guaranteed to be accessible during booting.
`/sys’
This holds the configuration data which go to build the system kernel. (See below.)
`/export’
Network servers only use this. This contains the disk space set aside for client machines which do not have their own disks. It is like a `virtual disk’ for diskless clients.
`/dev, /devices’
A place where all the `logical devices’ are collected. These are called `device nodes’ in unix and are created by mknod. Logical devices are UNIX’s official entry points for writing to devices. For instance, /dev/console is a route to the system console, while /dev/kmem is a route for reading kernel memory. Device nodes enable devices to be treated as though they were files.
`/home’
(Called /users on some systems.) Each user has a separate login directory where files can be kept. These are normally stored under /home by some convention decided by the system administrator.
`/var’
System 5 and mixed systems have a separate directory for spooling. Under old BSD systems, /usr/spool contains spool queues and system data. /var/spool and /var/adm etc are used for holding queues and system log files.
`/vmunix’
This is the program code for the unix kernel (see below). On HPUX systems with file is called `hp-ux’. On linux it is called `linux’.
`/kernel’
On newer systems the kernel is built up from a number of modules which are placed in this directory.
Every unix directory contains two `virtual’ directories marked by a single dot and two dots.
ls -a
. ..
The single dot represents the directory one is already in (the current directory). The double dots mean the directory one level up the tree from the current location. Thus, if one writes
cd /usr/local
cd ..
the final directory is /usr. The single dot is very useful in C programming if one wishes to read `the current directory’. Since this is always called `.’ there is no need to keep track of what the current directory really is.
`.’ and `..’ are `hard links’ to the true directories.
Symbolic links
A symbolic link is a pointer or an alias to another file. The command
ln -s fromfile /other/directory/tolink
makes the file fromfile appear to exist at /other/directory/tolink simultaneously. The file is not copied, it merely appears to be a part of the file tree in two places. Symbolic links can be made to both files and directories.
A symbolic link is just a small file which contains the name of the real file one is interested in. It cannot be opened like an ordinary file, but may be read with the C call readlink() See section lstat and readlink. If we remove the file a symbolic link points to, the link remains — it just points nowhere.
Hard links
A hard link is a duplicate inode in the filesystem which is in every way equivalent to the original file inode. If a file is pointed to by a hard link, it cannot be removed until the link is removed. If a file has @math{n} hard links — all of them must be removed before the file can be removed. The number of hard links to a file is stored in the filesystem index node for the file.
Getting started
If you have never met unix, or another multiuser system before, then you might find the idea daunting. There are several things you should know.
Logging in
Each time you use unix you must log on to the system by typing a username and a password. Your login name is sometimes called an `account’ because some unix systems implement strict quotas for computer resources which have to be paid for with real money(2).

login: mark
password:

Once you have typed in your password, you are `logged on’. What happens then depends on what kind of system you are logged onto and how. If you have a colour monitor and keyboard in front of you, with a graphical user interface, you will see a number of windows appear, perhaps a menu bar. You then use a mouse and keyboard just like any other system.
This is not the only way to log onto unix. You can also log in remotely, from another machine, using the Secure Shell ssh program ( this replaces the now antiquated telnet and rlogin programs). If you use these programs, you will normally only get a text or command line interface (though graphical interfaces can easily be arranged).
Once you have logged in, a short message will be printed (called Message of the Day or motd) and you will see the C-shell prompt: the name of the host you are logged onto followed by a percent sign, e.g.

Linux cube 2.2.19pre13 #2 Mon Feb 26 15:53:31 MET 2001 i686 unknown

This is GNU/Linux – send problems to help@example.org

10:44pm up 8 days, 13:34, 3 users, load average: 0.08, 0.02, 0.01

There are 480 messages in your incoming mailbox.

Remember that every UNIX machine is a separate entity: it is not like logging onto a PC system where you log onto the `network’ i.e. the PC file server. Every UNIX machine is a server, or a client — more correctly a “peer” (equal partner). The network, in unix-land, has lots of players.
The first thing you should do once you have logged on is to set a reliable password. A poor password might be okay on a PC which is not attached to a large network, but once you are attached to the Internet, you have to remember that the whole world will be trying to crack your password. Don’t think that no one will bother: some people really have nothing better to do. A password should not contain any word that could be in a list of words (in any language), or be a simple concatenation of a word and a number (e.g. mark123). It takes seconds to crack such a password. Choose instead something which is easy to remember. Feel free to use the PIN number from your bankers card in your password! This will leave you with fewer things to remember. e.g. Ma9876rk). Passwords can be up to eight characters long.
Some sites allow you to change your password anywhere. Other sites require you to log onto a special machine to change your password:

dax%
dax% passwd
Change your password on host nexus
You cannot change it here
dax% rlogin nexus
password: ******

nexus% passwd
Changing password for mark
Enter login password: ********
Enter new password: ********
Reenter new passwd: ********

You will be prompted for your old password and your new password twice. If your network is large, it might take the system up to an hour or two to register the change in your password, so don’t forget the old one right away!
Mouse buttons
UNIX has three mouse buttons. On some PC’s running GNU/Linux or some other PC unix, there are only two, but the middle mouse button can be simulated by pressing both mouse buttons simultaneously. The mouse buttons have the following general functions. They may also have additional functions in special software.
index finger
This is used to select and click on objects. It is also used to mark out areas and copy by dragging. This is the button you normally use.
middle finger
Used to pull down menus. It is also used to paste a marked area somewhere at the mouse position.
outer finger
Pulls down menus.
On a left-handed system right and left are reversed.
E-mail
Reading electronic mail on unix is just like any other system, but there are many programs to choose from. There are very old programs from the seventies such as
mail
and there are fully graphical mail programs such as
tkrat
mailtool
Choose the program you like best. Not all of the programs support modern multimedia extensions because of their age. Some programs like tkrat have immediate mail notification alerts. To start a mail program you just type its name. If you have an icon-bar, you can click on the mail-icon.
Simple commands
Inexperienced computer users often prefer to use file-manager programs to avoid typing anything. With a mouse you can click your way through directories and files without having to type anything (e.g. the kfm or tkdesk programs). More experienced users generally find this to be slow and tedious after a while and prefer to use written commands. UNIX has many short cuts and keyboard features which make typed commands extremely fast and much more powerful than use of the mouse.
Today the CDE, KDE and GNOME projects are the most important efforts to write graphical user interfaces for computers. The CDE (Common Desktop Environment) is a commercial program developed by IBM, Hewlett-Packard, Sun Microsystems and many other vendors. KDE (a German effort, a pun on CDE) and GNOME are free software window systems which have taken windowing to the next level. While they have borrowed and stolen many ideas from Windows’ innovative Windows 95 user interface, they have taken windowing beyond this.
If you come from a Windows environment, the UNIX commands can be a little strange. It is a different way of thinking: using language to ask for exactly what you want, instead of pointing to a menu of limited choices. It is also a strange language. Because they stem from an era when keyboards had to be hit with hammer force, and machines were very slow, the UNIX command names are as short as possible, so they seem pretty cryptic. Some familiar ones which DOS borrowed from UNIX include,
cd
mkdir
which change to a new directory and make a new directory respectively. To list the files in the current directory you use,
ls
To rename a file, you `move’ it:
mv old-name new-name
Text editing and word processing
Text editing is one of the things which people spend most time doing on any computer. It is important to distinguish text editing from word processing. On a PC or MacIntosh, you are perhaps used to Word or WordPerfect for writing documents.
UNIX has a Word-like program called lyx, and even several Office clones (e.g. Star Office soffice), but for the most part UNIX users do not use word processors. It is more common in the UNIX community to write all documents, regardless of whether they are letters, books or computer programs, using a non-formatting text editor. (UNIX word processors like Framemaker do exist, but they are very expensive. A version of MS-Word also exists for some unices.) Once you have written a document in a normal text editor, you call up a text formatter to make it pretty. You might think this strange, but the truth of the matter is that this two-stage process gives you the most power and flexibility–and that is what most UNIX folks like.
For writing programs, or anything else, you edit a file by typing:

emacs myfile

emacs is one of dozens of text-editors. It is not the simplest or most intuitive, but it is the most powerful and if you are going to spend time learning an editor, it wouldn’t do any harm to make it this one. You could also click on emacs’ icon if you are relying on a window system. Emacs is almost certainly the most powerful text editor that exists on any system. It is not a word-processor, it is not for formatting printed documents, but it can be linked to almost any other program in order to format and print text. It contains a powerful programming language and has many intelligent features. We shall not go into the details of document formatting in this book, but only mention that programs like troff and Tex or Latex are used for this purpose to obtain typeset-quality printing. Text formatting is an area where UNIX folks do things differently to PC folks.
The login environment
UNIX began as a timesharing mainframe system in the seventies, when the only terminals available were text based teletype terminals or tty-s. Later, the Massachusetts Institute of Technology (MIT) developed the X-windows interface which is now a standard across UNIX platforms. Because of this history, the X-window system works as a front end to the standard UNIX shell and interface, so to understand the user environment we must first understand the shell.
Shells
A shell is a command interpreter. In the early days of UNIX, a shell was the only way of issuing commands to the system. Nowadays many window-based application programs provide menus and buttons to perform simple commands, but the UNIX shell remains the most powerful and flexible way of interacting with the system.
After logging in and entering a password, the UNIX process init starts a shell for the user logging in. UNIX has several different kinds of shell to choose from, so that each user can pick his/her favourite command interface. The type of shell which the system starts at login is determined by the user’s entry in the passwd database. On most systems, the standard login shell is a variant of the C-shell.
Shells provide facilities and commands which
“ Start and stop processes (programs)
“ Allow two processes to communicate through a pipe
“ Allow the user to redirect the flow of input or output
“ Allow simple command line editing and command history
“ Define aliases to frequently used commands
“ Define global “environment” variables which are used to configure the default behaviour of a variety of programs. These lie in an “associated array” for each process and may be seen with the `env’ command. Environment variables are inherited by all processes which are started from a shell.
“ Provide wildcard expansion (joker notation) of filenames using `*,?,[]‘
“ Provide a simple script language, with tests and loops, so that users can combine system programs to create new programs of their own.
“ Change and remember the location of the current working directory, or location within the file hierarchy.
The shell does not contain any more specific functions–all other commands, such as programs which list files or create directories etc., are executable programs which are independent of the shell. When you type `ls’, the shell looks for the executable file called `ls’ in a special list of directories called the command path (which is contained in the environment variable $PATH) and attempts to start this program. This allows such programs to be developed and replaced independently of the actual command interpreter.
Each shell which is started can be customized and configured by editing a setup file. For the Bash shell this file is called `.bashrc’, and for the C-shell and its variants it is called `.profile’. (Note that files which begin with leading dots are not normally visible with the `ls’ command. Use `ls -a’ to view these.) Any commands which are placed in these files are interpreted by the shell before the first command prompt is issued. These files are typically used to define a command search path and terminal characteristics.
On each new command line you can use the cursor keys to edit the line. The up-arrow browses back through earlier commands. CTRL-a takes you to the start of the line. CTRL-e takes you to the end of the line. The TAB can be used to save typing with the `completion’ facility See section Command/filename completion.
Shell commands generally
Shell commands are commands like cp, mv, passwd, cat, more, less, cc, grep, ps etc..
One thing you can always bet on with Unix is that there is not just one way of doing things — there are so many standards, that there is often a bewildering array to choose from. UNIX has two main command shells. They are called sh (Bourne Shell) and csh C-shell. Their modern implementations are called Bash (Bourne Again Shell) and tcsh (T-C shell).
Very few commands are actually built into the shell command line interpreter, in the same way that they are built into DOS. Rather commands are programs which exist as actual program files. When we type a command, the shell searches for a program with the same name and tries to execute it. This is very flexible, since anyone is free to write their own programs and therefore extend the command language of the system. The file must be executable, or a Command not found error will result. To see what actually happens when you type a command like gcc, try typing the following into a GNU/Linux system: (you can type this exactly as shown into a Bash shell)
cube$ IFS=:

cube$ for dir in $PATH # for every directory in the list path
>do
> if [ -x $dir/gcc ] # if the file is executable
> then
> echo Found $dir/gcc # Print message found!
> break # break out of loop
> else
> echo Searching $dir/gcc
> fi
>done
If you use C-shell (e.g. tcsh), try typing in the following C-shell commands directly into a C-shell.

nexus% foreach dir ( $path ) # for every directory in the list path
> if ( -x $dir/gcc ) then # if the file is executable
> echo Found $dir/gcc # Print message found!
> break # break out of loop
> else
> echo Searching $dir/gcc
> endif
> end
The output of these command sequences is something like this:
Searching /usr/lang/gcc
Searching /usr/openwin/bin/gcc
Searching /usr/openwin/bin/xview/gcc
Searching /physics/lib/framemaker/bin/gcc
Searching /physics/motif/bin/gcc
Searching /physics/mutils/bin/gcc
Searching /physics/common/scripts/gcc
Found /physics/bin/gcc
If you type
echo $PATH
in Bourne Shell, or
echo $path
in C-shell you will see the entire list of directories which are searched by the shell. If we had left out the `break’ command, we might have discovered that UNIX often has several programs with the same name, in different directories! For example,
/bin/mail
/usr/ucb/mail
/bin/Mail

/bin/make
/usr/local/bin/make.
Also, different versions of UNIX have different conventions for placing the commands in directories, so the path list needs to be different for different types of UNIX machine. In Bash a few basic commands like cd and kill are built into the shell (as in DOS).
You can find out which directory a command is stored in using
type
command. For example
cube$ type cd
cd is a shell builtin
cube$ type mv
mv is /bin/mv
cube$
type only searches the directories in $PATH and quits after the first match, so if there are several commands with the same name, you will only see the first of them using type.
Finally, in the C-shell the command corresponding to type is built in and called which. In Bash which is a program:
cube$ type which
which is /usr/bin/which
cube$ tcsh
cube% which which
which: shell built-in command.
Take a look at the script /usr/bin/which. It is a script written in bash.
Environment and shell variables
Environment variables are variables which the shell keeps. They are normally used to configure the behaviour of utility programs like lpr (which sends a file to the printer) and mail (which reads and sends mail) so that special options do not have to be typed in every time you run these programs.
Any program can read these variables to find out how you have configured your working environment. We shall meet these variables frequently. Here are some important variables
PATH # The search path for shell commands (bash)
TERM # The terminal type (bash and csh)
DISPLAY # X11 – the name of your display
LD_LIBRARY_PATH # Path to search for object and shared libraries
HOSTNAME # Name of this UNIX host
PRINTER # Default printer (lpr)
HOME # The path to your home directory (bash)
PS1 # The default prompt for bash

path # The search path for shell commands (csh)
term # The terminal type (csh)
prompt # The default prompt for csh
home # The path to your home directory (csh)

These variables fall into two groups. Traditionally the first group always have names in uppercase letters and are called environment variables, whereas variables in the second group have names with lowercase letters and are called shell variables– but this is only a convention. The uppercase variables are global variables, whereas the lower case variables are local variables. Local variables are not defined for programs or sub-shells started by the current shell, while global variables are inherited by all sub-shells.
The Bash-shell and the C-shell use these conventions differently and not always consistently. You will see how to define these below. For now you just have to know that you can use the command env can be used in Bash shell to see all of the defined global environment variables while set lists both the global and the local variables.
Wildcards
Sometimes you want to be able to refer to several files in one go. For instance, you might want to copy all files ending in `.c’ to a new directory. To do this one uses wildcards. Wildcards are characters like * ? which stand for any character or group of characters. In card games the joker is a `wild card’ which can be substituted for any other card. Use of wildcards is also called filename substitution in the UNIX manuals, in the sections on sh and csh.
The wildcard symbols are,
`?’
Match single character. e.g. ls /etc/rc.????
`*’
Match any number of characters. e.g. ls /etc/rc.*
`[...]‘
Match any character in a list enclosed by these brackets. e.g. ls [abc].C
Here are some examples and explanations.
`/etc/rc.????’
Match all files in /etc whose first three characters are rc. and are 7 characters long.
`*.c’
Match all files ending in `.c’ i.e. all C programs.
`*.[Cc]‘
List all files ending on `.c’ or `.C’ i.e. all C and C++ programs.
`*.[a-z]‘
Match any file ending in .a, .b, .c, … up to .z etc.
It is important to understand that the shell expands wildcards. When you type a command, the program is not invoked with an argument that contains * or ?. The shell expands the special characters first and invokes commands with the entire list of files which match the patterns. The programs never see the wildcard characters, only the list of files they stand for. To see this in action, you can type
echo /etc/rc*
which gives
/etc/rc0 /etc/rc0.d /etc/rc1 /etc/rc1.d /etc/rc2 /etc/rc2.d /etc/rc3
/etc/rc3.d /etc/rc5 /etc/rc6 /etc/rcS /etc/rcS.d
All shell commands are invoked with a command line of this form. This has an important corollary. It means that multiple renaming cannot work!
UNIX files are renamed using the mv command. In many microcomputer operating systems one can write
rename *.x *.y
which changes the file extension of all files ending in `.x’ to the same name with a `.y’ extension. This cannot work in UNIX, because the shell tries expands everything before passing the arguments to the command line.
Regular expressions
The wildcards belong to the shell. They are used for matching filenames. UNIX has a more general and widely used mechanism for matching strings, this is through regular expressions.
Regular expressions are used by the egrep utility, text editors like ed, vi and emacs and sed and awk. They are also used in the C programming language for matching input as well as in the Perl programming language and lex tokenizer. Here are some examples using the egrep command which print lines from the file /etc/rc which match certain conditions. The construction is part of egrep. Everything in between these symbols is a regular expression. Notice that special shell symbols ! * & have to be preceded with a backslash \ in order to prevent the shell from expanding them!

# Print all lines beginning with a comment #
egrep ‘(^#)’ /etc/rc

# Print all lines which DON’T begin with #
egrep ‘(^[^#])’ /etc/rc

# Print all lines beginning with e, f or g.
egrep ‘(^[efg])’ /etc/rc

# Print all lines beginning with uppercase
egrep ‘(^[A-Z])’ /etc/rc

# Print all lines NOT beginning with uppercase
egrep ‘(^[^A-Z])’ /etc/rc

# Print all lines containing ! * &
egrep ‘([\!\*\&])’ /etc/rc

# All lines containing ! * & but not starting #
egrep ‘([^#][\!\*\&])’ /etc/rc
Regular expressions are made up of the following `atoms’.
These examples assume that the file `/etc/rc’ exists. If it doesn’t exist on the machine you are using, try to find the equivalent by, for instance, replacing /etc/rc with /etc/rc* which will try to find a match beginning with the rc.
`.’
Match any single character except the end of line.
`^’
Match the beginning of a line as the first character.
`$’
Match end of line as last character.
`[..]‘
Match any character in the list between the square brackets.(see below).
`*’
Match zero or more occurrences of the preceding expression.
`+’
Match one or more occurrences of the preceding expression.
`?’
Match zero or one occurrence of the preceding expression.
You can find a complete list in the UNIX manual pages. The square brackets above are used to define a class of characters to be matched. Here are some examples,
“ If the square brackets contain a list of characters, $[a-z156]$ then a single occurrence of any character in the list will match the regular expression: in this case any lowercase letter or the numbers 1, 5 and 6.
“ If the first character in the brackets is the caret symbol `^’ then any character except those in the list will be matched.
“ Normally a dash or minus sign `-’ means a range of characters. If it is the first character after the `[' or after `[^' then it is treated literally.
Nested shell commands and "
The backwards apostrophes `...` can be used in all shells and also in the programming language Perl. When these are encountered in a string the shell tries to execute the command inside the quotes and replace the quoted expression by the result of that command. For example:

UNIX$ echo "This system's kernel type is `/usr/bin/file /boot/vmlinuz-2.2.19pre13`"
This system's kernel type is /boot/vmlinuz-2.2.19pre13: Linux kernel x86 boot executable bzImage, version 2.2.19pre13

UNIX$ for file in `ls /local/ssl/misc/*`
> do
> echo I found a config file $file
> echo Its type is `/usr/bin/file $file`
> done
I found a config file /local/ssl/misc/CA.pl
Its type is /local/ssl/misc/CA.pl: perl script text
I found a config file /local/ssl/misc/CA.sh
Its type is /local/ssl/misc/CA.sh: Bourne shell script text
I found a config file /local/ssl/misc/c_hash
Its type is /local/ssl/misc/c_hash: Bourne shell script text
I found a config file /local/ssl/misc/c_info
Its type is /local/ssl/misc/c_info: Bourne shell script text
I found a config file /local/ssl/misc/c_issuer
Its type is /local/ssl/misc/c_issuer: Bourne shell script text
I found a config file /local/ssl/misc/c_name
Its type is /local/ssl/misc/c_name: Bourne shell script text
I found a config file /local/ssl/misc/der_chop
Its type is /local/ssl/misc/der_chop: perl script text
This is how we insert the result of a shell command into a text string or variable.
UNIX command overview
Important keys
TAB
The TAB key is used by Bash and Emacs for "filename completion", i.e. when you are uncertain of the correct name of something, or simply can't be bothered to type it out, you can hit TAB to either finish off the word, or show you alternative choices. e.g. try in Bash
cube$ loadTAB
loadkeys loadmeter loadunimap
This shows the possible completions of commands which match "load". Type one more letter and TAB, and the rest will be filled in.
CTRL-A
Jump to start of line. If `screen' is active, this prefixes all control key commands for `screen' and then the normal CTRL-A is replaced by CTRL-a a.
CTRL-C
Interrupt or break key. Sends signal 15 to a process.
CTRL-D
Signifies `EOF' (end of file) or shows expansion matches in command/filename completion See section Command/filename completion.
CTRL-E
Jump to end of line.
CTRL-L
Clear screen in newer shells and in emacs. Same as `clear' in the shell.
CTRL-Z
Suspend the present process, but do not destroy it. This sends signal 18 to the process.
Alternative shells
bash
The Bourne Again shell, an improved sh.
csh
The standard C-shell.
jsh
The same as sh, with C-shell style job control.
ksh
The Korn shell, an improved sh.
sh
The original Bourne shell.
sh5
On ULTRIX systems the standard Bourne shell is quite stupid. sh5 corresponds to the normal Bourne shell on these systems.
tcsh
An improved C-shell.
zsh
An improved sh.
Window based terminal emulators
xterm
The standard X11 terminal window.
shelltool, cmdtool
Openwindows terminals from Sun Microsystems. These are not completely X11 compatible during copy/paste operations.
screen
This is not a window in itself, but allows you to emulate having several windows inside a single (say) xterm window. The user can switch between different windows and open new ones, but can only see one window at a time See section Multiple screens.
Remote shells and logins
The best way to log onto another system is to use the Secure Shell command ssh. This replaces the now obsolete commands:
rlogin
Login onto a remote UNIX system.
rsh
Open a shell on a remote system (require access rights).
telnet
Open a connection to a remove system using the telnet protocol.
These old commands are insecure andnote very flexible. The Secure Shell offers encryption, strong authentication and greater functionality. It can be used to run a single program on a remote machine, or to login on the remote machine.

cube$ ssh metaverse date
cube$ ssh metaverse
Text editors
ed
An ancient line-editor.
vi
Visual interface to ed. This is the only "standard" UNIX text editor supplied by vendors.
emacs
The most powerful UNIX editor. A fully configurable, user programmable editor which works under X11 and on tty-terminals.
xemacs
A pretty version of emacs for X11 windows.
pico
A tty-terminal only editor, comes as part of the PINE mail package.
xedit
A test X11-only editor supplied with X-windows.
textedit
A simple X11-only editor supplied by Sun Microsystems.
File handling commands
ls
List files in specified directory (like dir on other systems).
cp
Copy files.
mv
Move or rename files.
touch
Creates an empty new file if none exists, or updates date and time stamps on existing files.
rm, unlink
Remove a file or link (delete).
mkdir, rmdir
Make or remove a directory. A directory must be empty in order to be able to remove it.
cat
Concatenate or join together a number of files. The output is written to the standard output by default. Can also be used to simply print a file on screen.
lp, lpr
Line printer. Send a file to the default printer, or the printer defined in the `PRINTER' environment variable.
lpq, lpstat
Show the status of the print queue.
File browsing
more
Shows one screen full at a time. Possibility to search for a string and edit the file. This is like `type file | more' in DOS.
less
An enhanced version of more.
mc
Midnight commander, a free version of the `Norton Commander' PC utility for UNIX. (Only for non-serious UNIX users...)
kfm
A window based file manager with icons and all that nonsense.
Ownership and granting access permission
chmod
Change file access mode.
chown, chgrp
Change owner and group of a file. The GNU version of chown allows both these operations to be performed together using the syntax chown owner.group file.
acl
On newer Unices, Access control lists allow access to be granted on a per-user basis rather than by groups.
Extracting from and rebuilding files
cut
Extract a column in a table
paste
Merge several files so that each file becomes a column in a table.
sed
A batch text-editor for searching, replacing and selecting text without human intervention.
awk
A prerunner to the Perl language, for extracting and modifying textfiles.
rmcr
Strip carriage return (ASCII 13) characters from a file. Useful for converting DOS files to UNIX.
Locating files
find
Search for files from a specified directory using various criteria.
locate
Fast search in a global file database for files containing a search-string.
whereis
Look for a command and its documentation on the system.
Disk usage.
du
Show number of blocks used by a file or files.
df
Show the state of usage for one or more disk partitions.
Show other users logged on
users
Simple list of other users.
finger
Show who is logged onto this and other systems.
who
List of users logged into this system.
w
Long list of who is logged onto this system and what they are doing.
Contacting other users
write
Send a simple message to the named user, end with CTRL-D. The command `mesg n' switches off messages receipt.
talk
Interactive two-way conversation with named user.
irc
Internet relay chat. A conferencing system for realtime multi-user conversations, for addicts and losers.
Mail senders/readers
mail
The standard (old) mail interface.
Mail
Another mail interface.
elm
Electronic Mail program. Lots of functionality but poor support for multimedia.
pine
Rumours (untrue) are that pine stands for Pine is Not Elm; it actually stands for nothing at all. Improved support for multimedia but very slow and rather stupid at times. Some of the best features of elm have been removed!
mailtool
Sun's openwindows client program.
rmail
A mail interface built into the emacs editor.
netscape mail
A mail interface built into the netscape navigator.
zmail
A commercial mail package.
tkrat
A graphical mail reader which supports most MIME types, written in tcl/tk. This program has a nice feel and allows you to create a searchable database of old mail messages, but has a hopeless locking mechanism.
File transfer
ftp
The File Transfer program - copies files to/from a remote host.
ncftp
An enhanced ftp for anonymous login.
Compilers
cc
The C compiler.
CC
The C++ compiler.
gcc
The GNU C compiler.
g++
The GNU C++ compiler.
javac
A generator of Java bytecode.
java
A Java Virtual Machine.
ld
The system linker/loader.
ar
Archive library builder.
dbx
A symbolic debugger.
gdb
The GNU symbolic debugger.
xxgdb
The GNU debugger with a window driven front-end.
ddd
A motif based front-end to the gdb debugger.
Other interpreted languages
perl
Practical extraction an report language.
tcl
A perl-like language with special support for building user interfaces and command shells.
php
Personal Home Page Tools (officially "PHP: Hypertext Preprocessor"). A server-side HTML-embedded scripting language.
scheme
A lisp-like extensible scripting language from GNU.
mercury
A prolog-like language for artificial intelligence.
Processes and system statistics
ps
List system process table.
vmstat
List kernel virtual-memory statistics.
netstat
List network connections and statistics.
rpcinfo
Show rpc information.
showmount
Show clients mounting local filesystems.
System identity
uname
Display system name and operating system release.
hostname
Show the name of this host.
domainname
Show the name of the local NIS domain. Normally this is chosen to be the same as the BIND/DNS domain, but it need not be.
nslookup
Interrogate the DNS/BIND name service (hostname to IP address conversion).
Internet resources
archie, xarchie
Search the internet ftp database for files.
xrn, fnews
Read news (browser).
netscape, xmosaic
Read world wide web (WWW) (browser).
Text formatting and postscript
tex, latex
Donald Knuth's text formatting language, pronounced "tek" (the x is really a Greek "chi"). Used widely for technical publications. Compiles to dvi (device independent) file format.
texinfo
A hypertext documentation system using tex and "info" format. This is the GNU documentation system. This UNIX guide is written in texinfo!!!
xdvi
View a tex dvi file on screen.
dvips
Convert dvi format into postscript.
ghostview, ghostscript
View a postscript file on screen.
Picture editors and processors
xv
Handles, edits and processes pictures in a variety of standard graphics formats (gif, jpg, tiff etc). Use xv -quit to place a picture on your root window.
xpaint
A simple paint program.
xfig
A line drawing figure editor. Produces postscript, tex, and a variety of other output formats.
xmgr
A graphing and analysis program.
xsetroot
Load an X-bitmap image into the screen (root window) background. Small images are tiled.
Miscellaneous
date
Print the date and time.
ispell
Spelling checker.
xcalc
A graphical calculator.
dc,bc
Text-based calculators.
xclock
A clock!
ping
Send a "sonar" ping to see if another UNIX host is alive.
Terminals
In order to communicate with a user, a shell needs to have access to a terminal. UNIX was designed to work with many different kinds of terminals. Input/output commands in UNIX read and write to a virtual terminal. In reality a terminal might be a text-based Teletype terminal (called a tty for short) or a graphics based terminal; it might be 80-characters wide or it might be wider or narrower. UNIX take into account these possibility by defining a number of instances of terminals in a more or less object oriented way.
Each user's terminal has to be configured before cursor based input/output will work correctly. Normally this is done by choosing one of a number of standard terminal types a list which is supplied by the system. In practice the user defines the value of the environment variable `TERM' to an appropriate name. Typical examples are `vt100' and `xterm'. If no standard setup is found, the terminal can always be configured manually using UNIX's most cryptic and opaque of commands: `stty'.
The job of configuring terminals is much easier now that hardware is more standard. Users' terminals are usually configured centrally by the system administrator and it is seldom indeed that one ever has to choose anything other than `vt100' or `xterm'.
The X window system
Because UNIX originated before windowing technology was available, the user-interface was not designed with windowing in mind. The X window system attempts to be like a virtual machine park, running a different program in each window. Although the programs appear on one screen, they may in fact be running on UNIX systems anywhere in the world, with only the output being local to the user's display. The standard shell interface is available by running an X client application called `xterm' which is a graphical front-end to the standard UNIX textual interface.
The `xterm' program provides a virtual terminal using the X windows graphical user interface. It works in exactly the same way as a tty terminal, except that standard graphical facilities like copy and paste are available. Moreover, the user has the convenience of being able to run a different shell in every window. For example, using the `rlogin' command, it is possible to work on the local system in one window, and on another remote system in another window. The X-window environment allows one to cut and paste between windows, regardless of which host the shell runs on.
The components of the X-window system
The X11 system is based on the client-server model. You might wonder why a window system would be based on a model which was introduced for interprocess communication, or network communication. The answer is straightforward.
The designers of the X window system realized that network communication was to be the paradigm of the next generation of computer systems. They wanted to design a system of windows which would enable a user to sit at a terminal in Massachusetts and work on a machine in Tokyo -- and still be able to get high quality windows displayed on their terminal. The aim of X windows from the beginning is to create a distributed window environment.
When I log onto my friend's Hewlett Packard workstation to use the text editor (because I don't like the one on my EUNUCHS workstation) I want it to work correctly on my screen, with my keyboard -- even though my workstation is manufactured by a different company. I also want the colours to be right despite the fact that the HP machine uses a completely different video hardware to my machine. When I press the curly brace key {, I want to see a curly brace, and not some hieroglyphic because the HP station uses a different keyboard.
These are the problems which X tries to address. In a network environment we need a common window system which will work on any kind of hardware, and hide the differences between different machines as far as possible. But it has to be flexible enough to allow us to change all of the things we don't like -- to choose our own colours, and the kind of window borders we want etc. Other windowing systems (like Microsoft windows) ignore these problems and thereby lock the user to a single vendors products and a single operating system. (That, of course, is no accident.)
The way X solves this problem is to use the client server model. Each program which wants to open a window on somebody's compute screen is a client of the X window service. To get something drawn on a user's screen, the client asks a server on the host of interest to draw windows for it. No client ever draws anything itself -- it asks the server to do it on its behalf. There are several reasons for this:
" The clients can all talk a common `window language' or protocol. We can hide the difference between different kinds of hardware by making the machine-specific part of drawing graphics entirely a problem of implementing the server on the particular hardware. When a new type of hardware comes along, we just need to modify the server -- none of the clients need to be modified.
" We can contact different servers and send our output to different hardware -- thus even though a program is running on a CPU in Tokyo, it can ask the server in Massachusetts to display its window for it.
" When more than one window is on a user's display, it eventually becomes necessary to move the windows around and then figure out which windows are on top of which other windows etc. If all of the drawing information is kept in a server, it is straightforward to work out this information. If every client drew where it wanted to, it would be impossible to know which window was supposed to be on top of another.
In X, the window manager is a different program to the server which does the drawing of graphics -- but the client-server idea still applies, it just has one more piece to its puzzle.
How to set up X windows
The X windows system is large and complex and not particularly user friendly. When you log in to the system, X reads two files in your home directory which decide which applications will be started what they will look like. The files are called
.Xsession
This file is a shell script which starts up a number of applications as background processes and exits by calling a window manager. Here is a simple example file
#!/bin/bash
#
# .xsession file
#
#

PATH="/usr/bin:/bin:/local/gnu/bin:/usr/X11R6/bin"

#
# List applications here, with & at the end
# so they run in the background
#

xterm -T NewTitle -sl 1000 -geometry 90x45+16+150 -sb &
xclock &
xbiff -geometry 80x80+510+0 &
netscape -iconic&

# Start a window manager. Exec replaces this script with
# the fvwm process, so that it doesn't exist as a separate
# (useless) process.

exec /local/bin/fvwm

.Xdefaults
This file specifies all of the resources which X programs use. It can be used to change the colours used by applications, or font types etc. The subject of X-resources is a large one and we don't have time for it here. Here is a simple example, which shows how you can make your over-bright xterm and emacs windows less bright grey shade.
xterm*background: LightGrey
Emacs*background: grey92
Xemacs*background: grey92
X displays and authority
In the terminology used by X11, every client program has to contact a display in order to open a window. A display is a virtual screen which is created by the X server on a particular host. X can create several separate displays on a given host, though most machines only have one.
When an X client program wants to open a window, it looks in the UNIX environment variable `DISPLAY' for the IP address of a host which has an X server it can contact. For example, if we wrote
DISPLAY="myhost:0"
export DISPLAY
the client would try to contact the X server on `myhost' and ask for a window on display number zero (the usual display). If we wrote
DISPLAY="198.112.208.35:0"
export DISPLAY
the client would try to open display zero on the X server at the host with the IP address `198.112.208.35'.
Clearly there must be some kind of security mechanism to prevent just anybody from opening windows on someone's display. X has two such mechanisms:
xhost
This mechanism is now obsolete. The `xhost' command is used to define a list of hosts which are allowed to open windows on the user's display. It cannot distinguish between individual users. i.e. the command xhost yourhost would allow anyone using yourhost to access the local display. This mechanism is only present for backward compatibility with early versions of X windows. Normally one should use the command xhost - to exclude all others from accessing the display.
Xauthority
The Xauthority mechanism has replaced the xhost scheme. It provides a security mechanism which can distinguish individual users, not just hosts. In order for a user to open a window on a display, he/she must have a ticket--called a "magic cookie". This is a binary file called `.Xauthority' which is created in the user's home directory when he/she first starts the X-windows system. Anyone who does not have a recent copy of this file cannot open windows or read the display of the user's terminal. This mechanism is based on the idea that the user's home directory is available via NFS on all hosts he/she will log onto, and thus the owner of the display will always have access to the magic cookie, and will therefore always be able to open windows on the display. Other users must obtain a copy of the file in order to open windows there. The command xauth is an interactive utility used for controlling the contents of the `.Xauthority' file. See the `xauth' manual page for more information.
Multiple screens
The window paradigm has been very successful in many ways, but anyone who has used a window system knows that the screen is simply not big enough for all the windows one would like! UNIX has several solutions to this problem.
One solution is to attach several physical screens to a terminal. The X window system can support any number of physical screens of different types. A graphical designer might want a high resolution colour screen for drawing and a black and white screen for writing text, for instance. The disadvantage with this method is the cost of the hardware.
A cheaper solution is to use a window manager such as `fwvm' which creates a virtual screen of unlimited size on a single monitor. As the mouse pointer reaches the edge of the true screen, the window manager replaces the display with a new "blank screen" in which to place windows. A miniaturized image of the windows on a control panel acts as a map which makes it possible to find the applications on the virtual screen.
Yet another possibility is to create virtual displays inside a single window. In other words, one can collapse several shell windows into a single `xterm' window by running the program `screen'. The screen command allows you to start several shells in a single window (using CTRL-a CTRL-c) and to switch between them (by typing CTRL-a CTRL-n). It is only possible to see one shell window at a time, but it is still possible to cut and paste between windows and one has a considerable saving of space. The `screen' command also allows you to suspend a shell session, log out, log in again later and resume the session precisely where you left off.
Here is a summary of some useful screen commands:
screen
Start the screen server.
screen -r
Resume a previously suspended screen session if possible.
CTRL-a CTRL-c
Start a new shell on top of the others (a fresh `screen') in the current window.
CTRL-a CTRL-n
Switch to the next `screen'.
CTRL-a CTRL-a
Switch to the last screen used.
CTRL-a a
When screen is running, CTRL-a is used for screen commands and cannot therefore be used in its usual shell meaning of `jump to start of line'. CTRL-a a replaces this.
CTRL-a CTRL-d
Detach the screen session from the current window so that it can be resumed later. It can be resumed with the `screen -r' command.
CTRL-a ?
Help screen.
Files and access
To prevent all users from being able to access all files on the system, UNIX records information about who creates files and also who is allowed to access them later.
Each user has a unique username or loginname together with a unique user id or uid. The user id is a number, whereas the login name is a text string -- otherwise the two express the same information. A file belongs to user A if it is owned by user A. User A then decides whether or not other users can read, write or execute the file by setting the protection bits or the permission of the file using the command chmod.
In addition to user identities, there are groups of users. The idea of a group is that several named users might want to be able to read and work on a file, without other users being able to access it. Every user is a member of at least one group, called the login group and each group has both a textual name and a number (group id). The uid and gid of each user is recorded in the file /etc/passwd (See chapter 6). Membership of other groups is recorded in the file /etc/group or on some systems /etc/logingroup.
Protection bits
The following output is from the command ls -lag executed on a SunOS type machine.

lrwxrwxrwx 1 root wheel 7 Jun 1 1993 bin -> usr/bin
-r--r--r-- 1 root bin 103512 Jun 1 1993 boot
drwxr-sr-x 2 bin staff 11264 May 11 17:00 dev
drwxr-sr-x 10 bin staff 2560 Jul 8 02:06 etc
drwxr-sr-x 8 root wheel 512 Jun 1 1993 export
drwx------ 2 root daemon 512 Sep 26 1993 home
-rwxr-xr-x 1 root wheel 249079 Jun 1 1993 kadb
lrwxrwxrwx 1 root wheel 7 Jun 1 1993 lib -> usr/lib
drwxr-xr-x 2 root wheel 8192 Jun 1 1993 lost+found
drwxr-sr-x 2 bin staff 512 Jul 23 1992 mnt
dr-xr-xr-x 1 root wheel 512 May 11 17:00 net
drwxr-sr-x 2 root wheel 512 Jun 1 1993 pcfs
drwxr-sr-x 2 bin staff 512 Jun 1 1993 sbin
lrwxrwxrwx 1 root wheel 13 Jun 1 1993 sys->kvm/sys
drwxrwxrwx 6 root wheel 732 Jul 8 19:23 tmp
drwxr-xr-x 27 root wheel 1024 Jun 14 1993 usr
drwxr-sr-x 10 bin staff 512 Jul 23 1992 var
-rwxr-xr-x 1 root daemon 2182656 Jun 4 1993 vmUNIX

The first column is a textual representation of the protection bits for each file. Column two is the number of hard links to the file (See exercises below). The third and fourth columns are the user name and group name and the remainder show the file size in bytes and the creation date. Notice that the directories /bin and /sys are symbolic links to other directories.
There are sixteen protection bits for a UNIX file, but only twelve of them can be changed by users. These twelve are split into four groups of three. Each three-bit number corresponds to one octal number.
The leading four invisible bits gives information about the type of file: is the file a plain file, a directory or a link. In the output from ls this is represented by a single character: -, d or l.
The next three bits set the so-called s-bits and t-bit which are explained below.
The remaining three groups of three bits set flags which indicate whether a file can be read `r', written to `w' or executed `x' by (i) the user who created them, (ii) the other users who are in the group the file is marked with, and (iii) any user at all.
For example, the permission
Type Owner Group Anyone
d rwx r-x ---
tells us that the file is a directory, which can be read and written to by the owner, can be read by others in its group, but not by anyone else.
Note about directories. It is impossible to cd to a directory unless the x bit is set. That is, directories must be `executable' in order to be accessible.
Here are some examples of the relationship between binary, octal and the textual representation of file modes.
Binary Octal Text

001 1 x
010 2 w
100 4 r
110 6 rw-
101 5 r-x
- 644 rw-r--r--

It is well worth becoming familiar with the octal number representation of these permissions.
Chmod
The chmod command changes the permission or mode of a file. Only the owner of the file or the superuser can change the permission. Here are some examples of its use. Try them.
# make read/write-able for everyone
chmod a+w myfile

# add the 'execute' flag for directory
chmod u+x mydir/

# open all files for everyone
chmod 755 *

# set the s-bit on my-dir's group
chmod g+s mydir/

# descend recursively into directory opening all files
chmod -R a+r dir
Umask
When a new file gets created, the operating system must decide what default protection bits to set on that file. The variable umask decides this. umask is normally set by each user in his or her .cshrc file (see next chapter). For example
umask 077 # safe
umask 022 # liberal
According the UNIX documentation, the value of umask is `XOR'ed (exclusive `OR') with a value of 666 & umask for plain files or 777 & umask for directories in order to find out the standard protection. Actually this is not quite true: `umask' only removes bits, it never sets bits which were not already set in 666. For instance
umask Permission

077 600 (plain)
077 700 (dir)
022 644 (plain)
022 755 (dir)
The correct rule for computing permissions is not XOR but `NOT AND'.
Making programs executable
A UNIX program is normally executed by typing its pathname. If the x execute bit is not set on the file, this will generate a `Permission denied' error. This protects the system from interpreting nonsense files as programs. To make a program executable for someone, you must therefore ensure that they can execute the file, using a command like
chmod u+x filename
This command would set execute permissions for the owner of the file;
chmod ug+x filename
would set execute permissions for the owner and for any users in the same group as the file. Note that script programs must also be readable in order to be executable, since the shell has the interpret them by reading.
chown and chgrp
These two commands change the ownership and the group ownership of a file. Only the superuser can change the ownership of a file on most systems. This is to prevent users from being able to defeat quota mechanisms. (On some systems, which do not implement quotas, ordinary users can give a file away to another user but not get it back again.) The same applies to group ownership.
Making a group
Normally users other than root cannot define their own groups. This is a weakness in UNIX from older times which no one seems to be in a hurry to change.
s-bit and t-bit (sticky bit)
The s and t bits have special uses. They are described as follows.
Octal Text Name

4000 chmod u+s Setuid bit
2000 chmod g+s Setgid bit
1000 chmod +t Sticky bit
The effect of these bits differs for plain files and directories and differ between different versions of UNIX. You should check the manual page man sticky to find out about your system! The following is common behaviour.
For executable files, the setuid bit tells UNIX that regardless of who runs the program it should be executed with the permissions and rights of owner of the file. This is often used to allow normal users limited access to root privileges. A setuid-root program is executed as root for any user. The setgid bit sets the group execution rights of the program in a similar way.
In BSD UNIX, if the setgid bit is set on a directory then any new files created in that directory assume the group ownership of the parent directory and not the logingroup of the user who created the file. This is standard policy under system 5.
A directory for which the sticky bit is set restrict the deletion of files within it. A file or directory inside a directory with the t-bit set can only be deleted or renamed by its owner or the superuser. This is useful for directories like the mail spool area and /tmp which must be writable to everyone, but should not allow a user to delete another user's files.
(Ultrix) If an executable file is marked with a sticky bit, it is held in the memory or system swap area. It does not have to be fetched from disk each time it is executed. This saves time for frequently used programs like ls.
(Solaris 1) If a non-executable file is marked with the sticky bit, it will not be held in the disk page cache -- that is, it is never copied from the disk and held in RAM but is written to directly. This is used to prevent certain files from using up valuable memory.
On some systems (e.g. ULTRIX), only the superuser can set the sticky bit. On others (e.g. SunOS) any user can create a sticky directory.
Bourne Again shell
The Bourne Again shell (Bash) is the command interpreter which you use to run programs and utilities. It contains a simple programming language for writing tailor-made commands, and allows you to join together UNIX commands with pipes. It is a configurable environment, and once you know it well, it is the most efficient way of working with UNIX.
The Bourne Again shell was written by the Free Software Foundation as a part of the GNU project and Bash is the default shell in most GNU/Linux distributions. Because of its command line editing features, it is much more efficient for interactive use than Bourne shell, the original UNIX shell. Most of the system scripts in UNIX are written in the Bourne shell. Although Bash includes many extensions and features not found in the Bourne shell, it maintains compatibility with it so that you can run Bourne shell scripts under Bash. On many GNU/Linux systems Bourne shell (`/bin/sh') is symbolically linked to Bash (`/bin/bash') so that the scripts that require the presence of the Bourne shell still run. If you want to write a platform independent shell script able to run on as many UNIX variants as possible, you should stick to Bourne shell syntax and avoid the Bash extensions.
`~/.bashrc' and `~/.bash_profile' files
When you log on to a GNU/Linux system and your login shell is defined in `/etc/passwd' to be Bash, it first executes commands in the `/etc/profile' file. It then searches for the `~/.bash_profile', `~/.bash_login' or `~/.profile' file, in this order, and executes commands in the first of these that is found and is readable. When a login exits, it executes commands in the `~/.bash_logout' file.
When you start an non-login interactive Bash shell, it only executes commands in the `~/.bashrc' file, if it exists and is readable. However, this shell inherits any environment (exported) variables from the parent shell, so environment variables set in `/etc/profile' and `~/.bash_profile' are passed onto the non-login shells and later to its subshells.
Here is a very simple example `~/.bashrc' file:
#
# .bashrc - read in by every bash that starts.
#

umask 077 # Set the default file creation mask
PATH="~/bin:$PATH" # Inserts own bin directory first in PATH
PS1="`uname`:\h\$ " # prompt
PS2="\h > " # prompt for foreach and while
PRINTER=myprinter

# Aliases are shortcuts to UNIX commands
alias h=history
alias ll="ls -l"
alias cp='cp -i'
alias rm='rm -i'
alias c='ssh cube'

In order to make sure your `~/.bashrc' file is read when logging on with ssh to another machine, you may start your `~/.bash_profile' file like this:
#
# .bash_profile - read in every login.
#

if [ -f ~/.bashrc ]
then
source ~/.bashrc # runs .bashrc as if they where
# typed into this file
fi
Variables and export
Shell variables are defined using the syntax
VARIABLE=”username is”
myname=”`whoami`”
It is important that there be no space between the variable and the equals sign. These variables are then referred to using the dollar `$’ symbol.
$ echo “My $VARIABLE $myname”
My username is mark
When assigning values to variables the dollar symbol is never used. By default these variables are local – that is they will not be passed on to programs and sub-shells running under the current shell. To make them global (so that child processes will inherit them) we use the command
export VARIABLE
This adds the variable to the process environment. Under Bash (but not under the old Bourne shell) it is also possible to declare a variable to be global on a single line by
export GLOBALVAR=”global”
The command
set -a
changes the default so that all variables, after the command are created global.
Arrays or lists are often simulated in Bourne shell by sandwiching the colon `:’ symbol between items
PATH=/bin:/usr/bin:/etc:/local/bin:.

LD_LIBARAY_PATH=/usr/lib:/usr/openwin/lib:/local/lib
but there is no real facility for arrays in the Bourne shell. Note that the UNIX `cut’ command can be used to extract the elements of the list. Loops can also read such lists directly See section Loops in Bash. However, Bash version 2.x supports arrays as seen in the next section.
The value of a variable is given by the dollar symbol. It is also possible to use curly braces around the variable name to `protect’ the variable from interfering text. For example:
$ animal=worm
$ echo book$animal
bookworm
$ thing=book
$ echo $thingworm
(nothing..)
$ echo ${thing}worm
bookworm

Default values can be given to variables in the Bourne shell. The following commands illustrate this.
echo ${var-”No value set”}
echo ${var=”Octopus”}
echo ${var+”Forced value”}
echo ${var?”No such variable”}
The first of these prints out the contents of `$var’, if it is defined. If it is not defined the variable is substituted for the string “No value set”. The value of `var’ is not changed by this operation. It is only for convenience.
The second command has the same effect as the first, but here the value of `$var’ is actually changed to “Octopus” if `$var’ is not set.
The third version is slightly peculiar. If `$var’ is already set, its value will be forced to be “Forced value”, otherwise it is left undefined.
Finally the last instance issues an error message “No such variable” if `$var’ is not defined.
In Bash 2.x it is possible to extract parts of the string a variable is set to using the construction ${variable:offset:length}| as shown in the next example.
var=”abcdefg”
middle=${var:2:3}
echo $middle
cde
An offset of 2 skips the first 2 characters and a string of length 3 is extracted from the middle of the string.
Bash arrays
The original Bourne shell does not have arrays. Bash version 2.x does have arrays, however. An array can be assigned from a string of words separated by whitespaces or the individual elements of the array can be set individually.
colours=(red white green)
colours[3]=”yellow”
An element of the array must be referred to using curly braces.
echo ${colours[1]}
white
Note that the first element of the array has index 0. The set of all elements is referred to by ${colours[*]}.
echo ${colours[*]}
red white green yellow
echo ${#colours[*]}
4
As seen the number of elements in an array is given by ${#colours[*]}.
Stdin, stdout, stderr and redirection to and from files
When the shell starts up, it inherits three files: `stdin’, `stdout’, and `stderr’. Standard input normally comes from the keyboard. Standard output and standard error normally go to the screen. There are times you want to read input from a file or send output of errors to a file. This can be accomplished by using I/O redirection.
In Bash and the Bourne shell, the standard input/output files are referred to by numbers rather than by names.
stdin
File number 0
stdout
File number 1
stderr
File number 2
The default routes for these files can be changed by redirection. The output of the command echo is by default sent to the screen, that is the stdout with file number 1 is sent to the screen. Using redirection operators it is possible to redirect the standard out of echo to where we want it. We can send output to a file with the following command.
echo “should be sent to a file” > file.txt
This creates a new file `file.txt’ containing the string ’should be sent to a file’. The redirection operator could have been given as 1>, but it is understood that standard out is meant when skipping the number of the file handle. The single ‘>’ always creates a new file, while ‘>>’ appends to the end of a file.
If you had mistyped the command echo the result would have been:
ehco “should be sent to a file” > file.txt
bash: ehco: command not found
The standard error with file handle 2 is by default sent to the screen, independent of where standard out (1) is sent. If you like you can redirect stdout to another or the same file.
ehco “should be sent to a file” > file.txt 2> error.txt
cat error.txt
bash: ehco: command not found
There are several ways to send stderr to the same file as stdin is redirected to. The following three commands are equivalent.
ehco “should be sent to a file” >& file.txt
ehco “should be sent to a file” > file.txt 2> file.txt
ehco “should be sent to a file” > file.txt 2>&1
The string 2>&1 means that stderr(2) should be sent to the same file as stdout(1). This is the only why to do this under the Bourne shell and this construction is therefore often seen in system shell scripts.
Furthermore it is possible to force a command which by default takes standard input from the keyboard, to read input from a file by redirecting stdin. The mail-command expects input from keyboard, but the ‘<’ redirection operator makes it send the password file to the user mark:
/bin/mail mark < /etc/passwd
The following table summarizes the most important redirection operators:
Redirection operator What it does

Redirects output
>> Appends output
2> Redirects error
>& Redirects output and error (Bash only)
2>&1 Redirects error where output (1) is going

Pipes
A pipe takes the output from the command on the left-hand side of the pipe symbol and sends it to the input of the command on the right-hand side of the pipe symbol. A pipeline can consist of several pipes and this makes pipes a very powerful tool. It enables us to combine all the small and efficient UNIX commands in any thinkable way. If you want to count the number of people logged on, you could save the output of the command who in the temporary file `tmp’, use wc -l to count the number of lines in `tmp’ and finally remove the temporary file.
$ who > tmp
$ wc -l tmp
4 tmp
$ rm tmp
Using a pipe saves disk space and time: the stdout from who can be redirected to the stdin of wc -l through a pipe and there is no need for temporarily storing the output from who.
$ who | wc -l
4
Most UNIX-commands are constructed with piping in mind and this makes it possible to solve complex tasks easily, by joining commands along a pipeline. Consider the following pipeline:
cat big.jpg | djpeg | pnmscale -pixels 150000 | cjpeg > small.jpg
The command cat sends the large JPEG-image to djpeg which decompresses it and sends the resulting bitmap to stdout. The stream of data floats through the next pipe to pnmscale which scales the bitmap image down to the given size. The scaled image is piped to the command cjpeg which compresses the standard input and finally produces a JPEG-image of reduced size which is stored in the file `small.jpg’.
Command history
The history feature in Bash means that you do not have to type commands over and over again. You can use the UP ARROW key to browse back through the list of commands you have typed previously and the keys LEFT ARROW and RIGHT ARROW to edit these commands.
In addition there are a couple of commands which selects commands from the history list.
`!!’
Execute the last command again.
`!4′
Execute command number 4.
The first of these simply repeats the last command. The second command gives an absolute number. The absolute command number can be seen by typing `history’.
Command/filename completion
In Bash you can save hours worth of typing errors by using the completion mechanism. This feature is based on the TAB key.
The idea is that if you type half a filename and press TAB, the shell will try to guess the remainder of the filename. It does this by looking at the files which match what you have already typed and trying to fill in the rest. If there are several files which match, the shell sounds the “bell” or beeps. You can then type TAB twice to obtain a list of the possible alternatives. Here is an example: suppose you have just a single file in the current directory called `very_long_filename’, typing
more TAB
results in the following appearing on the command line
more very_long_filename
The shell was able to identify a unique file. Now suppose that you have two files called `very_long_filename’ and `very_big_filename’, typing
more TAB
results in the following appearing on the command line
more very_
and the shell beeps, indicating that the choice was not unique and a decision is required. Next, you type TAB twice(3) to see which files you have to choose from and the shell lists them and returns you to the command line, exactly where you were. You now choose `very_long_filename’ by typing `l’. This is enough to uniquely identify the file. Pressing the TAB key again results in
more very_long_filename
on the screen. As long as you have written enough to select a file uniquely, the shell will be able to complete the name for you.
Completion also works on shell commands, but it is a little slower since the shell must search through all the directories in the command path to complete commands.
Single and double quotes
Two kinds of quotes can be used in shell apart from the backward quotes we mentioned above. The essential difference between them is that certain shell commands work inside double quotes but not inside single quotes. For example

cube$ echo /etc/rc*
/etc/rc.boot /etc/rc0.d /etc/rc1.d /etc/rc2.d /etc/rc3.d /etc/rc4.d

cube$ echo “/etc/rc*”
/etc/rc*

cube$ echo “`whoami` — my name is $USER”
mark — my name is mark

cube$ echo ‘`whoami` — my name is $USER’
`whoami` — my name is $USER

We see that the single quotes prevent variable substitution and sub-shells. Wildcards do not work inside either single or double quotes.
Job control, break key, `fg’, `bg’
So far we haven’t mentioned UNIX’s ability to multitask. In the Bourne shell (`sh’) there are no facilities for controlling several user processes. Bash provides some commands for starting and stopping processes. These originate from the days before windows and X11, so some of them may seem a little old-fashioned. They are still very useful nonetheless.
Let’s begin by looking at the commands which are true for any shell. Most programs are run in the foreground or interactively. That means that they are connected to the standard input and send their output to the standard output. A program can be made to run in the background, if it does not need to use the standard I/O. For example, a program which generates output and sends it to a file could run in the background. In a window environment, programs which create their own windows can also be started as background processes, leaving standard I/O in the shell free.
Background processes run independently of what you are doing in the foreground.
UNIX Processes and BSD signals
A background process is started using the special character `&’ at the end of the command line.
find / -name ‘*lib*’ -print >& output &
The final `&’ on the end of this line means that the job will be run in the background. Note that this is not confused with the redirection operator `>&’ since it must be the last character on the line. The command above looks for any files in the system containing the string `lib’ and writes the list of files to a file called `output’. This might be a useful way of searching for missing libraries which you want to include in your environment variable `LD_LIBRARY_PATH’. Searching the entire disk from the root directory `/’ could take a long time, so it pays to run this in the background.
If we want to see what processes are running, we can use the `ps’ command. `ps’ without any arguments lists all of your processes, i.e. all processes owned by the user name you logged in with in the current shell. `ps’ takes many options, for instance `ps auxg’ will list all processes in gruesome detail (The “g” is for group, not gruesome!). `ps’ reads the kernel’s process tables directly.
Processes can be stopped and started, or killed one and for all. The `kill’ command does this. There are, in fact, two versions of the `kill’ command. One of them is built into Bash and the other is not. If you use Bash then you will never care about the difference. We shall nonetheless mention the special features of Bash built-ins below. The kill command takes a number called a signal as an argument and another number called the process identifier or PID for short. Kill send signals to processes. Some of these are fatal and some are for information only. The two commands
kill -15 127
kill 127

are identical. They both send signal 15 to PID 127. This is the normal termination signal and it is often enough to stop any process from running.
Programs can choose to ignore certain signals by trapping signals with a special handler. One signal they cannot ignore is signal 9.
kill -9 127
is a sure way of killing PID 127. Even though the process dies, it may not be removed from the kernel’s process table if it has a parent (see next section).
Here is the complete list of signals which the Linux kernel send to processes in different circumstances.
#define SIGHUP 1 /* Hangup (POSIX). */
#define SIGINT 2 /* Interrupt (ANSI). */
#define SIGQUIT 3 /* Quit (POSIX). */
#define SIGILL 4 /* Illegal instruction (ANSI). */
#define SIGTRAP 5 /* Trace trap (POSIX). */
#define SIGABRT 6 /* Abort (ANSI). */
#define SIGIOT 6 /* IOT trap (4.2 BSD). */
#define SIGBUS 7 /* BUS error (4.2 BSD). */
#define SIGFPE 8 /* Floating-point exception (ANSI). */
#define SIGKILL 9 /* Kill, unblockable (POSIX). */
#define SIGUSR1 10 /* User-defined signal 1 (POSIX). */
#define SIGSEGV 11 /* Segmentation violation (ANSI). */
#define SIGUSR2 12 /* User-defined signal 2 (POSIX). */
#define SIGPIPE 13 /* Broken pipe (POSIX). */
#define SIGALRM 14 /* Alarm clock (POSIX). */
#define SIGTERM 15 /* Termination (ANSI). */
#define SIGSTKFLT 16 /* Stack fault. */
#define SIGCLD SIGCHLD /* Same as SIGCHLD (System V). */
#define SIGCHLD 17 /* Child status has changed (POSIX). */
#define SIGCONT 18 /* Continue (POSIX). */
#define SIGSTOP 19 /* Stop, unblockable (POSIX). */
#define SIGTSTP 20 /* Keyboard stop (POSIX). */
#define SIGTTIN 21 /* Background read from tty (POSIX). */
#define SIGTTOU 22 /* Background write to tty (POSIX). */
#define SIGURG 23 /* Urgent condition on socket (4.2 BSD). */
#define SIGXCPU 24 /* CPU limit exceeded (4.2 BSD). */
#define SIGXFSZ 25 /* File size limit exceeded (4.2 BSD). */
#define SIGVTALRM 26 /* Virtual alarm clock (4.2 BSD). */
#define SIGPROF 27 /* Profiling alarm clock (4.2 BSD). */
#define SIGWINCH 28 /* Window size change (4.3 BSD, Sun). */
#define SIGPOLL SIGIO /* Pollable event occurred (System V). */
#define SIGIO 29 /* I/O now possible (4.2 BSD). */
#define SIGPWR 30 /* Power failure restart (System V). */
#define SIGSYS 31 /* Bad system call. */
We have already mentioned 15 and 9 which are the main signals for users. Signal 1, or `HUP’ can be sent to certain programs by the superuser. For instance
kill -1
kill -HUP
which forces `inetd’ to reread its configuration file. Sometimes it is useful to suspend a process temporarily and then restart it later.
kill -20 # suspend process
kill -18 # resume process
Child Processes and zombies
When you start a process from a shell, regardless of whether it is a background process or a foreground process, the new process becomes a child of the original shell. Remember that the shell is just a UNIX process itself. Moreover, if one of the children starts a new process then it will be a child of the child (a grandchild?)! Processes therefore form hierarchies. Several children can have a common parent.
If we kill a parent, then (unless the child has detached itself from the parent) all of its children die too. If a child dies, the parent is not affected. Sometimes when a child is killed, it does not die but becomes “defunct” or a zombie process. This means that the child has a parent which is waiting for it to finish. If the parent has not yet been informed that the child has died, for example because it has been suspended itself, then the dead child is not removed from the kernel’s process table. When the parent wakes up and receives the message that the child has terminated, the process entry for the dead child can be removed.
Bash builtins: `jobs’, `kill’, `fg’,`bg’, break key
Now let’s look at some commands which are built into Bash for starting and stopping processes. Bash refers to user programs as `jobs’ rather than processes — but there is no real difference. The added bonus of Bash is that each shell has a job number in addition to its PID. The job numbers are simpler and are private for the shell, whereas the PIDs are assigned by the kernel and are often very large numbers which are difficult to to remember. When a command is executed in the shell, it is assigned a job number. If you never run any background jobs then there is only ever one job number: 1, since every job exits before the next one starts. However, if you run background tasks, then you can have several jobs “active” at any time. Moreover, by suspending jobs, Bash allows you to have several interactive programs running on the same terminal — the `fg’ and `bg’ commands allow you to move commands from the background to the foreground and vice-versa.
Take a look at the following shell session.
cube$ emacs myfile&
[3] 771
cube$

( other commands … , edit myfile and close emacs )

When a background job is done, the shell prints a message at a suitable moment between prompts.
[3]+ Done emacs myfile
cube$
This tells you that job number 1 finished normally. If the job exits abnormally then the word `Done’ may be replaced by some other message. For instance, if you kill the job, it will say
cube$ kill %3
cube$
[3]+ Terminated emacs myfile
cube$
You can list the jobs you have running using the `jobs’ command. The output looks something like
cube$ jobs
[1] Terminated xdvi unix
[2] Running xemacs unix.texinfo &
[3] Running xterm -sb -sl 10000 &
[4] Running ghostview &
[5] Running netscape &
[6] Running xterm -sb -sl 10000 &
[7] Running xemacs fil &
[8]+ Stopped emacs unix.log
[9]- Running gimp &
To suspend a program which you are running in the foreground you can type CTRL-z (this is like sending a `kill -20′ signal from the keyboard). (4) You can suspend any number of programs and then restart them one at a time using `fg’ and `bg’. If you want job 5 to be restarted in the foreground, you would type
fg %5
When you have had enough of job 5, you can type CTRL-z to suspend it and then type
fg %6
to activate job 6. Provided a job does not want to send output to `stdout’, you can restart any job in the background, using a command like.
bg %4
This method of working was useful before windows were available. Using `fg’ and `bg’, you can edit several files or work on several programs without have to quit to move from one to another.
See also some related commands for batch processing `at’, `batch’ and `atq’, `cron’.
NOTE: CTRL-c sends a `kill -2′ signal, which send a standard interrupt message to a program. This is always a safe way to interrupt a shell command.
Arithmetic in Bash
In Bourne shell arithmetic is performed entirely `by proxy’. To evaluate an expression we call the `expr’ command or the `bc’ precision calculator. Here are some examples of `expr’
a=`expr $a+1` # increment a
a=`expr 4 + 10 \* 5` # 4+10*5
check = `expr $a \> $b` # true=1, false=0. True if $a > $b
`expr’ is very sensitive to spaces and backslash characters and this makes it a bit awkward to do arithmetic under the Bourne shell.
Bash 2.0 provides a new and simpler way to do arithmetic using double parentheses. If you surround any integer arithmetic expression as in (( x = y + 1 )), you can perform most arithmetic operations with the same syntax as in Java and C.
(( x = 1 ))
echo $x
1
(( x++ ))
(( y = 4*x ))
echo $y
8
Note that you do not need to use the dollar symbol to refer to a variable within the double parentheses (but you may do it) and that spaces are allowed.
(( sum = 2 ))
(( total = 4*$sum + sum ))
echo $total
10
The variables within double parentheses are throughout treated as integers. Assigning a float value like 2.5 to a variable results in an syntax error while assigning a string to a variable cause the string to be stored as zero.
Scripts and arguments
Scripts are created by making an executable file which begins with the sequence of characters
#!/bin/bash
This construction is quite general: any executable file which begins with a sequence
#!myprogram -option
will cause the shell to attempt to execute
myprogam -option filename
where filename is the name of the file.
If a script is to accept arguments then these can be referred to as ` $1 $2 $3..$9′. There is a logical limit of nine arguments to a Bourne script, but Bash handles the next arguments as `${10}’. `$0′ is the name of the script itself.
Here is a simple Bash script which prints out all its arguments.

#!/bin/bash
#
# Print all arguments (version 1)
#

for arg in $*
do
echo Argument $arg
done

echo Total number of arguments was $#

The `$*’ symbol stands for the entire list of arguments and `$#’ is the total number of arguments.
Another way of achieving the same is to use the `shift’ command. We shall meet this again in the Perl programming language. `shift’ takes the first argument from the argument list and deletes it, moving all of the other arguments down one number — this is how we can handle long lists of arguments in the Bourne shell.
#!/bin/bash
#
# Print all arguments (version 2)
#

while ( true )
do
arg=$1;
shift;
echo $arg was an argument;
if [ $# -eq 0 ]; then
break
fi
done

Return codes
All programs which execute in UNIX return a value through the C `return’ command. There is a convention that a return value of zero (0) means that everything went well, whereas any other value implies that some error occurred. The return value is usually the value returned in `errno’, the external error variable in C.
Shell scripts can test for these values either by placing the command directly inside an `if’ test, or by testing the variable `$?’ which is always set to the return code of the last command. Some examples are given following the next two sections.
Tests and conditionals
Bash and the Bourne shell has an array of tests. They are written as follows. Notice that `test’ is itself not a part of the shell, but is a program which works out conditions and provides a return code. See the manual page on `test’ for more details.
test -f file
True if the file is a plain file
test -d file
True if the file is a directory
test -r file
True if the file is readable
test -w file
True if the file is writable
test -x file
True if the file is executable
test -s file
True if the file contains something
test -g file
True if setgid bit is set
test -u file
True if setuid bit is set
test s1 = s2
True if strings s1 and s2 are equal
test s1 != s2
True if strings s1 and s2 are unequal
test x -eq y
True if the integers x and y are numerically equal
test x -ne y
True if integers are not equal
test x -gt y
True if x is greater than y
test x -lt y
True if x is less than y
test x -ge y
True if x>=y
test x -le y
True if x <= y
!
Logical NOT operator
-a
Logical AND
-o
Logical OR
Note that an alternate syntax for writing these commands if to use the square brackets, instead of writing the word test.
[ $x -lt $y ] “==” test $x -lt $y
Just as with the arithmetic expressions, Bash 2.x provides a syntax for conditionals which are more similar to Java and C. While arithmetic C-like expressions can be used within double parentheses, C-like tests can be used within double square brackets.
[[ $var == "OK" || $var == "yes" ]]
This C-like syntax is not allowed in the Bourne shell, but is equivalent to
[ $var = "OK" -o $var = "yes" ]
which is valid in both shells.
Arithmetic C-like tests can be used within double parentheses so that under Bash 2.x the following tests are equivalent:
[ $x -lt $y ] “==” (( x < y ))
Conditional structures
The conditional structures have the following syntax.
if UNIX-command
then
command
else
commands
fi
The `else’ clause is, of course, optional. As noted before, the first UNIX command could be anything, since every command has a return code. The result is TRUE if it evaluates to zero and false otherwise (in contrast to the conventions in most languages). Multiple tests can be made using
if UNIX-command
then
commands
elif UNIX-command
then
commands
elif UNIX-command
then
commands
else
commands
fi
where `elif’ means `else-if’.
The equivalent of the C-school’s `switch’ statement is a more Pascal-like `case’ structure.
case UNIX-command-or-variable in

wildcard1) commands ;;
wildcard2) commands ;;
wildcard3) commands ;;

esac
This structure uses the wildcards to match the output of the command or variable in the first line. The first pattern which matches gets executed.
Input from the user in Bash
In shell you can read the value of a variable using the `read’ command, with syntax
read variable
This reads in a string from the keyboard and terminates on a newline character. Under the old Bourne shell another way to do this is to use the `input’ command to access a particular logical device. The keyboard device in the current terminal is `/dev/tty’, so that one writes
variable = `line /dev/tty
exit 1
fi

echo “Looking for $1″

until users | grep -s $1
do
sleep 60
done

echo “!!! WAKE UP !!!” > /dev/tty
echo “User $1 just logged in” > /dev/tty

This script uses `grep’ in `silent mode’ (-s option). i.e. grep never writes anything to the terminal. The only thing we are interested in is the return code the piped command produces. If `grep’ detects a line containing the username we are interested in, then the result evaluates to TRUE and the sleep-loop exits.
Our final example is the kind of script which is useful for a system administrator. It transfers over the Network Information Service database files so that a slave server is up to date. All we have to do is make a list of the files and place it in a `for’ loop. The names used below are the actual names of the NIS maps, well known to system administrators.
#!/bin/bash
#
# Update the NIS database maps on a client server. This program
# shouldn’t have to be run, but sometimes things go wrong and we
# have to force a download from the main sever.
#
PATH=/etc/yp:/usr/etc/yp:$PATH

MASTER=myNISserver

for map in auto.direct auto.master ethers.byaddr ethers.byname\
group.bygid group.byname hosts.byaddr hosts.byname\
mail.aliases netgroup.byhost netgroup.byuser netgroup\
netid.byname networks.byaddr networks.byname passwd.byname\
passwd.byuid priss.byname protocols.byname protocols.bynumber\
rpc.bynumber services.byname services usenetgroups.byname;
do
ypxfr $1 -h $MASTER $map
done

Procedures and traps
One of the worthy features of the Bourne shell is that it allows you to define subroutines or procedures. Subroutines work just like subroutines in any other programming language. They are executed in same shell (not as a sub-process).
Here is an interesting program which demonstrates two useful things at the same time. First of all, it shows how to make a hierarchical subroutine structure using the Bourne shell. Secondly, it shows how the `trap’ directive can be used to trap signals, so that Bourne shell programs can exit safely when they are killed or when CTRL-C is typed.
#!/bin/bash
#
# How to make a signal handler in Bourne Shell
# using subroutines
#

#####################################################
# Level 2
#####################################################

ReallyQuit()
{
while true
do
echo “Do you really want to quit?”
read answer

case $answer in

y* | Y* ) return 0;;
*) echo “Resuming…”
return 1;;

esac

done
}

#####################################################
# Level 1
#####################################################

SignalHandler()

{
if ReallyQuit # Call a function
then
exit 0
else
return 0
fi
}

#####################################################
# Level 0 : main program
#####################################################

trap SignalHandler 2 15 # Trap kill signals 2 and 15

echo “Type some lines of text…”

while read text
do

echo “$text – CTRL-C to exit”

done

Note that the logical tree structure of this program is upside down (the highest level comes at the bottom). This is because all subroutines must be defined before they are used.
This example concludes our survey of Bash and the Bourne shell.
setuid and setgid scripts
The superuser `root’ is the only privileged user in UNIX. All other users have only restricted access to the system. Usually this is desirable, but sometimes it is a nuisance.
A setuid script is a script which has its setuid-bit set. When such a script is executed by a user, it is run with all the rights and privileges of the owner of the script. All of the commands in the script are executed as the owner of the file and not with the user-id of the person who ran the script. If the owner of the setuid script is `root’ then the commands in the script are run with root privileges!
Setuid scripts are clearly a touchy security issue. When giving away one’s rights to another user (especially those of `root’) one is tempting hackers. Setuid scripts should be avoided.
A setgid program is almost the same, but only the group id is set to that of the owner of the file. Often the effect is the same.
An example of a setuid program is the `ps’ program. `ps’ lists all of the processes running in the kernel. In order to do this it needs permission to access the private data structures in the kernel. By making `ps’ setgid root, it allows ordinary users to be able to read as much as the writers of `ps’ thought fit, but no more.
Naturally, only the superuser can make a file setuid or setgid root.
Exercises
1. Write an improved `which’ command in Bash.
2. Make a counter program which records in a file how many times you log in to your account. You can call this in your .bashrc file.
3. Make a Bourne shell script to kill all the processes owned by a particular user. (Note, that if you are not the superuser, you cannot kill processes owned by other users.)
4. Write a script to replace the `rm’ command with something safer. Think about a way of implementing `rm’ so that it is possible to get deleted files back again in case of emergencies. This is not possible using the normal `rm’ command. Hint: save files in a hidden directory `.deleted’. Make your script delete files in the `.deleted’ directory if they are older than a week, so that you don’t fill up the disk with rubbish.
5. Suppose you have a bunch of files with a particular file-extension: write a script in Bash to change the extension to something else. e.g. to change *.C into *.c. Give the old and new extensions as arguments to the script.
6. Write a program in Bash to search for files in the current directory which contain a certain string. e.g. search for all files which contain the word “if”. Hint: use the “find” command.
7. Use the manual pages to find out about the commands `at’, `batch’ and `atq’. Test these commands by executing the shell command `date’ at some time of your choice. Use the `-m’ option so that the result of the job is mailed to you.
8. Write a script in Bash to list all of the files bigger than a certain size starting from the current directory, and including all subdirectories. This kind of program is useful for system administrators when a disk becomes full.
C shell
Programmers who are used to C or C++ often find it easier to program in C-shell because there are strong similarities between the two.
.cshrc and .login files
Most users run the C-shell `/bin/csh’ as their login environment, or these days, preferably the `tcsh’ which is an improved version of csh. When a user logs in to a UNIX system the C-shell starts by reading some files which configure the environment by defining variables like path.
“ The file `.cshrc’ is searched for in your home directory. i.e. `~/.cshrc’. If it is found, its contents are interpreted by the C-shell as C-shell instructions, before giving you the command prompt(5).
“ If and only if this is the login shell (not a sub-shell that you have started after login) then the file `~/.login’ is searched for and executed.
With the advent of the X11 windowing system, this has changed slightly. Since the window system takes over the entire login procedure, users never get to run `login shells’, since the login shell is used up by the X11 system. On an X-terminal or host running X the `.login’ file normally has no effect.
With some thought, the `.login’ file can be eliminated entirely, and we can put everything into the .cshrc file. Here is a very simple example `.cshrc’ file.
#
# .cshrc – read in by every csh that starts.
#

# Set the default file creation mask
umask 077

# Set the path
set path=( /usr/local/bin /usr/bin/X11 /usr/ucb /bin /usr/bin . )

# Exit here if the shell is not interactive
if ( $?prompt == 0 ) exit

# Set some variables

set noclobber notify filec nobeep
set history=100
set prompt=”`hostname`%”
set prompt2 = “%m %h>” # tcsh, prompt for foreach and while

setenv PRINTER myprinter
setenv LD_LIBRARY_PATH /usr/lib:/usr/local/lib:/usr/openwin/lib

# Aliases are shortcuts to UNIX commands

alias passwd yppasswd
alias dir ‘ls -lg \!* | more’
alias sys ‘ps aux | more’
alias h history
It is possible to make a much more complicated .cshrc file than this. The advent of distributed computing and NFS (Network file system) means that you might log into many different machines running different versions of UNIX. The command path would have to be set differently for each type of machine.
Defining variables with set, setenv
We have already seen in the examples above how to define variables in C-shell. Let’s formalize this. To define a local variable — that is, one which will not get passed on to programs and sub-shells running under the current shell, we write
set local = “some string”
set myname = “`whoami`”
These variables are then referred to by using the dollar `$’ symbol. i.e. The value of the variable `local’ is `$local’.
echo $local $myname
Global variables, that is variables which all sub-shells inherit from the current shell are defined using `setenv’
setenv GLOBAL “Some other string”
setenv MYNAME “`who am i`”
Their values are also referred to using the `$’ symbol. Notice that set uses an `=’ sign while `setenv’ does not.
Variables can be also created without a value. The shell uses this method to switch on and off certain features, using variables like `noclobber’ and `noglob’. For instance
nexus% set flag
nexus% if ($?flag) echo ‘Flag is set!’
Flag is set!
nexus% unset flag
nexus% if ( $?flag ) echo ‘Flag is set!’
nexus%
The operator `$?variable’ is `true’ if variable exists and `false’ if it does not. It does not matter whether the variable holds any information.
The commands `unset’ and `unsetenv’ can be used to undefine or delete variables when you don’t want them anymore.
Arrays
A useful facility in the C-shell is the ability to make arrays out of strings and other variables. The round parentheses `(..)’ do this. For example, look at the following commands.
nexus% set array = ( a b c d )
nexus% echo $array[1]
a
nexus% echo $array[2]
b
nexus% echo $array[$#array]
d

nexus% set noarray = ( “a b c d” )
nexus% echo $noarray[1]
a b c d
nexus% echo $noarray[$#noarray]
a b c d
The first command defines an array containing the elements `a b c d’. The elements of the array are referred to using square brackets `[..]‘ and the first element is `$array[1]‘. The last element is `$array[4]‘. NOTE: this is not the same as in C or C++ where the first element of the array is the zeroth element!
The special operator `$#’ returns the number of elements in an array. This gives us a simple way of finding the end of the array. For example
nexus% echo $#path
23

nexus% echo “The last element in path is $path[$#path]“
The last element in path is .
To find the next last element we need to be able to do arithmetic. We’ll come back to this later.
Pipes and redirection in csh
The symbols
>> <’. For example,
echo testing > myfile
produces a file called `myfile’ which contains the string `testing’. The single `>’ (greater than) sign always creates a new file, whereas the double `>>’ appends to the end of a file, if it already exists. So the first of the commands
echo blah blah >> myfile
echo Newfile > myfile
adds a second line to `myfile’ after `testing’, whereas the second command writes over `myfile’ and ends up with just one line `Newfile’.
Now suppose we mistype a command
ehco test > myfile
The command `ehco’ does not exist and so the error message `ehco: Command not found’ appears on the terminal. This error message was sent to stderr — so even though we redirected output to a file, the error message appeared on the screen to tell us that an error occurred. Even this can be changed. `stderr’ can also be redirected by adding an ampersand `&’ character to the `>’ symbol. The command
ehco test >& myfile
results in the file `myfile’ being created, containing the error message `ehco: Command not found’.
The input direction can be changed using the `<’ symbol for example
/bin/mail mark ’ command. If one tries to redirect output to an existing file, the following happens.
UNIX% set noclobber
UNIX% touch blah # create an empty file blah
UNIX% echo test > blah
blah: File exists.
If you are nervous about overwriting files, then you can set `noclobber’ in your `.cshrc’ file. `noclobber’ can be overridden using the pling `!’ symbol. So
UNIX% set noclobber
UNIX% touch blah # create an empty file blah
UNIX% echo test >! blah
writes over the file `blah’ even though `noclobber’ is set.
Here are some other combinations of redirection symbols
`>>’
Append, including `stderr’
`>>!’
Append, ignoring `noclobber’
`>>&!’
Append `stdout’, `stderr’, ignore `noclobber’
`<<’
See below.
The last of these commands reads from the standard input until it finds a line which contains a word. It then feeds all of this input into the program concerned. For example,
nexus% mail mark < Hello mark
nexus 2> Nothing much to say…
nexus 2> so bye
nexus 2>
nexus 2> quit
Sending mail…
Mail sent!
The mail message contains all the lines up to, but not including `marker’. This method can also be used to print text verbatim from a file without using multiple echo commands. Inside a script one may write:
cat < myfile

pwd

This program changes the working directory to /etc and then executes a subshell which inside the brackets changes directory to /usr/bin and lists the files there. The output of this private shell are sent to a file `myfile’. At the end we print out the current working directory just to show that the `cd’ command in brackets had no effect on the main program.
Normally both parentheses must be on the same line. If a subshell command line gets too long, so that the brackets are not on the same line, you have to use backslash characters to continue the lines,

( command \
command \
command \
)

Tests and conditions
No programming language would be complete without tests and loops. C-shell has two kinds of decision structure: the `if..then..else’ and the `switch’ structure. These are closely related to their C counterparts. The syntax of these is
if (condition) command

if (condition) then
command
command..
else
command
command..
endif
switch (string)

case one:
commands
breaksw

case two:
commands
breaksw

…

endsw

In the latter case, no commands should appear on the same line as a `case’ statement, or they will be ignored. Also, if the `breaksw’ commands are omitted, then control flows through all the commands for case 2, case 3 etc, exactly as it does in the C programming language.
We shall consider some examples of these statements in a moment, but first it is worth listing some important tests which can be used in `if’ questions to find out information about files.
`-r file’
True if the file exists and is readable
`-w file’
True if the file exists and is writable
`-x file’
True if the file exists and is executable
`-e file’
True if the file simply exists
`-z file’
True if the file exists and is empty
`-f file’
True if the file is a plain file
`-d file’
True if the file is a directory
We shall also have need of the following comparison operators.
`==’
is equal to (string comparison)
`!=’
is not equal to
`>’
is greater than
`=’
is greater than or equal to
`<=’
is less than or equal to
`=~’
matches a wildcard
`!~’
does not match a wildcard
The simplest way to learn about these statements is to use them, so we shall now look at some examples.
#!/bin/csh -f
#
# Safe copy from to
#
#

if ($#argv != 2) then

echo “Syntax: copy ”
exit 0

endif

if ( -f $argv[2] ) then

echo “File exists. Copy anyway?”

switch ( $ Makefile

echo “Making software. Type CTRL-C to abort and edit Makefile”

$MAKE software # call make to build program
chmod 755 software # set correct protection

Loops in csh
The C-shell has three loop structures: `repeat’, `while’ and `foreach’. We have already seen some examples of the `foreach’ loop.
The structure of these loops is as follows
repeat number-of-times command

while ( test expression )

commands

end

foreach control-variable ( list-or-array )

commands

end

The commands `break’ and `continue’ can be used to break out of the loops at any time. Here are some examples.
repeat 2 echo “Yo!” | write mark
This sends the message “Yo!” to mark’s terminal twice.
repeat 5 echo `echo “Shutdown time! Log out now” | wall ; sleep 30` ; halt
This example repeats the command `echo Shutdown time…’ five times at 30 second intervals, before shutting down the system. Only the superuser can run this command! Note the strange construction with `echo echo’. This is to force the repeat command to take two shell commands as an argument. (Try to explain why this works for yourself.)
Input from the user

# Test a user response

echo “Answer y/n (yes or no)”

set valid = false

while ( $valid == false )

switch ( $ 6 && $x 6 || $x > 2 ) # Bitwise shift right
@ back = ( $var << 2 ) # Bitwise shift left
These operators are precisely those found in the C programming language.
Examples
The following script uses the operators in the last two sections to take a list of files with a given file extension (say `.doc’) and change it for another (say `.tex’). This is a partial solution to the limitation of not being able to do multiple renames in shell.
#!/bin/csh -f
#############################################################
#
# Change file extension for multiple files
#
#############################################################

if ($#argv < 2) then
echo Syntax: chext oldpattern newextension
echo “e.g: chext *.doc tex “
exit 0
endif

mkdir /tmp/chext.$user # Make a scratch area

set newext=”$argv[$#argv]“ # Last arg is new ext
set oldext=”$argv[1]:e”

echo “Old extension was ($oldext)”"
echo “New extension ($newext) — okay? (y/n)”

switch( $ $argv[2]
#
###########################################################
if (”`whoami`” != “root”) then
echo Permission denied
exit 0
endif

if ( $#argv 2 ) then
echo Usage: KILL username lowest-pid
exit 0
endif

if ( $argv[1] == “root”) then
echo No! Too dangerous — system will crash
exit 0
endif

############################################################
# Kill everything
############################################################

if ( $#argv == 1 ) then

set killarray = ( `ps aux | awk ‘{ if ($1 == user) \
{printf “%s “,$2}}’ user=$argv[1]` )

foreach process ($killarray)

kill -1 $process
kill -15 $process > /dev/null
kill -9 $process > /dev/null

if (”`kill -9 $process | egrep -e ‘No such process’`” == “”) then
echo “Warning – $process would not die – try again”
endif
end

#############################################################
# Start from a certain PID
#############################################################

else if ( $#argv == 2 ) then

set killarray = ( `ps aux | awk ‘{ if ($1 == user && $2 > uid) \
{printf “%s “,$2}}’ user=$argv[1] uid=$argv[2]` )

foreach process ($killarray)

kill -1 $process > /dev/null
kill -15 $process
sleep 2
kill -9 $process > /dev/null

if (”`kill -9 $process | egrep -e ‘No such process’`” == “”) then
echo “Warning – $process would not die – try again”
endif
end

endif

This program would be better written in C or Perl.
Summary: Limitations of shell programming
To summarize the last two long and oppressive chapters we shall take a step back from the details and look at what we have achieved.
The idea behind the shell is to provide a user interface, with access to the system’s facilities at a simple level. In the 70’s user interfaces were not designed to be user-friendly. The UNIX shell is not particularly use friendly, but it is very powerful. Perhaps it would have been enough to provide only commands to allow users to write C programs. Since all of the system functions are available from C, that would certainly allow everyone to do what anything that UNIX can do. But shell programming is much more immediate than C. It is an environment of frequently used tools. Also for quick programming solutions: C is a compiled language, whereas the shell is an interpreter. A quick shell program can solve many problems in no time at all, without having to compile anything.
Shell programming is only useful for `quick and easy’ programs. To use it for anything serious is an abuse. Programming difficult things in shell is clumsy, and it is difficult to get returned-information (like error messages) back in a useful form. Besides, shell scripts are slow compared to real programs since they involve starting a new program for each new command.
These difficulties are solved partly by Perl, which we shall consider next — but in the final analysis, real programs of substance need to be written in C. Contrary to popular belief, this is not more difficult than programming in the shell — in fact, many things are much simpler, because all of the shell commands originated as C functions. The shell is an extra layer of the UNIX onion which we have to battle our way through to get where we’re going.
Sometimes it is helpful to be shielded from low level details — sometimes it is a hindrance. In the remaining chapters we shall consider more involved programming needs.
Perl
So far, we have been looking at shell programming for performing fairly simple tasks. Now let’s extend the idea of shell programming to cover more complex tasks like systems programming and network communications. Perl is a language which was designed to retain the immediateness of shell languages, but at the same time capture some of the flexibility of C. Perl is an acronym for Practical extraction and report language. In this chapter, we shall not aim to teach Perl from scratch — the best way to learn it is to use it! Rather we shall concentrate on demonstrating some principles.
Sed and awk, cut and paste
One of the reasons for using Perl is that it is extremely good at textfile handling–one of the most important things for UNIX users, and particularly useful in connection with CGI script processing on the World Wide Web. It has simple built-in constructs for searching and replacing text, storing information in arrays and retrieving them in sorted form. All of the these things have previously been possible using the UNIX shell commands
sed
awk
cut
paste
but these commands were designed to work primarily in the Bourne shell and are a bit `awk’ward to use for all but the simplest applications.
`sed’
is a stream editor. It takes command line instructions, reads input from the stream stdin and produces output on stdout according to those instructions. `sed’ works line by line from the start of a textfile.
`awk’
is a pattern matching and processing language. It takes a textfile and reads it line by line, matching regular expressions and acting on them. `awk’ is powerful enough to have conditional instructions like `if..then..else’ and uses C’s `printf’ construction for output.
`cut’
Takes a line of input and cuts it into fields, separated by some character. For instance, a normal line of text is a string of words separated by spaces. Each word is a different field. `cut’ can be used, for instance, to pick out the third column in a table. Any character can be specified as the separator.
`paste’
is the logical opposite of cut. It concatenates @math{n} files, and makes each line in the file into a column of a table. For instance, `paste one two three’ would make a table in which the first column consisted of all lines in `one’, the second of all lines in `two’ and the third of all lines in `three’. If one file is longer than the others, then some columns have blank spaces.
Perl unifies all of these operations and more. It also makes them much simpler.
Program structure
To summarize Perl, we need to know about the structure of a Perl program, the conditional constructs it has, its loops and its variables. In the latest versions of Perl (Perl 5), you can write object oriented programs of great complexity. We shall not go into this depth, for the simple reason that Perl’s strength is not as a general programming language but as a specialized language for textfile handling. The syntax of Perl is in many ways like the C programming language, but there are important differences.
“ Variables do not have types. They are interpreted in a context sensitive way. The operators which acts upon variables determine whether a variable is to be considered a string or as an integer etc.
“ Although there are no types, Perl defines arrays of different kinds. There are three different kinds of array, labelled by the symbols `$’, `@’ and `%’.
“ Perl keeps a number of standard variables with special names e.g. `$_ @ARGV’ and `%ENV’. Special attention should be paid to these. They are very important!
“ The shell reverse apostrophe notation `command` can be used to execute UNIX programs and get the result into a Perl variable.
Here is a simple `structured hello world’ program in Perl. Notice that subroutines are called using the `&’ symbol. There is no special way of marking the main program — it is simply that part of the program which starts at line 1.
#!/local/bin/perl
#
# Comments
#

&Hello();
&World;

# end of main

sub Hello
{
print “Hello”;
}

sub World
{
print “World\n”;
}

The parentheses on subroutines are optional, if there are no parameters passed. Notice that each line must end in a semi-colon.
Perl variables
Scalar variables
In Perl, variables do not have to be declared before they are used. Whenever you use a new symbol, Perl automatically adds the symbol to its symbol table and initializes the variable to the empty string.
It is important to understand that there is no practical difference between zero and the empty string in perl — except in the way that you, the user, choose to use it. Perl makes no distinction between strings and integers or any other types of data — except when it wants to interpret them. For instance, to compare two variables as strings is not the same as comparing them as integers, even if the string contains a textual representation of an integer. Take a look at the following program.
#!/local/bin/perl
#
# Nothing!
#

print “Nothing == $nothing\n”;

print “Nothing is zero!\n” if ($nothing == 0);

if ($nothing eq “”)
{
print STDERR “Nothing is really nothing!\n”;
}

$nothing = 0;

print “Nothing is now $nothing\n”;
The output from this program is

Nothing ==
Nothing is zero!
Nothing is really nothing!
Nothing is now 0

There are several important things to note here. First of all, we never declare the variable `nothing’. When we try to write its value, perl creates the name and associates a NULL value to it i.e. the empty string. There is no error. Perl knows it is a variable because of the `$’ symbol in front of it. All scalar variables are identified by using the dollar symbol.
Next, we compare the value of `$nothing’ to the integer `0′ using the integer comparison symbol `==’, and then we compare it to the empty string using the string comparison symbol `eq’. Both tests are true! That means that the empty string is interpreted as having a numerical value of zero. In fact any string which does not form a valid integer number has a numerical value of zero.
Finally we can set `$nothing’ explicitly to a valid integer string zero, which would now pass the first test, but fail the second.
As extra spice, this program also demonstrates two different ways of writing the `if’ command in perl.
The default scalar variable.
The special variable `$_’ is used for many purposes in Perl. It is used as a buffer to contain the result of the last operation, the last line read in from a file etc. It is so general that many functions which act on scalar variables work by default on `$_’ if no other argument is specified. For example,
print;
is the same as
print $_;
Array (vector) variables
The complement of scalar variables is arrays. An array, in Perl is identified by the `@’ symbol and, like scalar variables, is allocated and initialized dynamically.

@array[0] = “This little piggy went to market”;
@array[2] = “This little piggy stayed at home”;

print “@array[0] @array[1] @array[2]“;

The index of an array is always understood to be a number, not a string, so if you use a non-numerical string to refer to an array element, you will always get the zeroth element, since a non-numerical string has an integer value of zero.
An important array which every program defines is
@ARGV
This is the argument vector array, and contains the commands line arguments by analogy with the C-shell variable `$argv[]‘.
Given an array, we can find the last element by using the `$#’ operator. For example,
$last_element = $ARGV[$#ARGV];
Notice that each element in an array is a scalar variable. The `$#’ cannot be interpreted directly as the number of elements in the array, as it can in the C-shell. You should experiment with the value of this quantity — it often necessary to add 1 or 2 to its value in order to get the behaviour one is used to in the C-shell.
Perl does not support multiple-dimension arrays directly, but it is possible to simulate them yourself. (See the Perl book.)
Special array commands
The `shift’ command acts on arrays and returns and removes the first element of the array. Afterwards, all of the elements are shifted down one place. So one way to read the elements of an array in order is to repeatedly call `shift’.
$next_element=shift(@myarray);
Note that, if the array argument is omitted, then `shift’ works on `@ARGV’ by default.
Another useful function is `split’, which takes a string and turns it into an array of strings. `split’ works by choosing a character (usually a space) to delimit the array elements, so a string containing a sentence separated by spaces would be turned into an array of words. The syntax is
@array = split; # works with spaces on $_
@array = split(pattern,string); # Breaks on pattern
($v1,$v2…) = split(pattern,string); # Name array elements with scalars
In the first of these cases, it is assumed that the variable `$_’ is to be split on whitespace characters. In the second case, we decide on what character the split is to take place and on what string the function is to act. For instance
@new_array = split(”:”,”name:passwd:uid:gid:gcos:home:shell”);
The result is a seven element array called `@new_array’, where `$new_array[0]‘ is `name’ etc.
In the final example, the left hand side shows that we wish to capture elements of the array in a named set of scalar variables. If the number of variables on the lefthand side is fewer than the number of strings which are generated on the right hand side, they are discarded. If the number on the left hand side is greater, then the remainder variables are empty.
Associated arrays
One of the very nice features of Perl is the ability to use one string as an index to another string in an array. For example, we can make a short encyclopedia of zoo animals by constructing an associative array in which the keys (or indices) of the array are the names of animals, and the contents of the array are the information about them.

$animals{”Penguin”} = “A suspicious animal, good with cheese crackers…”;
$animals{”dog”} = “Plays stupid, but could be a cover…”;

if ($index eq “fish”)
{
$animals{$index} = “Often comes in square boxes. Very cold.”;
}

An entire associated array is written `%array’, while the elements are `$array{$key}’.
Perl provides a special associative array for every program called `%ENV’. This contains the environment variables defined in the parent shell which is running the Perl program. For example
print “Username = $ENV{”USER”}\n”;

$ld = “LD_LIBRARY_PATH”;
print “The link editor path is $ENV{$ld}\n”;

To get the current path into an ordinary array, one could write,
@path_array= split(”:”,$ENV{”PATH”});
Array example program
Here is an example which prints out a list of files in a specified directory, in order of their UNIX protection bits. The least protected file files come first.
#!/local/bin/perl
#
# Demonstration of arrays and associated arrays.
# Print out a list of files, sorted by protection,
# so that the least secure files come first.
#
# e.g. arrays
# arrays *.C
#
############################################################

print “You typed in “,$#ARGV+1,” arguments to command\n”;

if ($#ARGV < 1)
{
print “That’s not enough to do anything with!\n”;
}

while ($next_arg = shift(@ARGV))
{
if ( ! ( -f $next_arg || -d $next_arg))
{
print “No such file: $next_arg\n”;
next;
}

($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size) = stat($next_arg);
$octalmode = sprintf(”%o”,$mode & 0777);

$assoc_array{$octalmode} .= $next_arg.
” : size (”.$size.”), mode (”.$octalmode.”)\n”;
}

print “In order: LEAST secure first!\n\n”;

foreach $i (reverse sort keys(%assoc_array))
{
print $assoc_array{$i};
}

Loops and conditionals
Here are some of the most commonly used decision-making constructions and loops in Perl. The following is not a comprehensive list — for that, you will have to look in the Perl bible: Programming Perl, by Larry Wall and Randal Schwartz. The basic pattern follows the C programming language quite closely. In the case of the `for’ loop, Perl has both the C-like version, called `for’ and a `foreach’ command which is like the C-shell implementation.
if (expression)
{
block;
}
else
{
block;
}

command if (expression);

unless (expression)
{
block;
}
else
{
block;
}

while (expression)
{
block;
}

do
{
block;
}
while (expression);

for (initializer; expression; statement)
{
block;
}

foreach variable(array)
{
block;
}

In all cases, the `else’ clauses may be omitted.
Strangely, perl does not have a `switch’ statement, but the Perl book describes how to make one using the features provided.
The for loop
The for loop is exactly like that in C or C++ and is used to iterate over a numerical index, like this:

for ($i = 0; $i < 10; $i++)
{
print $i, “\n”;
}

The foreach loop
The foreach loop is like its counterpart in the C shell. It is used for reading elements one by one from a regular array. For example,

foreach $i ( @array )
{
print $i, “\n”;
}

Iterating over elements in arrays
One of the main uses for `for’ type loops is to iterate over successive values in an array. This can be done in two ways which show the essential difference between for and foreach.
If we want to fetch each value in an array in turn, without caring about numerical indices, the it is simplest to use the foreach loop.

@array = split(” “,”a b c d e f g”);

foreach $var ( @array )
{
print $var, “\n”;
}

This example prints each letter on a separate line. If, on the other hand, we are interested in the index, for the purposes of some calculation, then the for loop is preferable.

@array = split(” “,”a b c d e f g”);

for ($i = 0; $i <= $#array; $i++)
{
print $array[$i], “\n”;
}

Notice that, unlike the for-loop idiom in C/C++, the limit is `$i <= $#array’, i.e. `less than or equal to’ rather than `less than’. This is because the `$#’ operator does not return the number of elements in the array but rather the last element.
Associated arrays are slightly different, since they do not use numerical keys. Instead they use a set of strings, like in a database, so that you can use one string to look up another. In order to iterate over the values in the array we need to get a list of these strings. The keys command is used for this.
$assoc{”mark”} = “cool”;
$assoc{”GNU”} = “brave”;
$assoc{”zebra”} = “stripy”;

foreach $var ( keys %assoc )
{
print “$var , $assoc{$var} \n”;
}
The order of the keys is not defined in the above example, but you can choose to sort them alphabetically by writing
foreach $var ( sort keys %assoc )
instead.
Iterating over lines in a file
Since Perl is about file handling we are very interested in reading files. Unlike C and C++, perl likes to read files line by line. The angle brackets are used for this, See section Files in perl. Assuming that we have some file handle `’, for instance `’, we can always read the file line by line with a while-loop like this.

while ($line = )
{
print $line;
}

Note that $line includes the end of line character on the end of each line. If you want to remove it, you should add a `chop’ command:

while ($line = )
{
chop $line;
print “line = ($line)\n”;
}

Files in perl
Opening files is straightforward in Perl. Files must be opened and closed using — wait for it — the commands `open’ and `close’. You should be careful to close files after you have finished with them — especially if you are writing to a file. Files are buffered and often large parts of a file are not actually written until the `close’ command is received.
Three files are, of course, always open for every program, namely `STDIN’, `STDOUT’and `STDERR’.
Formally, to open a file, we must obtain a file descriptor or file handle. This is done using `open’;
open (file_descrip,”Filename”);
The angular brackets `’ are used to read from the file. For example,
$line = ;
reads one line from the file associated with `file_descrip’.
Let’s look at some examples of filing opening. Here is how we can implement UNIX’s `cut’ and `paste’ commands in perl:
#!/local/bin/perl
#
# Cut in perl
#

# Cut second column

while ()
{
@cut_array = split;

print “@cut_array[1]\n”;
}

This is the simplest way to open a file. The empty file descriptor `’ tells perl to take the argument of the command as a filename and open that file for reading. This is really short for `while($_=)’ with the standard input redirected to the named file.
The `paste’program can be written as follows:
#!/local/bin/perl
#
# Paste in perl
#
# Two files only, syntax : paste file 1file2
#

open (file1,”@ARGV[0]“) || die “Can’t open @ARGV[0]\n”;
open (file2,”@ARGV[1]“) || die “Can’t open @ARGV[1]\n”;

while (($line1 = ) || ($line2 = ))
{
chop $line1;
chop $line2;

print “$line1 $line2\n”; # tab character between
}

Here we see more formally how to read from two separate files at the same time. Notice that, by putting the read commands into the test-expression for the `while’ loop, we are using the fact that `’ returns a non-zero (true) value unless we have reached the end of the file.
To write and append to files, we use the shell redirection symbols inside the `open’ command.
open(fd,”> filename”); # open file for writing
open(fd,”>> filename”); # open file for appending
We can also open a pipe from an arbitrary UNIX command and receive the output of that command as our input:
open (fd,”/bin/ps aux | “);
A simple perl program
Let us now write the simplest perl program which illustrates the way in which perl can save time. We shall write it in three different ways to show what the short cuts mean. Let us implement the `cat’ command, which copies files to the standard output. The simplest way to write this is perl is the following:

#!/local/bin/perl

while ()
{
print;
}

Here we have made heavy use of the many default assumptions which perl makes. The program is simple, but difficult to understand for novices. First of all we use the default file handle which means, take one line of input from a default file. This object returns true as long as it has not reached the end of the file, so this loop continues to read lines until it reaches the end of file. The default file is standard input, unless this script is invoked with a command line argument, in which case the argument is treated as a filename and perl attempts to open the argument-filename for reading. The print statement has no argument telling it what to print, but perl takes this to mean: print the default variable `$_’.
We can therefore write this more explicitly as follows:

#!/local/bin/perl

open (HANDLE,”$ARGV[1]“);

while ()
{
print $_;
}
Here we have simply filled in the assumptions explicitly. The command `’ now reads a single line from the named file-handle into the default variable `$_’. To make this program more general, we can eliminate the defaults entirely.

#!/local/bin/perl

open (HANDLE,”$ARGV[1]“);

while ($line=)
{
print $line;
}
== and `eq’
Be careful to distinguish between the comparison operator for integers `==’ and the corresponding operator for strings `eq’. These do not work in each other’s places so if you get the wrong comparison operator your program might not work and it is quite difficult to find the error.
chop
The command `chop’ cuts off the last character of a string. This is useful for removing newline characters when reading files etc. The syntax is
chop; # chop $_;

chop $scalar; # remove last character in $scalar
Perl subroutines
Subroutines are indicated, as in the example above, by the ampersand `&’ symbol. When parameters are passed to a Perl subroutine, they are handed over as an array called `@_’. Which is analogous to the `$_’ variable. Here is a simple example:
#!/local/bin/perl

$a=”silver”;
$b=”gold”;

&PrintArgs($a,$b);

# end of main

sub PrintArgs

{
($local_a,$local_b) = @_;

print “$local_a, $local_b\n”;
}

die – exit on error
When a program has to quit and give a message, the `die’ command is normally used. If called without an argument, Perl generates its own message including a line number at which the error occurred. To include your own message, you write
die “My message….”;
If the string is terminated with a `\n’ newline character, the line number of the error is not printed, otherwise Perl appends the line number to your string.
When opening files, it is common to see the syntax:
open (filehandle,”Filename”) || die “Can’t open…”;
The logical `OR’ symbol is used, because `open’ returns true if all goes well, in which case the right hand side is never evaluated. If `open’ is false, then die is executed. You can decide for yourself whether or not you think this is good programming style — we mention it here because it is common practice.
The stat() idiom
The UNIX library function stat() is used to find out information about a given file. This function is available both in C and in Perl. In perl, it returns an array of values. Usually we are interested in knowing the access permissions of a file. stat() is called using the syntax

@array = stat (”filename”);

or alternatively, using a named array

($device,$inode,$mode) = stat(”filename”);

The value returned in the mode variable is a bit-pattern, See section Protection bits. The most useful way of treating these bit patterns is to use octal numbers to interpret their meaning.
To find out whether a file is readable or writable to a group of users, we use a programming idiom which is very common for dealing with bit patterns: first we define a mask which zeroes out all of the bits in the mode string except those which we are specifically interested in. This is done by defining a mask value in which the bits we want are set to 1 and all others are set to zero. Then we AND the mask with the mode string. If the result is different from zero then we know that all of the bits were also set in the mode string. As in C, the bitwise AND operator in perl is called `&’.
For example, to test whether a file is writable to other users in the same group as the file, we would write the following.

$mask = 020; # Leading 0 means octal number

($device,$inode,$mode) = stat(”file”);

if ($mode & $mask)
{
print “File is writable by the group\n”;
}

Here the 2 in the second octal number means “write”, the fact that it is the second octal number from the right means that it refers to “group”. Thus the result of the if-test is only true if that particular bit is true. We shall see this idiom in action below.
Perl example programs
The passwd program and `crypt()’ function
Here is a simple implementation of the UNIX `passwd’ program in Perl.
#!/local/bin/perl
#
# A perl version of the passwd program.
#
# Note – the real passwd program needs to be much more
# secure than this one. This is just to demonstrate the
# use of the crypt() function.
#
#############################################################

print “Changing passwd for $ENV{’USER’} on $ENV{’HOST’}\n”;

system ’stty’,’-echo’;
print “Old passwd: “;

$oldpwd = ;
chop $oldpwd;

($name,$coded_pwd,$uid,$gid,$x,$y,$z,$gcos,$home,$shell)
= getpwnam($ENV{”USER”});

if (crypt($oldpwd,$coded_pwd) ne $coded_pwd)
{
print “\nPasswd incorrect\n”;
exit (1);
}

$oldpwd = “”; # Destroy the evidence!

print “\nNew passwd: “;

$newpwd = ;

print “\nRepeat new passwd: “;

$rnewpwd = ;

chop $newpwd;
chop $rnewpwd;

if ($newpwd ne $rnewpwd)
{
print “\n Incorrectly typed. Password unchanged.\n”;
exit (1);
}

$salt = rand();
$new_coded_pwd = crypt($newpwd,$salt);

print “\n\n$name:$new_coded_pwd:$uid:$gid:$gcos:$home:$shell\n”;

Example with `fork()’
The following example uses the `fork’ function to start a daemon which goes into the background and watches the system to which process is using the greatest amount of CPU time each minute. A pipe is opened from the BSD `ps’ command.
#!/local/bin/perl
#
# A fork() demo. This program will sit in the background and
# make a list of the process which uses the maximum CPU average
# at 1 minute intervals. On a quiet BSD like system this will
# normally be the swapper (long term scheduler).
#

$true = 1;
$logfile=”perl.cpu.logfile”;

print “Max CPU logfile, forking daemon…\n”;

if (fork())
{
exit(0);
}

while ($true)
{
open (logfile,”>> $logfile”) || die “Can’t open $logfile\n”;
open (ps,”/bin/ps aux |”) || die “Couldn’t open a pipe from ps !!\n”;

$skip_first_line = ;
$max_process = ;
close(ps);

print logfile $max_process;
close(logfile);
sleep 60;

($a,$b,$c,$d,$e,$f,$g,$size) = stat($logfile);

if ($size > 500)
{
print STDERR “Log file getting big, better quit!\n”;
exit(0);
}
}

Example reading databases
Here is an example program with several of the above features demonstrated simultaneously. This following program lists all users who have home directories on the current host. If the home area has sub-directories, corresponding to groups, then this is specified on the command line. The word `home’ causes the program to print out the home directories of the users.
#!/local/bin/perl
##################################################################
#
# allusers – list all users on named host, i.e. all
# users who can log into this machine.
#
# Syntax: allusers group
# allusers mygroup home
# allusers myhost group home
#
# NOTE : This command returns only users who are registered on
# the current host. It will not find users which cannot
# be validated in the passwd file, or in the named groups
# in NIS. It assumes that the users belonging to
# different groups are saved in subdirectories of
# /home/hostname.
#
##################################################################

&arguments();

die “\n” if ( ! -d “/home/$server” );

$disks = `/bin/ls -d /home/$server/$group`;

foreach $home (split(/\s/,$disks))
{
open (LS,”cd $home; /bin/ls $home |”) || die “allusers: Pipe didn’t open”;

while ()
{
$exists = “”;
($user) = split;
($exists,$pw,$uid,$gid,$qu,$cm,$gcos,$dir)=getpwnam($user);

if ($exists)
{
if ($printhomes)
{
print “$dir\n”;
}
else
{
print “$user\n”;
}
}
}
close(LS);
}

########################################################

sub arguments
{
$printhomes = 0;
$group = “*”;
$server = `/bin/hostname`;
chop $server;

foreach $arg (@ARGV)
{
if (substr($arg,0,1) eq “u”)
{
$group = $arg;
next;
}

if ($arg eq “home”)
{
$printhomes = 1;
next;
}

$server= $arg; #default is to interpret as a server.
}
}

Pattern matching and extraction
Perl has regular expression operators for identifying patterns. The operator

/regular expression/

returns true of false depending on whether the regular expression matches the contents of $_. For example

if (/perl/)
{
print “String contains perl as a substring”;
}

if (/(Sat|Sun)day/)
{
print “Weekend day….”;
}
The effect is rather like the grep command. To use this operator on other variables you would write:

$variable =~ /regexp/

Regular expression can contain parenthetic sub-expressions, e.g.

if (/(Sat|Sun)day (..)th (.*)/)
{
$first = $1;
$second = $2;
$third = $3;
}
in which case perl places the objects matched by such sub-expressions in the variables $1, $2 etc.
Searching and replacing text
The `sed’-like function for replacing all occurances of a string is easily implemented in Perl using
while ()
{
s/$search/$replace/g;
print output;
}
This example replaces the string inside the default variable. To replace in a general variable we use the operator `=~’, with syntax:
$variable =~ s/search/replace/
Here is an example of some of this operator in use. The following is a program which searches and replaces a string in several files. This is useful program indeed for making a change globally in a group of files! The program is called `file-replace’.
#!/local/bin/perl
##############################################################
#
# Look through files for findstring and change to newstring
# in all files.
#
##############################################################

#
# Define a temporary file and check it doesn’t exist
#

$outputfile = “tmpmarkfind”;
unlink $outputfile;

#
# Check command line for list of files
#

if ($#ARGV < 0)
{
die “Syntax: file-replace [file list]\n”;
}

print “Enter the string you want to find (Don’t use quotes):\n\n:”;
$findstring=;
chop $findstring;

print “Enter the string you want to replace with (Don’t use quotes):\n\n:”;
$replacestring=;
chop $replacestring;

#

print “\nFind: $findstring\n”;
print “Replace: $replacestring\n”;
print “\nConfirm (y/n) “;
$y = ;
chop $y;

if ( $y ne “y”)
{
die “Aborted — nothing done.\n”;
}
else
{
print “Use CTRL-C to interrupt…\n”;
}

#
# Now shift default array @ARGV to get arguments 1 by 1
#

while ($file = shift)
{
if ($file eq “file-replace”)
{
print “Findmark will not operate on itself!”;
next;
}

#
# Save existing mode of file for later
#

($dev,$ino,$mode)=stat($file);

open (INPUT,$file) || warn “Couldn’t open $file\n”;
open (OUTPUT,”> $outputfile”) || warn “Can’t open tmp”;

$notify = 1;

while ()
{
if (/$findstring/ && $notify)
{
print “Fixing $file…\n”;
$notify = 0;
}
s/$findstring/$replacestring/g;
print OUTPUT;
}

close (OUTPUT);

#
# If nothing went wrong (if outfile not empty)
# move temp file to original and reset the
# file mode saved above
#

if (! -z $outputfile)
{
rename ($outputfile,$file);
chmod ($mode,$file);
}
else
{
print “Warning: file empty!\n.”;
}
}
Similarly we can search for lines containing a string. Here is the grep program written in perl
#!/local/bin/perl
#
# grep as a perl program
#

# Check arguments etc

while ()
{
print if (/$ARGV[1]/);
}

The operator `/search-string/’ returns true if the search string is a substring of the default variable $_. To search an arbitrary string, we write
…. if (teststring =~ /search-string/);
Here teststring is searched for occurrances of search-string and the result is true if one is found.
In perl you can use regular expressions to search for text patterns. Note however that, like all regular expression dialects, perl has its own conventions. For example the dollar sign does not mean “match the end of line” in perl, instead one uses the `\n’ symbol. Here is an example program which illustrates the use of regular expressions in perl:
#!/local/bin/perl
#
# Test regular expressions in perl
#
# NB – careful with \ $ * symbols etc. Use ” quotes since
# the shell interprets these!
#

open (FILE,”regex_test”);

$regex = $ARGV[$#ARGV];

print “Looking for $ARGV[$#ARGV] in file…\n”;

while ()
{
if (/$regex/)
{
print;
}
}

#
# Test like this:
#
# regex ‘.*’ – prints every line (matches everything)
# regex ‘.’ – all lines except those containing only blanks
# (. doesn’t match ws/white-space)
# regex ‘[a-z]‘ – matches any line containing lowercase
# regex ‘[^a-z]‘ – matches any line containg something which is
# not lowercase a-z
# regex ‘[A-Za-z]‘ – matches any line containing letters of any kind
# regex ‘[0-9]‘ – match any line containing numbers
# regex ‘#.*’ – line containing a hash symbol followed by anything
# regex ‘^#.*’ – line starting with hash symbol (first char)
# regex ‘;\n’ – match line ending in a semi-colon
#

Try running this program with the test data on the following file which is called `regex_test’ in the example program.

# A line beginning with a hash symbol

JUST UPPERCASE LETTERS

just lowercase letters

Letters and numbers 123456

123456

A line ending with a semi-colon;

Line with a comment # COMMENT…

Example: convert mail to WWW pages
Here is an example program which you could use to automatically turn a mail message of the form
From: Newswire
To: Mail2html
Subject: Nothing happened

On the 13th February at kl. 09:30 nothing happened. No footprints
were found leading to the scene of a terrible murder, no evidence
of a struggle …. etc etc

into an html-file for the world wide web. The program works by extracting the message body and subject from the mail and writing html-commands around these to make a web page. The subject field of the mail becomes the title. The other headers get skipped, since the script searches for lines containing the sequence “colon-space” or `: ‘. A regular expression is used for this.
#!/local/bin/perl
#
# Make HTML from mail
#

&BeginWebPage();
&ReadNewMail();
&EndWebPage();

##########################################################

sub BeginWebPage

{
print “\n”;
print “\n”;
}

##########################################################

sub EndWebPage

{
print “\n”;
print “\n”;
}

##########################################################

sub ReadNewMail

{
while ()
{
if (/Subject:/) # Search for subject line
{
# Extract subject text…

chop;
($left,$right) = split(”:”,$_);
print “

$right

print;
}
}

Generate WWW pages automagically
The following program scans through the password database and build a standardized html-page for each user it finds there. It fills in the name of the user in each case. Note the use of the `< $outputfile”) || die “Can’t open $outputfile\n”;

&MakePage;

close (OUT);
}

####################################################################
# Level 1
####################################################################

sub MakePage

{
print OUT <<ENDMARKER;

$FullName{$user}’s Home Page

$FullName{$user}’s Home Page

Hi welcome to my home page. In case you hadn’t
got it yet my name is: $FullName{$user}…

I study at Høgskolen i Oslo.

ENDMARKER
}

Other supported functions
Perl has very many functions which come directly from the C library. To give a taster, a few are listed here. The Perl book contains a comprehensive description of these.
Fork
The standard UNIX fork command for spawning new processes.
Sockets
Support for network socket communication.
Directories
Directory opening and handling routines.
Databases
Reading from the password files and the host databases is supported through the standard C functions `getpasswdbyname’ etc. dressed up to look like Perl.
Crypt
The password encryption function.
Regexp
Regular expressions and pattern matching, search and replace functions as in `sed’.
Operators
Perl has the full set of C’s logical operators.
File testing
Tests from the shell like `if (-f file)’.
Here are some of the most frequently used functions
chmod
Change the file mode of a file. e.g. chmod 755,filename
chdir
Change the current working directory. e.g. chdir /etc
stat
Get info about permissions, ownership and type of a file.
open
Open a file for reading, `>’ writing, `|’ as a pipe.
close
Close an open file handle.
system
Execute a shell command as a child process. e.g. system “ls”;
split
Split a string variable into an array of elements, by searching for a special character (space or `:’ etc.) e.g. @array=split(”:”,$string).
rename
Rename a file. e.g. rename old name new-name
mkdir
Make a new directory. mkdir newdir
shift
Read the first element of an array and delete it, shifting all the array elements down by one. (e.g. $first=shift(@array);).
chop
Chops off the last character of a string. Often used for deleting the end-of-line character when reading from a file.
oct
Interprets a number as octal (converts to decimal). e.g. $decimal = oct(755);
kill
Send a kill signal to a list of processes. e.g. kill -9, pid1,pid2…
You should explore Perl’s possibilities yourself. Perl is a good alternative to the shell which has much of the power of C and is therefore ideal for simple and more complex system programming tasks. If you intend to be a system administrator for UNIX systems, you could do much worse than to read the Perl book and learn Perl inside out.
Summary
The Practical Extraction and Report Language is a powerful tool which goes beyond shell programming, but which retains much of the immediateness of shell programming in a more formal programming environment.
The success of Perl has led many programmers to use it exclusively. In the next section, I would like to argue that programming directly in C is not much harder. In fact it has advantages in the long run. The power of Perl is that it is as immediate as shell programming. If you are inexperienced, Perl is a little easier than C because many features are ready programmed into the language, but with time one also builds up a repertoire of C functions which can do the same tricks.
Exercises
1. Write a program which prints out all of its arguments alphabetically together with the first and the last, and the number of arguments.
2. Write a program which prints out the pathname of the home directory for a given user. The user’s login name should be given as an argument.
3. Write a program called `search-replace’ which looks for a given string in a list of files and replaces it with a new string. You should be able to specify a list of files using ordinary UNIX wildcards. e.g. `search-replace search-string replace-string *.text’. This is a dangerous operation! What if the user types the strings incorrectly? How can you may the program safer?
4. Write a program which opens a pipe from `ps’ and computes the total cpu-time used by each user. Print the results in order of maximum to minimum. Hint: use an associated array to store the information.
5. Write a program which forks and goes into the background. Make the program send you mail when some other user of your choice logs in. Use sleep to check only every few minutes.
6. Open a pipe from `find’ and collect statistics over how many files there are in all of your sub-directories.
Project
Write a program which checks the `sanity’ of your UNIX system.
1. Check that the password file /etc/passwd is not writable by general users.
2. Check that the processes `cron’ and `sendmail’ are running.
3. Check that, if the file `/etc/exports’ or `/etc/dfs/dfstab’ exists, the nfsd daemon is running.
4. Check that if the filesystem table `/etc/fstab’ (or its equivalent on non-BSD systems) contains NFS mounted filesystems, the `biod’ or `nfsiod’ daemon is running.
5. Check that the file `/etc/resolv.conf’ contains the correct domain name. It may or may not be the same as that returned by the shell command `domainname’. If it is not the same, you should print the message `NIS domain has different name to DNS domain’.
WWW and CGI programming
CGI stands for the Common Gateway Interface. It is the name given to scripts which can be executed from within pages of the world wide web. Although it is possible to use any language in CGI programs (hence the word `common’), the usual choice is Perl, because of the ease with which Perl can handle text.
The CGI interface is pretty unintelligent, in order to be as general as possible, so we need to do a bit of work in order to make scripts work.
Permissions
The key thing about the WWW which often causes a lot of confusion is that the W3 service runs with a user ID of `nobody’. The purpose of this is to ensure that nobody has the right to read or write files unless they are opened very explicitly by the user who owns them.
In order for files to be readable on the WWW, they must have file mode `644′ and they must lie in a directory which has mode `755′. In order for a CGI program to be executable, it must have permission `755′ and in order for such a program to write to a file in a user’s directory, it must be possible for the file to be created (if necessary) and everyone must be able to write to it. That means that files which are written to by the WWW must have mode `666′ and must either exist already or lie in a directory with permission `777′(6).
Protocols
CGI script programs communicate with W3 browsers using a very simple protocol. It goes like this:
“ A web page sends data to a script using the `forms’ interface. Those data are concatenated into a single line. The data in separate fields of a form are separated by `&’ signs. New lines are replaced by the text `%0D%0A’, which is the DOS ASCII representation of a newline, and spaces are replaced by `+’ symbols.
“ A CGI script reads this single line of text on the standard input.
“ The CGI script replies to the web browser. The first line of the reply must be a line which is tells the browser what mime-type the data are sent in. Usually, a CGI script replies in HTML code, in which case the first line in the reply must be:
“ Content-type: text/html
This must be followed by a blank line.
HTML coding of forms
To start a CGI program from a web page we use a form which is a part of the HTML code enclosed with the parentheses

…

The method `post’ means that the data which get typed into this form will be piped into the CGI program via its standard input. The `action’ specifies which program you want to start. Note that you cannot simply use the absolute path of the file, for security reasons. You must use something called a `script alias’ to tell the web browser where to find the program. If you do not have a script alias defined for you personally, then you need to get one from your system administrator. By using a script alias, no one from outside your site can see where your files are located, only that you have a `cgi-bin’ area somewhere on your system.
Within these parentheses, you can arrange to collect different kinds of input. The simplest kind of input is just a button which starts the CGI program. This has the form

This code creates a button. When you click on it the program in your `action’ string gets started. More generally, you will want to create input boxes where you can type in data. To create a single line input field, you use the following syntax:

This creates a single line text field of width 40 characters. This is not the limit on the length of the string which can be typed into the field, only a limit on the amount which is visible at any time. It is for visual formatting only. The NAME field is used to identify the data in the CGI script. The string you enter here will be sent to the CGI script in the form `variable-name=value of input…’. Another type of input is a text area. This is a larger box where one can type in text on several lines. The syntax is:

which means: create a text area of fifty rows by fifty columns with a prompt to the left of the box. Again, the size has only to do with the visual formatting, not to do with limits on the amount of text which can be entered.
As an example, let’s create a WWW page with a complete form which can be used to make a guest book, or order form.

Example form

Write in my guest book…

Please leave a comment using the form below.

Your Name/e-mail:

The reset button clears the form. When the submit button is pressed, the CGI program is activated.
Perl and the web
Interpreting data from forms
To interpret and respond to the data in a form, we must write a program which satisfies the protocol above, See section Protocols. We use perl as a script language. The simplest valid CGI script is the following:

#!/local/bin/perl

#
# Reply with proper protocol
#

print “Content-type: text/html\n\n”;

#
# Get the data from the form …
#

$input = ;

#
# … and echo them back
#

print $input, “\n Done! \n”;

Although rather banal, this script is a useful starting point for CGI programming, because it shows you just how the input arrives at the script from the HTML form. The data arrive all in a single, enormously long line, full of funny characters. The first job of any script is to decode this line.
Before looking at how to decode the data, we should make an important point about the protocol line. If a web browser does not get this `Content-type’ line from the CGI script it returns with an error:

500 Server Error

The server encountered an internal error or misconfiguration and was
unable to complete your request.

Please contact the server administrator, and inform them of the time
the error occurred, and anything you might have done that may have
caused the error.

Error: HTTPd: malformed header from script www/cgi-bin/comment.pl

Before finishing your CGI script, you will probably encounter this error several times. A common reason for getting the error is a syntax error in your script. If your program contains an error, the first thing a browser gets in return is not the `Content-type’ line, but an error message. The browser does not pass on this error message, it just prints the uninformative message above.
If you can get the above script to work, then you are ready to decode the data which are sent to the script. The first thing is to use perl to split the long line into an array of lines, by splitting on `&’. We can also convert all of the `+’ symbols back into spaces. The script now looks like this:
#!/local/bin/perl

#
# Reply with proper protocol
#

print “Content-type: text/html\n\n”;

#
# Get the data from the form …
#

$input = ;

#
# … and echo them back
#

print “$input\n\n\n”;

$input =~ s/\+/ /g;

#
# Now split the lines and convert
#

@array = split(’&’,$input);

foreach $var ( @array )
{
print “$var\n”;
}

print “Done! \n”;

We now have a series of elements in our array. The output from this script is something like this:

variable1=Mark+Burgess&variable2=%0D%0AI+just+called+to+say+ (wrap)
….%0D%0A…hey+pig%2C+nothing%27s+working+out+the+way+I+planned
variable1=Mark Burgess variable2=%0D%0AI just called to say (wrap)
….%0D%0A…hey pig%2Cnothing%27s working out the way I planned Done!

As you can see, all control characters are converted into the form `%XX’. We should now try to do something with these. Since we are usually not interested in keeping new lines, or any other control codes, we can simply null-out these with a line of the form

$input =~ s/%..//g;

The regular expression `%..’ matches anything beginning with a percent symbol followed by two characters. The resulting output is then free of these symbols. We can then separate the variable contents from their names by splitting the input. Here is the complete code:
#!/local/bin/perl

#
# Reply with proper protocol
#

print “Content-type: text/html\n\n”;

#
# Get the data from the form …
#

$input = ;

#
# … and echo them back
#

print “$input\n\n\n”;

$input =~ s/%..//g;

$input =~ s/\+/ /g;

@array = split(’&’,$input);

foreach $var ( @array )
{
print “$var
”;
}

print “

($name,$variable1) = split(”variable1=”,$array[0]);
($name,$variable2) = split(”variable2=”,$array[1]);

print “
var1 = $variable1
”;
print “
var2 = $variable2
”;

print “
Done! \n”;

and the output
variable1=Mark+Burgess&variable2=%0D%0AI+just+called+to+say (wrap)
+….%0D%0A…hey+pig%2C+nothing%27s+working+out+the+way+I+planned
variable1=Mark Burgess
variable2=I just called to say …….hey pig nothings working (wrap)
out the way I planned

var1 = Mark Burgess

var2 = I just called to say …….hey pig nothings working out (wrap)
the way I planned

Done!
A complete guestbook example in perl
Let us now use this technique to develop a guest book application. Based on the code above, analyze the following code.
#!/local/bin/perl
####################################################################
#
# Guest book
#
####################################################################

$guestbook_page = “/iu/nexus/ud/mark/www/tmp/cfguest.html”;

$tmp_page = “/iu/nexus/ud/mark/www/tmp/guests.tmp”;

$remote_host = $ENV{”REMOTE_HOST”};

print “Content-type: text/html\n\n”;
print “

\n”;
print “best wishes,

”;
print “-Mark

”;
print “Return to menu\n”;

$input = ;

$input =~ s/%..//g;

$input =~ s/\+/ /g;

@array = split(’&’,$input);

($skip,$name) = split(”var1=”,$array[0]);
($skip,$message) = split(”var2=”,$array[1]);

if (! open (PAGE, $guestbook_page))
{
print “Content-type: text/html\n\n”;
print “couldn’t open guestbook page file!”;
}

if (! open (TMP, “+>$tmp_page”))
{
print “Content-type: text/html\n\n”;
print “couldn’t open temporary output file!”;
}

while ($line = )
{
if ($line =~ /

Number of entries: (..)/)
{
$entry_no = $1;
$entry_no++;
$line = “

Number of entries: $entry_no

\n”;
}

if ($line =~ //)
{
$date = `date +”%A, %b %d %Y”`;
print TMP “Entry $date from host: $remote_host\n

\n”;
print TMP “From: $name\n

\n”;
print TMP $message;
print TMP “\n

print TMP “$line”;
}

close PAGE;
close TMP;

if (! rename ($tmp_page, $guestbook_page))
{
print “Oops! Rename operation failed!\n”;
}

chmod (0600, $guestbook_page);

This script works by reading through the old guest book file, opening a new copy of the guest book file and appending a new messages at the end. The end of the message section (not counting the `’ tags) is marked by a comment line.

Note that a provisional guest book file has to exist in the first place. The script writes to a new file and then swaps the new file for the old one. The guest book file looks something like this:

Comments

My guest book

Entry no. Wednesday, Feb 28 1996
from host: dax

From: Mark.Burgess@iu.hio.no

Just to start the ball rolling….

Entry no. Tuesday, Mar 26 1996
from host: enterprise.subspace.net

From: spock@enterprise

Registering a form of energy never before encountered.

Mark
Burgess – Mark.Burgess@iu.hio.no

The directory in which this file lies needs to be writable to the user nobody (the WWW user) and the files within need to be deletable by nobody but no one else. Some users try to make guest book scripts setuid-themselves in order to overcome the problem that httpd runs with uid nobody, but this opens many security issues. In short it is asking for trouble. Unfortunately an ordinary user cannot use chown in order to give access only to the WWW user nobody, so this approach needs the cooperation of the system administrator. Nevertheless this is the most secure approach. Try to work through this example step for step.
PHP and the web
The PHP language makes the whole business of web programming rather simpler than perl. It hides the business of translating variables from forms into new variables in a CGI program and it even allows you to embed active code into you HTML pages. PHP has special support for querying data in an SQL database like MySQL or Oracle. PHP documentation lives at @uref{http://www.php.net}.
Embedded PHP
PHP code can be embedded inside HTML pages provided your WWW server is configurered with PHP support. PHP code lives inside a tag with the general form

For example, we could use this to import one file into another and print out a table of numbers:

<?php

include “file.html”

for ($i = 0; $i < 10; $i++)
{
print “Counting $i
”;
}

?>

This makes it easy to generate WWW pages with a fixed visual layout:
<?php
#
# Standard layout
#

# Set $title, $comment and $contents

##########################################################################

print “\n”;
print “”;

print “

”$title

”;
print “$comment”;
print “

\n”;

include $contents;

print (”

\n”);
print (”\n”);
print (”\n”);

Variables are easily set by calling PHP code in the form of a CGI program from a form.
PHP and forms
PHP is particularly good at dealing with forms, as a CGI scripting language. Consider the following form:

Name:
Email:
Preferred language:

English
Norwegian
Gobbledigook

This produces a page into which one types a name and email address and chooses a language from a list of three possible choices. When the user clicks on a button marked by the file `image.gif’ the form is posted. Here is a program which unravels the data sent to the CGI program:

#!/local/bin/php

<?php
#
# A CGI program which handles a form
# Variables are translated automatically
#

$title = “This page title”;
$comment = “This pages talks about the following…..”;

##########################################################################

echo “”;
echo “

$title

”;
echo “$comment”;
echo “

\n”;

###

echo “Your name is $personal[name]

”;
echo “Your email is $personal[email]

”;

echo “Language options: “;
echo “
“;

for ($i = 0; strlen($language[$i]) > 0; $i++)
{
echo “

”;
}

if ($language[0] == “Norwegian”)
{
echo “Hei alle sammen

”;
}
else
{
echo “Greetings everyone, this page will be in English

”;
}

echo “

Variable language[$i] = $language[$i]

“;

###

echo (”

\n”);
echo (”\n”);
echo (”\n”);
?>

A complete PHP guestbook
If your web-server supports PHP there is no need for separate CGI-scripts handling form output. A single PHP-script can create the form and handle the output simultaneously. In addition this script can be placed wherever the web-server is able to read HTML files. PHP defines a special variable $PHP_SELF which provides the action= assignment of the form with the script itself. Moreover such PHP-scripts checks whether the user has submitted any data by checking if the variables of the form is set with the command isset(). The following code shows how easily a guestbook can be made using PHP compared to the Perl-code shown in a previous section. See section A complete guestbook example in perl.

My Guestbook

Welcome to my Guestbook

Please write me a little note below

<form action=”” method=”POST”>

<?php
$file = “/iu/nexus/ud/haugerud/www/cgi-out/guestbook.txt”;
if(isset($note))
{
$date = date(”F j, Y, G:i”);
$buffer = “

Message sent from IP-address $REMOTE_ADDR

\n”;
$buffer .= “

$date

\n”;
$buffer .= nl2br($note).’
’;

$handle = fopen ($file, “r”);
while (!feof($handle))
{
$buffer .= fread($handle,4096);
}
fclose($handle);

$outhandle=fopen ($file,”w”);
fputs($outhandle,$buffer);
fclose($outhandle);
}
?>

The entries so far:

C programming
This section is not meant to teach you C. It is a guide to using C in UNIX and it is assumed that you have a working knowledge of the language. See the GNU C-Tutorial for an introduction to basics.
Shell or C?
In the preceding chapters we have been looking at ways to get simple programming tasks done. The immediateness of the script languages is a great advantage when we just want to get a job done as quickly as possible. Scripts lend themselves to simple system administration tasks like file processing, but they do not easily lend themselves to more serious programs.
Although some system administrators have grown to the idea that shell programming is easier, I would argue that this is not really true. First of all, most of the UNIX shell commands are just wrapper programs for C function calls. Why use the wrapper when you can use the real thing? Secondly, the C function calls return data in pointers and structures which are very easy to manipulate, whereas piping the output of shell programs into others can be a very messy and awkward way of working. Here are some of the reasons why we also need a more traditional programming language like C.
1. The shell languages do not allow us to create an acceptable user-interface, like X-windows, or the curses (cursor manipulation) library. They are mainly intended for file-processing. (Though recently the Tk library has provided a way of creating user interfaces in Tcl and Perl.)
2. Shell commands read their input line-by-line. Not all input is generated in this simple way — we also need to be able to read through lines i.e. the concept of a data stream.
3. More advanced data structures are needed for most applications, such as linked lists and binary trees, acyclic graphs etc.
4. Compilers help to sort out simple typographical and logical errors by compile-time checking source code.
5. Compiled code is faster than interpreted code.
6. Many tools have been written to help in the programming of C code (dbx, lex, yacc etc.).
C program structure
The form of a C program
A C program consists of a set of function, beginning with the main program:

main () /* This is a comment */

{

Commands …

}

The source code of a C program can be divided into several text files. C compiles all functions separately; the linker ld joins them all up at the end. This means that we can plan out a strategy for writing large programs in a clear and efficient manner.
NOTE: C++ style comments `//…’ are not allowed by most C compilers.
Macros and declarations
Most UNIX systems now have ANSI C compatible compilers, but this has not always been the case. Most UNIX programs written in a version of C which is older than the ANSI standard, so you will need an appreciation of old Kernighan and Ritchie C conventions for C programming. See for example my C book.
An obvious difference between ANSI C and K&R C is that the C++ additions to the language are not included. Here are some useful points to remember.
“ K&R C does not allow `const’ data, it uses the C preprocessor with `#define’ instead. i.e. instead of
“
“ const int blah = 1;
“
use
#define blah 1
Remember that the hash symbol `#’ must be the first character on a line under UNIX.
“ K&R C doesn’t use function prototypes or declarations of the form:
“
“ void function (char *string, int a, int b)
“
“ {
“ }
“
Instead one writes:

void function (string, a, b)

char *string;
int a,b;

{
}

Several files
Most UNIX programs are very large and are split up into many files. Remember, when you split up programs into several files, you must declare variables as `extern’ in file A if they are really declared in file B. in which you want to use them. This tells the compiler that it should not try to create local storage for the variable, because this was already done in another file.
A note about UNIX system calls and standards
Most of the system calls in UNIX return data in the form of `struct’ variables. Sometimes these are structures used by the operating system itself — in other cases they are just put together so that programmers can handle a packet of data in a convenient way.
If in doubt, you can find the definitions of these structures in the relevant include files under `/usr/include’.
Since UNIX comes in many flavours the system calls are not always compatible and may have different options and arguments. Because of this there is a number of standardizing organizations for UNIX. One of them is POSIX which is an organization run by the major UNIX vendors. Programs written for UNIX are now expected to be POSIX compliant. This is not something you need to think about at the level of this course, but you should certainly remember that there exist programming standards and that these should be adhered to. The aim is to work towards a single standard UNIX.
Compiling: `cc’, `ld’ and `a.out’
The C compiler on the UNIX system is traditionally called `cc’ and has always been a traditional part of every UNIX environment. Recently several UNIX vendors have stopped including the C compiler as a part of their operating systems and instead sell a compiler separately. Fortunately there is a public domain Free Software version of the compiler called `gcc’ (the GNU C compiler). We shall use this in all the examples.
To compile a program consisting of several files of code, we first compile all of the separate pieces without trying to link them. There are therefore two stages: first we turn `.c’ files into `.o’ files. This compiles code but does not fix any address references. Then we link all `.o’ files into the final executable, including any libraries which are used.
Let’s suppose we have files `a.c’, `b.c’ and `c.c’. We write:

gcc -c a.c b.c c.c

This creates files `a.o’, `b.o’ and `c.o’. Next we link them into one file called `myprog’.

gcc -o myprog a.o b.o c.o

If the naming option `-o myprog’ is not used, the link `ld’ uses the default name a.out for the executable file.
Libraries and `LD_LIBRARY_PATH’
The resulting file is called `myprog’ and includes references only to the standard library `libc’. If we wish to link in the math library `libm’ or the cursor movement library `libcurses’ — or in general, a library called `libBLAH’ , we need to use the `-l’ directive.

gcc -o myprog files.o -lm -lcurses -lBLAH

The compiler looks for a suitable library in all of the directories listed in the environment variable `LD_LIBRARY_PATH’. Alternatively we can add a directory to the search path by using the `-L’. option:

gcc -o myprog files.o -L/usr/local/lib -lm -lcurses -lBLAH

Include files
Normally the compiler looks for include files only in the directory `/usr/include’. We can add further paths to search using the `-I’ option.

gcc -o myprog file.c -I/usr/local/include -I/usr/local/X11/include

Previously, UNIX libraries have been in `a.out’ code format, but recent releases of UNIX have gone over to a more efficient and flexible format called ELF (executable and linking format).
Shared and static libraries
Libraries are collections of C functions which the operating system creators have written for our convenience. The source code for such a library is just the source for a collection of functions — there is no main program.
There are two kinds of library used by modern operating systems: archive libraries or static libraries and shared libraries or dynamical libraries. An archive library has a name of the form

libname.a

When an archive library is linked to a program, it is appended lock, stock and barrel to the program code. This uses a lot of disk space and makes the size of the compiled program very large. Shared libraries (shared objects `so’ or shared archives `sa’ generally have names of the form)

libname.so
libname.sa

often with version numbers appended. When a program is linked with a shared library the code is not appended to the program. Instead pointers to the shared objects are created and the library is loaded at runtime, thus avoiding the problem of having to store the library effectively multiple times on the disk.
To make an archive library we compile all of the functions we wish to include in the library

gcc -c function1.c function2.c …

and then join the files using the `ar’ command.

ar rcv libMYLIB.a function1.o
ar rcv libMYLIB.a function2.o

To make a shared library one provides an option to the linker program. The exact method is different in different operating systems, so you should look at the manual page for ld on your system. Under SunOS 4 we take the object files `*.o’ and run

ld -o libMYLIB.so.1.1 -assert pure-text *.o

Under HPUX, we write

ld -b -o libMYLIB.so.1.1 *.o

With the GNU linker, you write

ld -shared -o libMYLIB.so.1.1 *.o

NOTE: when you add a shared library to the system under SunOS or GNU/Linux you must run the command `ldconfig’, making sure that the path to the library is included in `LD_LIBRARY_PATH’. SunOS and GNU/Linux use a cache file `/etc/ld.so.cache’ to keep current versions of libraries. GNU/Linux also uses a configuration file called `/etc/ld.so.conf’.
Knowing about important paths: directory structure
It is important to understand how the C compiler finds the files it needs. We have already mentioned the `-I’ and `-L’ options to the compilation command line. In general, all system include files can be found in the directory `/usr/include’ and subdirectories of this directory. All system libraries can be found in `/usr/lib’.
Many packages build their own libraries and keep the relevant files in separate directories so that if the system gets reinstalled, they do not get deleted. This is true for example of the X-windows system. The include and library files for this are typically kept in directories which look something like `/usr/local/X11R5/include’ and `/usr/X11R6/lib’. That means that we need to give all of this information to the compiler. Compiling a program becomes a complicated task in many cases so we need some kind of script to help us perform the task. The UNIX tool make was designed for this purpose.
Make
Nowadays compilers are often sold with fancy user environments driven by menus which make it easier to compile programs. UNIX has similar environments but all of them use shell-based command line compilation beneath the surface. That is because UNIX programmers are used to writing large and complex programs which occupy many directories and subdirectories. Each directory has to be adapted or configured to fit the particular flavour of UNIX system it is being compiled upon. Interactive user environments are very poor at performing this kind of service. UNIX solves the problem of compiling enormous trees of software (such as the UNIX system itself!) by using a compilation language called `make’. Such language files can be generated automatically by scripts, allowing very complex programs to configure and compile themselves from a single control script.
Compiling large projects
Typing lines like

cc -c file1.c file2.c …
cc -o target file1.o ….

repeatedly to compile a complicated program can be a real nuisance. One possibility would therefore be to keep all the commands in a script. This could waste a lot of time though. Suppose you are working on a big project which consists of many lines of source code — but are editing only one file. You really only want to recompile the file you are working on and then relink the resulting object file with all of the other object files. Recompiling the other files which hadn’t changed would be a waste of time. But that would mean that you would have to change the script each time you change what you need to compile.
A better solution is to use the `make’ command. `make’ was designed for precisely this purpose. To use `make’, we create a file called `Makefile’ in the same directory as our program. `make’ is a quite general program for building software. It is not specifically tied to the C programming language— it can be used in any programming language.
A `make’ configuration file, called a `Makefile’, contains rules which describe how to compile or build all of the pieces of a program. For example, even without telling it specifically, make knows that in order to go from `prog.c’ to `prog.o’ the command `cc -c prog.c’ must be executed. A Makefile works by making such associations. The Makefile contains a list of all of the files which compose the program and rules as to how to get to the finished product from the source.
The idea is that, to compile a program, we just have to type make. `make’ then reads the Makefile and compiles all of the parts which need compiling. It does not recompile files which have not changed since the last compilation! How does it do this? `make’ works by comparing the time-stamp on the file it needs to create with the time-stamp on the file which is to be compiled. If the compiled version exists and is newer than its source then the source does not need to be recompiled.
To make this idea work in practice, `make’ has to know how to go through the steps of compiling a program. Some default rules are defined in a global configuration file, e.g.

/usr/include/make/default.mk

Let’s consider an example of what happens for the the three files `a.c’, `b.c’ and `c.c’ in the example above — and let’s not worry about what the Makefile looks like yet.
The first time we compile, only the `.c’ files exist. When we type `make’, the program looks at its rules and finds that it has to make a file called `myprog’. To make this it needs to execute the command

gcc -o myprog a.o b.o c.o

So it looks for `a.o’ etc and doesn’t find them. It now goes to a kind of subroutine and looks to see if it has any rules for making files called `.o’ and it discovers that these are made by compiling with the `gcc -c’ option. Since the files do not exist, it does this. Now the files `a.o b.o c.o’ exist and it jumps back to the original problem of trying to make `myprog’. All the files it needs now exist and so it executes the command and builds `myprog’.
If we now edit `a.c’, and type `make’ once again — it goes through the same procedure as before but now it finds all of the files. So it compares the dates on the files — if the source is newer than the result, it recompiles.
By using this recursive method, `make’ only compiles those parts of a program which need compiling.
Makefiles
To write a Makefile, we have to tell `make’ about dependencies. The dependencies of a file are all of those files which are required to build it. Thus, the dependencies of `myprog’ are `a.o’, `b.o’ and `c.o’. The dependencies of `a.o’ are simply `a.c’, the dependencies of `b.o’ are `b.c’ and so on.
A Makefile consists of rules of the form:
target : dependencies
TAB rule;

The target is the thing we want to build, the dependencies are like subroutines to be executed first if they do not exist. Finally the rule is to be executed if all if the dependencies exist; it takes the dependencies and turns them into the target. There are two important things to remember:
“ The file names must start on the first character of a line.
“ There must be a TAB character at the beginning of every rule or action. If there are spaces instead of tabs, or no tab at all, `make’ will signal an error. This bizarre feature can cause a lot of confusion.
Let’s look at an example Makefile for a program which consists of two course files `main.c’ and `other.c’ and which makes use of a library called `libdb’ which lies in the directory `/usr/local/lib’. Our aim is to build a program called database:

#
# Simple Makefile for `database’
#

# First define a macro

OBJ = main.o other.o

CC = gcc
CFLAGS = -I/usr/local/include
LDFLAGS = -L/usr/local/lib -ldb
INSTALLDIR = /usr/local/bin

#
# Rules start here. Note that the $@ variable becomes the name of the
# executable file. In this case it is taken from the ${OBJ} variable
#

database: ${OBJ}
${CC} -o $@ ${OBJ} ${LDFLAGS}

#
# If a header file changes, normally we need to recompile everything.
# There is no way that make can know this unless we write a rule which
# forces it to rebuild all .o files if the header file changes…
#

${OBJ}: ${HEADERS}

#
# As well as special rules for special files we can also define a
# “suffix rule”. This is a rule which tells us how to build all files
# of a certain type. Here is a rule to get .o files from .c files.
# The $< variable is like $? but is only used in suffix rules.
#

.c.o:
${CC} -c ${CFLAGS} $<

#######################################################################
# Clean up
#######################################################################

#
# Make can also perform ordinary shell command jobs
# “make tidy” here performs a cleanup operation
#

clean:
rm -f ${OBJ}
rm -f y.tab.c lex.yy.c y.tab.h
rm -f y.tab lex.yy
rm -f *% *~ *.o
rm -f mconfig.tab.c mconfig.tab.h a.out
rm -f man.dvi man.aux man.log man.toc
rm -f cfengine.tar.gz cfengine.tar cfengine.tar.Z
make tidy
rm -f cfengine

install: ${INSTALLDIR}/database
cp database ${INSTALLDIR}/database

The Makefile above can be invoked in several ways.

make
make database
make clean
make install

If we simple type `make’ i.e. the first of these choices, `make’ takes the first of the rules it finds as the object to build. In this case the rule is `database’, so the first two forms above are equivalent.
On the other hand, if we type
make clean
then execution starts at the rule for `clean’, which is normally used to remove all files except the original source code. Make `install’ causes the compiled program to be installed at its intended destination.
`make’ uses some special variables (which resemble the special variables used in Perl — but don’t confuse them). The most useful one is `$@’ which represents the current target — or the object which `make’ would like to compile. i.e. as `make’ checks each file it would like to compile, `$@’ is set to the current filename.
$@
This evaluates to the current target i.e. the name of the object you are currently trying to build. It is normal to use this as the final name of the program when compiling
$?
This is used only outside of suffix rules and means the name of all the files which must be compiled in order to build the current target.
target: file1.o file2.o
TAB cc -o $@ $?
$<
This is only used in suffix rules. It has the same meaning as `$?’ but only in suffix rules. It stands for the pre-requisite, or the file which must be compiled in order to make a given object.
Note that, because `make’ has some default rules defined in its configuration file, a single-file C program can be compiled very easily by typing
make filename.c
This is equivalent to
cc -c filename.c
cc -o filename filename.o
New suffix rules for C++
Standard rules for C++ are not often built into UNIX systems at the time of writing, but we can create them in our own Makefiles very easily. Here we shall use the GNU compiler g++’s conventions for C++ files. Here is a sample Makefile for using C++. Note that the `.SUFFIXES’ command must be used to declare new endings or file extensions.
##################################################################
#
# This is the Makefile for g++
#
##################################################################

OBJ = cpp-prog.o X.o Y.o Z.o

CCPLUS = g++

.SUFFIXES: .C .o .h

#
# Program Rules
#

filesys: ${OBJ}
$(CCPLUS) -o filesys $(OBJ)

#
# Extra dependencies on the header file
# (if the header file changes, we need to rebuild *.o)
#

cpp-prog.o: filesys.h
X.o: filesys.h
Y.o: filesys.h
Z.o: filesys.h

#
# Suffix rules
#

.C.o:
$(CCPLUS) -c $<

The general rule here tells make that a `.o’ file can be created from a `.C’ file by executing the command `$(CCPLUS) -c’. (This is identical to the C case, except for the name of the compiler). The extra dependencies tell make that, if we change the header file `filesys.h’, then we must recompile all the files which read in `filesys.h’, since this could affect all of these. Finally, the highest level rule says that to make `filesys’ from the `.o’ files, we have to run `$(CCPLUS) -o filesys *.o’.
The argv, argc and envp parameters
When we write C programs which reads command line arguments, they are fed to us as an array of strings called the argument vector. The mechanisms for the C-shell and Perl are derived from the C argument vector. To read in the command line, we write

main (argc,argv,envp)

int argc;
char *argv[], *envp[];

{
printf (”The first argument was %s\n”,argv[1]);
}
Argument zero is the name of the program itself and `argv[argc-1]‘ is the last argument. The above definitions are in Kernighan and Ritchie C style. In ANSI C, the arguments can be declared using prototype:
main (int argc, char **argv)

{

}
The array of strings `envp[]‘ is a list of values of the environment variables of the system, formatted by
NAME=value
This gives C programmers access to the shell’s global environment.
Environment variables in C
In addition to the `envp’ vector, it is possible to access the environment variables through the call `getenv()’. This is used as follows; suppose we want to access the shell environment variable `$HOME’.
char *string;

string = getenv(”HOME”);
`string’ is now a pointer to static but public data. You should not use `string’ as if it were you’re own property because it will be used again by the system. Copy it’s contents to another string before using the data.
char buffer[500];

strcpy (buffer,string);
Files and directories
All of the regular C functions from the standard library are available to UNIX programmers. The standard functions only address the issue of reading and writing to files however, they do not deal with operating system specific attributes such as file permissions and file types. Nor is there a mechanisms for obtaining lists of files within a directory. The reason for these omissions is that they are operating system dependent. To find out about these other attributes POSIX describes some standard UNIX system calls.
opendir, readdir
Files and directories are handled by functions defined in the header file `dirent.h’. In earlier UNIX systems the file `dir.h’ was used — and the definitions were slightly different, but not much. To get a list of files in a directory we must open the directory and read from it — just like a file. (A directory is just a file which contains data on its entries). The commands are
opendir
closedir
readdir
See the manual pages for dirent. These functions return pointers to a dirent structure which is defined in the file `/usr/include/dirent.h’. Here is an example ls command which lists the contents of the directory `/etc’. This header defines a structure

struct dirent
{
off_t d_off; /* offset of next disk dir entry */
unsigned long d_fileno; /* file number of entry */
unsigned short d_reclen; /* length of this record */
unsigned short d_namlen; /* length of string in d_name */
char d_name[255+1]; /* name (up to MAXNAMLEN + 1) */
};

which can be used to obtain information from the directory nodes.

#include
#include

main ()

{ DIR *dirh;
struct dirent *dirp;
static char mydir[20] = “/etc”;

if ((dirh = opendir(mydir)) == NULL)
{
perror(”opendir”);
return;
}

for (dirp = readdir(dirh); dirp != NULL; dirp = readdir(dirh))
{
printf(”Got dir entry: %s\n”,dirp->d_name);
}

closedir(dirh);
}

Notice that reading from a directory is like reading from a file with fgets(), but the entries are filenames rather than lines of text.
stat ()
To determine the file properties or statistics we use the function call `stat()’ or its corollary `lstat()’. Both these functions find out information about files (permissions, owner, filetype etc). The only difference between them is the way in which they treat symbolic links. If `stat’ is used on a symbolic link, it stats the file the link points to rather than the link itself. If `lstat’ is used, the data refer to the link. Thus, to detect a link, we must use `lstat’, See section lstat and readlink.
The data in the `stat’ structure are defined in the file `/usr/include/sys/stat.h’. Here are the most important structures.

struct stat
{
dev_t st_dev; /* device number*/
ino_t st_ino; /* file inode */
mode_t st_mode; /* permission */
short st_nlink; /* Number of hardlinks to file */
uid_t st_uid; /* user id */
gid_t st_gid; /* group id */
dev_t st_rdev;
off_t st_size; /* size in bytes */
time_t st_atime; /* time file last accessed */
time_t st_mtime; /* time file contents last modified */
time_t st_ctime; /* time last attribute change */
long st_blksize;
long st_blocks;
};

lstat and readlink
The function `stat()’ treats symbolic links as though they were the files they point to. In other words, if we use `stat()’ to read a symbolic link, we end up reading the file the link points to and not the link itself— we never see symbolic links. To avoid this problem, there is a different version of the stat function called `lstat()’ which is identical to `stat()’ except that it treats links as links and not as the files they point to. This means that we can test whether a file is a symbolic link, only if we use `lstat()’. (See the next paragraph.)
Once we have identified a file to be a symbolic link, we use the `readlink()’ function to obtain the name of the file the link points to.

#define bufsize 512
char buffer[bufsize];

readlink(”/path/to/file”,buffer,bufsize);

The result is returned in the string buffer.
stat() test macros
As we have already mentioned, the UNIX mode bits contain not only information about what permissions a file has, but also bits describing the type of file — whether it is a directory or a link etc. There are macros defined in UNIX to extract this information from the `st_mode’ member of the `stat’ structure. They are defined in the `stat.h’ headerfile. Here are some examples.
#define S_ISBLK(m) /* is block device */
#define S_ISCHR(m) /* is character device */
#define S_ISDIR(m) /* is directory */
#define S_ISFIFO(m) /* is fifo pipe/socket */
#define S_ISREG(m) /* is regular (normal) file */

#define S_ISLNK(m) /* is symbolic link */ /* Not POSIX */
#define S_ISSOCK(m) /* is a lock */

#define S_IRWXU /* rwx, owner */
#define S_IRUSR /* read permission, owner */
#define S_IWUSR /* write permission, owner */
#define S_IXUSR /* execute/search permission, owner */
#define S_IRWXG /* rwx, group */
#define S_IRGRP /* read permission, group */
#define S_IWGRP /* write permission, grougroup */
#define S_IXGRP /* execute/search permission, group */
#define S_IRWXO /* rwx, other */
#define S_IROTH /* read permission, other */
#define S_IWOTH /* write permission, other */
#define S_IXOTH /* execute/search permission, other */

These return true or false when acting on the mode member. Here is an example See section Example filing program.

struct stat statvar;

stat(”file”,&statvar);

/* test return values */

if (S_ISDIR(statvar.st_mode))
{
printf(”Is a directory!”);
}

Example filing program
The following example program demonstrates the use of the directory functions in dirent and the stat function call.
/********************************************************************/
/* */
/* Reading directories and `statting’ files */
/* */
/********************************************************************/

#include
#include
#include
#include

#define DIRNAME “/.”
#define bufsize 255

/********************************************************************/

main ()

{ DIR *dirh;
struct dirent *dirp;
struct stat statbuf;
char *pathname[bufsize];
char *linkname[bufsize];

if ((dirh = opendir(DIRNAME)) == NULL)
{
perror(”opendir”);
exit(1);
}

for (dirp = readdir(dirh); dirp != NULL; dirp = readdir(dirh))
{
if (strcmp(”.”,dirp->d_name) == 0 || strcmp(”..”,dirp->d_name) == 0)
{
continue;
}

if (strcmp(”lost+found”,dirp->d_name) == 0)
{
continue;
}

sprintf(pathname,”%s/%s”,DIRNAME,dirp->d_name);

if (lstat(pathname,&statbuf) == -1) /* see man stat */
{
perror(”stat”);
continue;
}

if (S_ISREG(statbuf.st_mode))
{
printf(”%s is a regular file\n”,pathname);
};

if (S_ISDIR(statbuf.st_mode))
{
printf(”%s is a directory\n”,pathname);
}

if (S_ISLNK(statbuf.st_mode))
{
bzero(linkname,bufsize); /* clear string */
readlink(pathname,linkname,bufsize);
printf(”%s is a link to %s\n”,pathname,linkname);
}

printf(”The mode of %s is %o\n\n”,pathname,statbuf.st_mode & 07777);
}

closedir(dirh);
}

Process control, fork(), exec(), popen() and system
There is a number of ways in which processes can interact with one another and in which we can control their behaviour. We shall not go into great detail in this course, only provide examples for reference.
The UNIX `fork()’ function is used to create child processes. This is the basis of all `heavyweight’ multitasking under UNIX. Here is a simple example of fork in which we start a child process from within a program and wait for it to finish. Note that the code for the parent and the child is is the same file. The only thing that distinguishes parent from child is the value returned by the fork function.
When `fork()’ is called, it duplicates the entire current process so that two parallel processes are then running. The only difference between these is that the child process (the copy) gets a return value of zero from `fork()’, whereas the parent gets a return value equal to the process identifier (pid) of the child. This value can be used by the parent to send messages or to wait for the child. Here we show a simple example in which the `wait(NULL)’ command is used to wait for the last child spawned by the parent.
/**************************************************************/
/* */
/* A brief demo of the UNIX process duplicator fork(). */
/* */
/**************************************************************/

#include

/***************************************************************/

main ()

{ int pid, cid;

pid = getpid();

printf (”Fork demo! I am the parent (pid = %d)\n”,pid);

if (! fork())
{
cid = getpid();
printf (”I am the child (cid = %d) of (pid=%d)\n”,cid,pid);
ChildProcess();
exit(0);
}

printf(”Parent waiting here for the child…\n”);

wait(NULL);

printf(”Child finished, parent quitting too!\n”);
}

/**************************************************************/

ChildProcess()

{ int i;

for (i = 0; i < 10; i++)
{
printf (”%d…\n”,i);
sleep(1);
}
}

Another possibility is that we might want to execute a program and wait to find out what the result of the program is before continuing. There are two ways to do this. The first is a variation on the theme above and uses fork().
Let’s create a function which runs a shell command from within a C program, and determines its return value. We make the result a boolean (integer) value, so that the function returns `true’ if the shell command exits normally, See section Return codes.
if (ShellCommandReturnsZero(shell-command))
{
printf (”Command %s went ok\n”,shell-command);
}

To do this we first have to fork a new process and then use one of the exec commands to load a new code image on top of the new process. shell commands from C This sounds complicated, but it is necessary because of the way UNIX handles processes. If we had no use for the return value, we could simply execute a shell command using the system(”shell command”) function, (which does all this for us) but when system() exits, we can only tell if the command was executed successfully or unsuccessfully–we learn nothing about what actually failed (the shell or command which was executed under the shell?) If we require detailed information about what happened to the child process then we need to do the following.
#include
#include

/* Send complete command as a string */
/* including all arguments */

ShellCommandReturnsZero(comm)

char *comm;

{ int status, i, argc;
pid_t pid;
char arg[maxshellargs][bufsize];
char **argv;

/* Build argument array for execv call*/

for (i = 0; i < maxshellargs; i++)
{
bzero (arg[i],bufsize);
}

argc = SplitCommand(comm,arg);

if ((pid = fork()) < 0)
{
FatalError(”Failed to fork new process”);
}
else if (pid == 0) /* child */
{
argv = malloc((argc+1)*sizeof(char *));

for (i = 0; i = maxshellargs-1)
{
yyerror(”Too many arguments in embedded script”);
FatalError(”Use a wrapper”);
}

while (*sp == ‘ ‘ || *sp == ‘\t’)
{
sp++;
}

switch (*sp)
{
case ‘\”‘: sscanf (++sp,”%[^\"]“,buff);
break;
case ‘\”: sscanf (++sp,”%[^\']“,buff);
break;
default: sscanf (sp,”%s”,buff);
break;
}

for (j = 0; j < bufsize; j++)
{
arg[i][j] = buff[j];
}

sp += strlen(arg[i]);
i++;
}
return (i);
}

In this example, the script waits for the exit signal from the child process before continuing. The return value from the child is available from the wait function with the help of a set of macros defined in `/usr/include/sys/wait.h’. The value is given by WTERMSIG(status).
In the final example, we can open a pipe to a process directly in a C program as though it were a file, by using the function popen(). Pipes may be opened for reading or for writing, in exactly the same way as a file is opened. The child process is automatically synchronized with the parent using this method. Here is a program which opens a UNIX command for reading (both stdout and stderr) from the child process are piped into the program. Notice that the syntax used in this call is that used by the Bourne shell, since this is build deeply into the UNIX execution design.

#define bufsize 1024

FILE *pp;
char VBUFF[bufsize];

…

if ((pp = popen( “/sbin/mount -va 2<&1″,”r”)) == NULL)
{
printf(”Failed to open pipe\n”);
return errorcode;
}

while (!feof(pp))
{
fgets(VBUFF,bufsize,pp);

/* Just write the output to stdout */

printf (”Pipe read: %s\n”,VBUFF);
}

pclose(pp);

A more secure popen()
One problem with the popen() system call is that it uses a shell to execute the command it obtains a pipe to. In the past this has been used to allow UNIX security breaches, using a so-called IFS attack which can trick the shell into executing a program with the name of the first node in the directory of the executable. For instance,if the pipe was to open the program `/bin/ps’, this coudl be tricked into executing a program in the current working directory of the process called `bin’ with argument `ps’.
The solution is not to use a shell at all, but to replace popen() with a version which calls exec() directly. Here is a safe version from the source code of cfengine:

#define bufsize 4096
#define maxshellargs 20

pid_t *CHILD;
int MAXFD = 20; /* Max number of simultaneous pipes */

/***************************************************************/

FILE *cfpopen(command, type)

char *command, *type;

{ char arg[maxshellargs][bufsize];
int i, argc, pd[2];
char **argv;
pid_t pid;
FILE *pp = NULL;

if ((*type != ‘r’ && *type != ‘w’) || (type[1] != ”))
{
errno = EINVAL;
return NULL;
}

if (CHILD == NULL) /* first time */
{
if ((CHILD = calloc(MAXFD,sizeof(pid_t))) == NULL)
{
return NULL;
}
}

if (pipe(pd) < 0) /* Create a pair of descriptors to this process */
{
return NULL;
}

if ((pid = fork()) == -1)
{
return NULL;
}

if (pid == 0)
{
switch (*type)
{
case ‘r’:

close(pd[0]); /* Don’t need output from parent */

if (pd[1] != 1)
{
dup2(pd[1],1); /* Attach pp=pd[1] to our stdout */
dup2(pd[1],2); /* Merge stdout/stderr */
close(pd[1]);
}

break;

case ‘w’:

close(pd[1]);

if (pd[0] != 0)
{
dup2(pd[0],0);
close(pd[0]);
}
}

for (i = 0; i 0)
{
close(CHILD[i]);
}

argc = SplitCommand(command,arg);
argv = (char **) malloc((argc+1)*sizeof(char *));

if (argv == NULL)
{
FatalError(”Out of memory”);
}

for (i = 0; i < argc; i++)
{
argv[i] = arg[i];
}

argv[i] = (char *) NULL;

if (execv(arg[0],argv) == -1)
{
sprintf(OUTPUT,”Couldn’t run %s”,arg[0]);
CfLog(cferror,OUTPUT,”execv”);
}

_exit(1);
}
}
else
{
switch (*type)
{
case ‘r’:

close(pd[1]);

if ((pp = fdopen(pd[0],type)) == NULL)
{
return NULL;
}
break;

case ‘w’:

close(pd[0]);

if ((pp = fdopen(pd[1],type)) == NULL)
{
return NULL;
}
}

CHILD[fileno(pp)] = pid;
return pp;
}
}

/***************************************************************/

cfpclose(pp)

FILE *pp;

{ int fd, status;
pid_t pid;

Debug(”cfpclose(pp)\n”);

if (CHILD == NULL) /* popen hasn’t been called */
{
return -1;
}

fd = fileno(pp);

if ((pid = CHILD[fd]) == 0)
{
return -1;
}

CHILD[fd] = 0;

if (fclose(pp) == EOF)
{
return -1;
}

Debug(”cfpopen – Waiting for process %d\n”,pid);

#ifdef HAVE_WAITPID

while(waitpid(pid,&status,0) < 0)
{
if (errno != EINTR)
{
return -1;
}
}

return status;

#else

if (wait(&status) != pid)
{
return -1;
}
else
{
if (WIFSIGNALED(status))
{
return -1;
}

if (! WIFEXITED(status))
{
return -1;
}

return (WEXITSTATUS(status));
}
#endif
}

/*******************************************************************/
/* Command exec aids */
/*******************************************************************/

SplitCommand(comm,arg)

char *comm, arg[maxshellargs][bufsize];

{ char *sp;
int i = 0, j;
char buff[bufsize];

for (sp = comm; sp = maxshellargs-1)
{
CfLog(cferror,”Too many arguments in embedded script”,””);
FatalError(”Use a wrapper”);
}

while (*sp == ‘ ‘ || *sp == ‘\t’)
{
sp++;
}

switch (*sp)
{
case ”: return(i-1);

case ‘\”‘: sscanf (++sp,”%[^\"]“,arg[i]);
break;
case ‘\”: sscanf (++sp,”%[^\']“,arg[i]);
break;
case ‘`’: sscanf (++sp,”%[^`]“,arg[i]);
break;
default: sscanf (sp,”%s”,arg[i]);
break;
}

sp += strlen(arg[i]);
i++;
}

return (i);
}

Traps and signals
Processes can receive signals from the UNIX kernel at any time. Some of these signals terminate the execution of the program. This can cause problems if the program is in the middle of critical activity such as writing to a file. For that reason we can trap signals and provide our own routine for handling them in a special way.
A signal handler is made by calling the function `signal()’ for each signal and by specifying a pointer to a function which will be called in the event of a signal. For example:

main ()

{ int HandleSignal();

signal(SIGTERM,HandleSignal);
}

HandleSignal()

{
/* Tidy up and exit cleanly */

exit(0);
}

`SIGTERM’ is the usual signal sent by the command `kill’. There are many other signals which can be sent to programs. Here is list. You have to decide for yourself whether or not you want to provide your own signal handling function. To ignore a signal, you write

signal(SIGtype,SIG_IGN);

To remove a signal handler and re-activate a signal, you write

signal(SIGtype,SIG_DFL);

Regular expressions
A regular expression is a pattern for matching strings of text. We have met regular expressions earlier in connection with the shell and Perl. Naturally these earlier encounters have their roots in C functions for handling expressions. A regular expression is used by first `compiling’ it into a convenient data structure. Then a matching function is used to compare the expression with a test string. In this example program we show how a regular expression typed in as an argument to the program is found within strings of input entered on the keyboard.

#include
#include

main (argc,argv)

int argc;
char **argv;

{
char buffer[1024];
regex_t rx;
regmatch_t match;
size_t nmatch = 1;

if (regcomp(&rx, argv[1], REG_EXTENDED) != 0)
{
perror(”regcomp”);
return;
}

while (!feof(stdin))
{
fgets(buffer,1024,stdin);

if (regexec(&rx,buffer,1,&match,0) == 0)
{
printf(”Matched:(%s) at %d to %d”,buffer,match.rm_so,match.rm_eo);
}

}

regfree(&rx);
}

Here is an example of its use. The output of the program is in italics

% a.out xyz
this is a string
another string
an xyz string
Matched: (an xyz string
) at 3 to 6
another xyz zyxxyz string
Matched: (another xyz xyz string
) at 8 to 11

% a.out ‘xyz|abc’
This is a string
An abc string
Matched: (An abc string
) at 3 to 6
Or an xyz string
Matched: (Or an xyz string
) at 6 to 9

If you don’t want the match data set &pm to NULL. To get an exact match rather than a substring check that the bounds are 0 and strlen(argv[1])-1.
DES encryption
Encryption with the SSLeay library, compile with command

gcc crypto.c -I/usr/local/ssl/include -L/usr/local/ssl/lib -lcrypto

Example of normal triple DES encryption which works only on an 8-byte buffer:

/*****************************************************************************/
/* */
/* File: crypto.c */
/* */
/* Compile with: gcc program.c -lcrypto (SSLeay) */
/* */
/*****************************************************************************/

#include
#include

#define bufsize 1024

/* Note how this truncates to 8 characters */

main ()

{ char in[bufsize],out[bufsize],back[bufsize];
des_cblock key1,key2,key3,seed = {0xFE,0xDC,0xBA,0×98,0×76,0×54,0×32,0×10};
des_key_schedule ks1,ks2,ks3;

strcpy(in,”1 2 3 4 5 6 7 8 9 a b c d e f g h i j k”);

des_random_seed(seed);

des_random_key(key1);
des_random_key(key2);
des_random_key(key3);

des_set_key((C_Block *)key1,ks1);
des_set_key((C_Block *)key2,ks2);
des_set_key((C_Block *)key3,ks3);
des_ecb3_encrypt((C_Block *)in,(C_Block *)out,ks1,ks2,ks3,DES_ENCRYPT);

printf(”Encrypted [%s] into [%s]\n”,in,out);

des_ecb3_encrypt((C_Block *)out,(C_Block *)back,ks1,ks2,ks3,DES_DECRYPT);

printf(”and back to.. [%s]\n”,back);
}

Triple DES, chaining mode, for longer strings (which must be a multiple of 8 bytes):

/*****************************************************************************/
/* */
/* File: crypto.c */
/* */
/* Compile with: gcc program.c -lcrypto (SSLeay) */
/* */
/*****************************************************************************/

#include
#include

#define bufsize 1024

/* This can be used on arbitrary length buffers */

main ()

{ char in[bufsize],out[bufsize],back[bufsize],workvec[bufsize];
des_cblock key1,key2,key3,seed = {0xFE,0xDC,0xBA,0×98,0×76,0×54,0×32,0×10};
des_key_schedule ks1,ks2,ks3;

strcpy(in,”1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z”);

des_random_seed(seed);

des_random_key(key1);
des_random_key(key2);
des_random_key(key3);

des_set_key((C_Block *)key1,ks1);
des_set_key((C_Block *)key2,ks2);
des_set_key((C_Block *)key3,ks3);

/* This work vector can be intialized t anything …*/

memset(workvec,0,bufsize);

des_ede3_cbc_encrypt((C_Block *)in,(C_Block *)out,(long)strlen(in),
ks1,ks2,ks3,(C_Block *)workvec,DES_ENCRYPT);

printf(”Encypted [%s] into [something]\n”,in);

/* .. but this must be initialized the same as above */

memset(workvec,0,bufsize);

/* Note that the length is the original length, not strlen(out) */

des_ede3_cbc_encrypt((C_Block *)out,(C_Block *)back,(long)strlen(in),
ks1,ks2,ks3,(C_Block *)workvec,DES_DECRYPT);

printf(”and back to.. [%s]\n”,back);
}

Device control: ioctl
The C function `ioctl’ (I/O control) is used to send special control commands to devices like the disk and the network interface. The syntax of the function is
int ioctl(fd, request, arg)
int fd, request;
long arg;
The first parameter is normally as device handle or socket descriptor. The second is a control parameter. Lists of valid control parameters are normally defined in the system `include’ files for a particular device. They are device and system dependent so you need a local manual and som detective work to find out what they are. The final parameter is a pointer to a variable which receives return data from the device.
`ioctl’ commands are device specific, by their nature. The commands for the ethernet interface device are only partially standardized, for example. We could read the ethernet device (which is called `le0′ on a Sun workstation), using the following command:

# include /* Typical includes for internet */
# include
# include
# include
# include
# include
# include
# include

struct ifreq IFR;
int sk;
struct sockaddr_in sin;

strcpy(IFR.ifr_name,”le0″);
IFR.ifr_addr.sa_family = AF_INET;

if ((sk = socket(AF_INET,SOCK_DGRAM,IPPROTO_IP)) == -1)
{
perror(”socket”);
exit(1);
}

if (ioctl(sk,SIOCGIFFLAGS, (caddr_t) &IFR) == -1)
{
perror (”ioctl”);
exit(1);
}
We shall not go into the further details of `ioctl’, but simply note its role in system programming.
Database example (Berkeley db)

DBT key,value;
DB *dbp;
DBC *dbcp;
db_recno_t recno;

if ((errno = db_open(CHECKSUMDB,DB_BTREE, DB_CREATE, 0664, NULL, NULL, &dbp)) != 0)
{
sprintf(OUTPUT,”cfd: couldn’t open checksum database %s\n”,CHECKSUMDB);
CfLog(cferror,OUTPUT,”db_open”);
return false;
}

bzero(&value,sizeof(value));
bzero(&key,sizeof(key));

key.data = filename;
key.size = strlen(filename)+1;
value.data = dbvalue;
value.size = sizeof(dbvalue);

if ((errno = dbp->del(dbp,NULL,&key,0)) != 0)
{
CfLog(cferror,””,”db_store”);
}

key.data = filename;
key.size = strlen(filename)+1;

if ((errno = dbp->put(dbp,NULL,&key,&value,0)) != 0)
{
CfLog(cferror,”put failed”,”db->put”);
}

if ((errno = dbp->get(dbp,NULL,&key,&value,0)) == 0)
{
/* Not found … */
return;
}

dbp->close(dbp,0);

Text parsing tools: `lex’ and `yacc’
This section is a taster only. You only need to know what lex and yacc are, not how they work.
`lex’ and `yacc’ are two tools for the C programmer who wishes to make a text parser. A text parser is a program which reads a text file and interprets the symbols in it. Every programming language must include a text parser, for instance.
The `yacc’ (yet another compiler compiler) program generates C code which parses a textfile, given a description of the syntax rules for the file. In other words, we define the logical structure of the text file, according to the way we wish to interpret it and give the rules to `yacc’. `yacc’ produces C code from this which does the job.
`lex’ is a `lexer’. It is normally used together with `yacc’. `lex’ tokenizes or identifies symbols in a file. What that means is that it reads in a file and matches types of string in the file which are defined in terms of regular expressions by the programmer, and returns symbolic values for those strings.
Although `lex’ can be used by independently of `yacc’, it is normally used to identify the different types of string which define the syntax of a file. For example, suppose `yacc’ was parsing a C program. On the beginning of a line, it might expect to find either a variable name or a preprocessor symbol. A variable name is just a string consisting of characters from the set `0-9a-Z_’, whereas a preprocessor command always starts with the character `#’. `yacc’ passes control to `lex’ which reads the file and matches the first object on the line. If it finds a variable, it returns to `yacc’ a token which is a number or value corresponding to `variable’. Similarly, if it finds a preprocessor command, it returns a token for that. If it doesn’t match either type it returns something else and `yacc’ signals a syntax error.
Here is a `yacc’ file which parses a file consisting of lines of the form a+b, where $a$ and $b$ are numbers — any other syntax is incorrect. We could have used this later in the example program for the client-server example, See section Socket streams.
You can learn more about lex and yacc in “Lex and Yacc”, J. Levine, T. Mason and D. Brown, O’Reilly and Assoc.
%{
/*******************************************************************/
/* */
/* PARSER for a + b protocol */
/* */
/* The section between the single %’s gets copied verbatim into */
/* the resulting C code yacc generates — including this comment! */
/* */
/*******************************************************************/

#include

extern char *yytext;

%}

%token NUMBER PLUS

%%

specification: { yyerror(”Warning: invalid statement”);}
| statement;

statement: NUMBER PLUS NUMBER;

The lexer to go with this parser generates the tokens NUMBER and PLUS used by `yacc’:
%{
/*******************************************************************/
/* */
/* LEXER for a + b protocol */
/* */
/* Returns token types NUMBER and PLUS to yacc, one at a time */
/* */
/*******************************************************************/

#include “y.tab.h” /* yacc produces this — need this line! */

%}

number [0-9]+
plus [+]

%%

number {
return NUMBER;
}

plus {
return PLUS;
}

. {
return yytext[0];
}

%%

/* EOF */

The main program which uses `yacc’ and `lex’ looks like this:

extern FILE *yyin;

main ()

{

if ((yyin = fopen(”My_Input_File”,”r”)) == NULL) /* Open file */
{
printf(”Can’t open file\n”);
exit (1);
}

while (!feof(yyin))
{
yyparse();
}

fclose (yyin);
}
Exercises
1. Write a daemon program with a signal handler which makes a log of the heaviest (maximum cpu) process running, every five minutes. The program should exit if the log file becomes greater than 5-kbytes.
2. Rewrite in C the perl program which lists all the files in the current directory containing a certain string.
3. Write a version of `more’ which prints control characters safely. See the `cat -e’ command.
4. Write a Makefile to create a shared library from a number of object files.
Network Programming
Client-server communication is the basis of modern operating system technology. The UNIX socket mechanism makes stream-based communication virtually transparent.
Socket streams
Analogous to filestreams are sockets or TCP/IP network connections. A socket is a two-way (read/write) pseudo-file node. An open socket stream is like an open file-descriptor. Berkeley sockets are part of the standard C library.
There are two main kinds of socket: TCP/IP sockets and UNIX domain sockets. UNIX sockets can be used to provide local interprocess communication using a filestream communication protocol. TCP/IP sockets open file descriptors across the network. A TCP/IP socket is a file stream associated with an IP address and a port number. We write to a socket descriptor just as with a file descriptor, either with write() or using send().
When sending binary data over a network we have to be careful about machine level representations of data. Operating systems (actually the hardware they run on) fall into two categories known as big endian and little endian. The names refer to the byte-order of numerical representations. The names indicate how large integers (which require say 32 bits or more) are stored in memory. Little endian systems store the least significant byte first, while big endian systems store the most significant byte first. For example, the representation of the number 34,677,374 has either of these forms.

———————————–
Big | 2 | 17 | 34 | 126 |
———————————–

———————————–
Little | 126 | 34 | 17 | 2 |
———————————–

Obviously if we are transferring data from one host to another, both hosts have to agree on the data representation otherwise there would be disastrous consequences. This means that there has to be a common standard of network byte ordering. For example, Solaris (SPARC hardware) uses network byte ordering (big endian), while GNU/Linux (Intel hardware) uses the opposite (little endian). This means that Intel systems have to convert the format every time something is transmitted over the network. UNIX systems provide generic functions for converting between host-byteorder and network-byteorder for small and long integer data:

htonl, htons, ntohl, ntohs

Here we list two example programs which show how to make a client-server pair. The server enters a loop, and listens for connections from any clients (the generic address `INADDR_ANY’ is a wildcard for any address on the current local network segment). The client program sends requests to the server as a protocol in the form of a string of the type `a + b’. Normally `a’ and `b’ are numbers, in which case the server returns their sum to the client. If the message has the special form `halt + *’, where the star is arbitrary, then the server shuts down. Any other form of message results in an error, which the server signals to the client.
The basic structure of the client-server components in terms of system calls is this:

Client:

socket() Create a socket
connect() Contact a server socket (IP + port)

while (?)
{
send() Send to server
recv() Receive from server
}

Server:

socket() Create a socket
bind() Associates the socket with a fixed address
listen() Create a listen queue

while()
{
reply=accept() Accept a connection request
recv() Receive from client
send() Send to client
}

/**********************************************************************/
/* */
/* The client part of a client-server pair. This simply takes two */
/* numbers and adds them together, returning the result to the client */
/* */
/* Compiled with: */
/* cc server.c */
/* */
/* User types: */
/* 3 + 5 */
/* a + b */
/* halt + server */
/**********************************************************************/

#include
#include
#include
#include
#include

#define PORT 9000 /* Arbitrary non-reserved port */
#define HOST “nexus.iu.hio.no”
#define bufsize 20

/**********************************************************************/
/* Main */
/**********************************************************************/

main (argc,argv)

int argc;
char *argv[];

{ struct sockaddr_in cin;
struct hostent *hp;
char buffer[bufsize];
int sd;

if (argc != 4)
{
printf(”syntax: client a + b\n”);
exit(1);
}

if ((hp = gethostbyname(HOST)) == NULL)
{
perror(”gethostbyname: “);
exit(1);
}

memset(&cin,0,sizeof(cin)); /* Another way to zero memory */

cin.sin_family = AF_INET;
cin.sin_addr.s_addr = ((struct in_addr *)(hp->h_addr))->s_addr;
cin.sin_port = htons(PORT);

printf(”Trying to connect to %s = %s\n”,HOST,inet_ntoa(cin.sin_addr));

if ((sd = socket(AF_INET,SOCK_STREAM,0)) == -1)
{
perror(”socket”);
exit(1);
}

if (connect(sd,&cin,sizeof(cin)) == -1)
{
perror(”connect”);
exit(1);
}

sprintf(buffer,”%s + %s”,argv[1],argv[3]);

if (send(sd,buffer,strlen(buffer),0) == -1)
{
perror (”send”);
exit(1);
}

if (recv(sd,buffer,bufsize,0) == -1)
{
perror(”recv”);
exit (1);
}

printf (”Server responded with %s\n”,buffer);

close (sd);
unlink(”./socket”);
}

/**********************************************************************/
/* */
/* The server part of a client-server pair. This simply takes two */
/* numbers and adds them together, returning the result to the client */
/* */
/* Compiled with: */
/* cc server.c */
/* */
/**********************************************************************/

#include
#include
#include
#include
#include

#define PORT 9000
#define bufsize 20
#define queuesize 5
#define true 1
#define false 0

/**********************************************************************/
/* Main */
/**********************************************************************/

main ()

{ struct sockaddr_in cin;
struct sockaddr_in sin;
struct hostent *hp;
char buffer[bufsize];
int sd, sd_client, addrlen;

memset(&sin,0,sizeof(sin)); /* Another way to zero memory */
sin.sin_family = AF_INET;
sin.sin_addr.s_addr = INADDR_ANY; /* Broadcast address */
sin.sin_port = htons(PORT);

if ((sd = socket(AF_INET,SOCK_STREAM,0)) == -1)
{
perror(”socket”);
exit(1);
}

if (bind(sd,&sin,sizeof(sin)) == -1) /* Must have this on server */
{
perror(”bind”);
exit(1);
}

if (listen(sd,queuesize) == -1)
{
perror(”listen”);
exit(1);
}

while (true)
{
if ((sd_client = accept(sd,&cin,&addrlen)) == -1)
{
perror(”accept”);
exit(1);
}

if (recv(sd_client,buffer,sizeof(buffer),0) == -1)
{
perror(”recv”);
exit(1);
}

if (!DoService(buffer))
{
break;
}

if (send(sd_client,buffer,strlen(buffer),0) == -1)
{
perror(”send”);
exit(1);
}

close (sd_client);
}

close (sd);
printf(”Server closing down…\n”);
}

/**************************************************************/

DoService(buffer)

char *buffer;

/* This is the protocol section. Here we must */
/* check that the incoming data are sensible */

{ int a=0,b=0;

printf(”Received: %s\n”,buffer);
sscanf(buffer,”%d + %d\n”,&a,&b);

if (a > 0 && b> 0)
{
sprintf(buffer,”%d + %d = %d”,a,b,a+b);
return true;
}
else
{
if (strncmp(”halt”,buffer,4) == 0)
{
sprintf(buffer,”Server closing down!”);
return false;
}
else
{
sprintf(buffer,”Invalid protocol”);
return true;
}
}
}

In the example we use `streams’ to implement a typical input/output behaviour for C. A stream interface is a so-called reliable protocol. There are other kinds of sockets too, called unrealiable, or UDP sockets. Features to notice on the server are that we must bind to a specific address. The client is always implicitly bound to an address since a socket connection always originates from the machine on which the client is running. On the server however we want to know which addresses we shall be receiving requests from. In the above example we use the generic wildcard address `INADDR_ANY’ which means that any host can connect to the server. Had we been more specific, we could have limited communication to two machines only.
By calling `listen()’ we set up a queue for incoming connections. Rather than forking a separate process to handle each request we set up a queue of a certain depth. If we exceed this depth then new clients rtying to connect will be refused connection.
The `accept’ call is the mechanism which extracts a `reply handle’ from the socket. Using the handle obtained from this call we can reply to the client without having to open a special socket explicitly.
An improved server side connection can be setup, reading the service name from `/etc/services’ and setting reusable socket options to avoid busy signals, like this:

struct sockaddr_in cin, sin;
struct servent *server;
int sd, addrlen = sizeof(cin);
int portnumber, yes=1;

if ((server = getservbyname(service-name,”tcp”)) == NULL)
{
CfLog(cferror,”Couldn’t get cfengine service”,”getservbyname”);
exit (1);
}

bzero(&cin,sizeof(cin));

/* Service returns network byte order */

sin.sin_port = (unsigned short)(server->s_port);
sin.sin_addr.s_addr = INADDR_ANY;
sin.sin_family = AF_INET;

if ((sd = socket(AF_INET,SOCK_STREAM,0)) == -1)
{
CfLog(cferror,”Couldn’t open socket”,”socket”);
exit (1);
}

if (setsockopt (sd, SOL_SOCKET, SO_REUSEADDR,
(char *) &yes, sizeof (int)) == -1)
{
CfLog(cferror,”Couldn’t set socket options”,”sockopt”);
exit (1);
}

if (bind(sd,(struct sockaddr *)&sin,sizeof(sin)) == -1)
{

}

/* etc */

Multithreading a server
All the arguments must be collected into a struct, since only one argument pointer can be sent to the pthread functions.

#include

SpawnCfGetFile(args)

struct cfd_thread_arg *args;

{ pthread_t tid;
void *CfGetFile();

pthread_attr_init(&PTHREADDEFAULTS);
pthread_attr_setdetachstate(&PTHREADDEFAULTS,PTHREAD_CREATE_DETACHED);

if (pthread_create(&tid,&PTHREADDEFAULTS,CfGetFile,args) != 0)
{
CfLog(cferror,”pthread_create failed”,”create”);
CfGetFile(args);
}

pthread_attr_destroy(&PTHREADDEFAULTS);
}

/***************************************************************/

void *CfGetFile(args)

struct cfd_thread_arg *args;

{ pthread_mutex_t mutex;

if (pthread_mutex_lock(&mutex) != 0)
{
CfLog(cferror,”pthread_mutex_lock failed”,”pthread_mutex_lock”);
free(args->replyfile); /* from strdup in each thread */
DeleteConn(args->connect);
free((char *)args);
return NULL;
}

ACTIVE_THREADS++; /* Global variable */

if (pthread_mutex_unlock(&mutex) != 0)
{
CfLog(cferror,”pthread_mutex_unlock failed”,”unlock”);
}

/* send data */

if (pthread_mutex_lock(&mutex) != 0)
{
CfLog(cferror,”pthread_mutex_lock failed”,”pthread_mutex_lock”);
return;
}

ACTIVE_THREADS–;

if (pthread_mutex_unlock(&mutex) != 0)
{
CfLog(cferror,”pthread_mutex_unlock failed”,”unlock”);
}

#endif

return NULL;
}

System databases
The C library calls which query the databases are, amongst others,
getpwnam get password data by name
getpwuid get password data by uid
getgrnam get group data by name
gethostent get entry in hosts database
getnetgrent get entry in netgroups database
getservbyname get servive by name
getservbyport get service by port
get protobyname get protocol by name
For a complete list and how to use these, see the UNIX manual.
The following example shows how to read the password file of the system. The functions used here can be used regardless of whether the network information service (NIS) is in use. The data are returned in a structure which is defined in `/usr/include/pwd.h’.
/******************************************************************/
/* */
/* Read the passwd file by name and sequentially */
/* */
/******************************************************************/

#include
#include

main ()

{ uid_t uid;
struct passwd *pw;

uid = getuid();

pw = getpwuid(uid);

printf (”Your login name is %s\n”,pw->pw_name);

printf (”Now here comes the whole file!\n\n”);

setpwent();

while (getpwent())
{
printf (”%s:%s:%s\n”,pw->pw_name,pw->pw_gecos,pw->pw_dir);
}

endpwent();
}

DNS – The Domain Name Service
The second network database service is that which converts host and domain names into IP numbers and vice versa. This is the domain name service, usually implemented by the BIND (Berkeley Internet Name Domain) software. The information here concerns version 4.9 of this software.
gethostbyname()
This is perhaps the most important function form hostname lookup. `gethostbyname()’ gets its information either from files, NIS or DNS. Its behaviour is configured by the files mentioned above, See section DNS – The Domain Name Service. It is used to look up the IP address of a named host (including domain name if DNS is used). On the configurable systems described above, the full list of servers is queried until a reply is obtained. The order in which the different services are queried is important here since DNS returns a fully qualified name (host name plus domain name) whereas NIS and the `/etc/hosts’ file database return only a hostname.
gethostbyname returns data in the form of a pointer to a static data structure. The syntax is
#include

struct hostent *hp;

hp = gethostbyname(”myhost.domain.country”)
The resulting structure varies on different implementations of UNIX, but the `old BSD standard’ is of the form:

struct hostent
{
char *h_name; /* official name of host */
char **h_aliases; /* alias list */
int h_addrtype; /* host address type */
int h_length; /* length of address */
char **h_addr_list; /* list of addresses from name server */
};

#define h_addr h_addr_list[0] /* address, for backward compatiblity */

The structure contains a list of addresses and or aliases from the nameserver. The interesting quantity is usually extracted by means of the macro `h_addr’ whcih gives the first value in the address list, though officially one should examine the whole list now.
This value is a pointer which can be converted into a text form by the following hideous type transformation:

#include
#include
#include

struct sockaddr_in sin;

cin.sin_addr.s_addr = ((struct in_addr *)(hp->h_addr))->s_addr;

printf(”IP address = %s\n”,inet_ntoa(cin.sin_addr));

See the client program in the first section of this chapter for an example of its use.
C support for NFS
The support for NFS mounting in the standard C library is through two sources. NFS is based on the Sun’s RPC system, so the basic calls are only instances of standard RPC protocols.
The C functions in the standard input/output library can be used to access NFS filesystems. Since NFS imitates the UNIX filesystem as closely as possible, NFS filesystems can be mounted in exactly the same way as ordinary filesystems. Unfortunately, the C functions which perform the mount operation in UNIX and depressingly non-standard. They differ on almost every implementation of UNIX.
The basic function which mounts a filesystem, in `mount’ (see man (2) mount). The mount table is stored in a file /etc/mtab on BSD systems (again the name varies wildly from UNIX to UNIX, mnttab on HPUX for instance). The file /etc/rmtab on an NFS server contains a list of remote-mounted filesystems which are mounted by remote clients. C functions exist which can read the filesystem tables and place the resulting data in C struct types. Alas, these struct defintions are also quite different on different systems. See `/usr/include/sys/mount.h’, so the user wishing to write system-independent code is confounded at the lowest level.
Exercises
1. Use `gethostbyname()’ to make a simple program like `nslookup’ which gives the internet address of a named host.
2. Modify the client server example above to make a `remote ls’ command called `rls’. You should be able to use the syntax
3. rls (options) hostname:/path/to/file
Summary of programming idioms.
True and false

# C shell

True – non-zero/non-empty value
False – zero or null string

# Bourne shell

True – 0 returned by shell command
False – non-zero returned by shell command

( Note that “test” converts from C shell style to Bourne shell)

# Perl

True – non-zero/non-empty value
False – zero or null string

/* C */

True – non zero integer
False – zero integer
Input from tty
# C shell

$<

# Bourne shell

line
read

# Perl

/* C */

scanf

Redirection of I/O

# C shell

command > file
command >& file
command >> file
command1 | command2

# Bourne shell

command > file
command > file 2>&1
command >> file
command1 | command2

# Perl

open (HANDLE,”>file”)
open (HANDLE,”>file 2>&1″)
open (HANDLE,”>>file”)
open (HANDLE,”command1 |”)
open (HANDLE,”| command2″)

/* C */

fopen (”file”,”w”); printf(..)
fopen (”file”,”w”); printf(..); fprintf(stderr,..)
fopen (”file”,”a”); printf(..)
popen (”command1″,”r”)
popen (”command2″,”w”)

Loops and tests

/* C */ Shell

foreach end if then else endif
while end switch case breaksw endsw
repeat

# Bourne shell

while do done if then else fi
until do done case in esac
for in do done

# Perl

while if then else
for unless else
foreach
until
do while
do until

/* C */

while if then else
do while switch case
for

Arguments from command line
# C shell

$argv[]
$#argv

# Bourne Shell

$1, $2, $3… $*
$#

# Perl

$ARGV[]
$#ARGV

/* C */

char argv[][]
int argc
Arithmetic
# C shell

a = $b + $c

# Bourne shell

a = `expr $b + $c`

# Perl

$a = $b + $c;

/* C */

a = b + c;
Numerical comparison
# C shell

if ( $x == $y ) then
endif

# Bourne shell

if [ $x -eq $y ]; then
fi

# Perl

if ( $x == $y )
{
}

/* C */

if ( x == y )
{
}
String comparison

# C shell

if ( $x == $y ) then
endif

# Bourne shell

if [ $x = $y ]; then
fi

# Perl

if ( $x eq $y ) then
{
}

/* C */

if (strcmp(x,y) == 0)
{
}

Opening a file

# C shell, Bourne shell – cannot be done (pipes only)

# Perl

open (READ_HANDLE,”filename”);
open (WRITE_HANDLE,”> filename”);
open (APPEND_HANDLE,”>> filename”);

/* C */

FILE *fp;

fp = fopen (”file”,”r”);
fp = fopen (”file”,”w”);
fp = fopen (”file”,”a”);

Opening a directory

# C shell

foreach dir ( directory/* )
…
end

# Bourne shell

for dir in directory/* ;
do
…
done

# Perl

opendir (HANDLE,”directory”) || die;

while ($entry = readdir(HANDLE))
{
}

closedir(HANDLE);

# C

#include
DIR *dirh;
struct dirent *dirp;

if ((dirh = opendir(name)) == NULL)
{
perror(”opendir”)
exit(1);
}

for (dirp = readdir(dirh); dirp != NULL; dirp = readdir(dirh))
{
… /* dirp->d_name points to child */
}

closedir(dirh);

Testing file types

# C shell

if ( -f file ) # plain file
if ( -d file ) # directory

# Bourne shell

if [ -f file ] # plain file
if [ -d file ] # directory

# Perl

if ( -f file ) # plain file
if ( -d file ) # directory
if ( -l file ) # symbolic link

/* C */

#include

struct stat statvar;

stat(”file”, &statvar);

if (S_ISREG(statvar.mode)) /* plain file */
if (S_ISDIR(statvar.mode)) /* directory */

lstat(”file”, &statvar);

if (S_ISLNK(statvar.mode)) /* symbolic link */

Posted in Uncategorized, UnixAdministration ]]> http://meherchilakalapudi.wordpress.com/2009/08/19/unix-administration/feed/ 0 meherchilakalapudi http://meherchilakalapudi.wordpress.com/2009/08/17/rmiremote-method-invoking/ http://meherchilakalapudi.wordpress.com/2009/08/17/rmiremote-method-invoking/#comments Mon, 17 Aug 2009 13:34:05 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=682 ]]>

                                   

                                                RMI   

                                    [RemoteMethodInvocation]

 

Three main Distributed Technologies which are in use are:

 

1. Java RMI(Remote Method Invocation) from Sun Micro Systems. 
2. CORBA(Common Object Request Broker Architecture) from OMG(ObjectManagementGroup)
3. DCOM(Distributed Component Object Model) from Microsoft

 

Distributed Computing is started with RPC(Remote Procedure Call) technology.

 

A special S/W called PortMapper takes the client request and Connect him to the requested Application. so client must remember the port number.

 

The Server does not maintain the state.

 

 

Advantages:

     

1. 1.      Every Object will have a Unique name. so that client can remember easily
2. 2.      we can send and receive objects through network
3. 3.      RMI separates Interfaces from Implementations.

 

 The stubs and skeletons performing the marshaling and unmarshaling of data

A Remote stub uses a remote reference to carry out a remote method invocation to a remote object.

Transport layer makes the stream based network connections over  TCP/IP between JVMs .It provides basic connectivity.

 

 

             

 

           

                                   

RMI allows object to object communication between different virtual machines(JVM). JVM can be distinct entities located on the same or separate computer yet one JVM can invokes methods belonging to an object stored in another JVM. This enables application to call object methods located remotely, sharing resources and processing load across systems. if both communication objects are written java programming langs we use RMI

RMI is easier to understand and convenient to use.

 

        In java, distributed object model, a remote object is one whose methods can be invoked from another java virtual machine Potentially on different host.  An object of this type can be described by one or more remote interface which are java interfaces that declare the methods of remote object.

 

 Remote method invocation is the action of invoking a methods of a remote interface on a remote object.

 

Client:

         

The JVM  that calls the remote object is usually referred to as a client.

 

Server:

 

The JVM that contains the remote object is usually referred to as a server.

 

Remote object is  always accessed via its remote interface. In other words, the client invokes the methods on the object only after casting the reference remote interface.

 

Stubs and skeleton layer:

 

The stub and skeleton acts as an interface between an application and rest of the RMI system. Its purpose is to transfer data to the remote reference layer via marshalling and unmarshalling. Marshalling refers to process of converting data or object being transferred into byte stream and unmarshalling is convert stream into an object or data. This conversion is achieved via object serialization.

 

The following list the sequence of events performed by the stub:

 

1. Initiates the connection with a remote virtual machine containing the remote object.

2. Marshall the parameters to the remote virtual machine.

3. Wait for the result of the method invocation.

4. Unmarshall the return value or exception returned.

5. Return the value to the caller.

 

 

 

 

Skeleton:

 

 The following list the sequence of the events performed by the skeleton.

 

1. Unmarshall the parameter for the remote method.

2. Invokes the method on the actual remote object implementation.

3. Marshall the result to the caller.

 

 

The jdk contains the rmic tool that creates the class files for stubs and skeleton

 

 

 

 

 

 

 

 

Locating Remote Object

 

Clients find the remote services by using the naming or directory services.

RMI naming service, a registry is a remote object that serves as  a directory Service for client. The behavior of the registry is defined by the interface java.rmi.registry.Registry. RMI registry runs on each machine that hosts remote object accepts queries for services by defined on the port 1099.

 

 In simple the remote object is associate with a name in this registry.  Anytime the client invokes methods on this remote object it obtains to a

reference to it by looking up the name. Naming class specially locates objects in the RMI registry.

 

Client connects to the Remote Registry by using the computer name and port no.

 

Naming class is used to bind the Objects to RMI Registry

 

 

 Method                                                                              Description

Public static void bind(String name, Remote obj)         Binds the remote

object  to a string name.

 

public static string [] list (String name)                    Returns an array of names

                                                                                       bounded to the registry.

 

Pubic static  Remote lookup(String name)               Return a reference a stub 

                                                                                       for the remote object                         

                                                                              associated with specified name.

 

public static void rebind (String name, Remote obj) Rebinds the specified

 

name if it is already in use to a new remote object.

 

Public static void unbind (String name)           Removes the binding with the

 

specified name.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Developing application with RMI:

 

 

Writing client server application using RMI involves six basic steps:

 

1. Defining a remote interface.

2. Implementing the remote interface.

3. Writing the client that uses the remote object.

4. Generating stubs and skeletons.

5. Starting the registry and registering object.

6. Running the server and client.

 

                                   

 

 

 

 

 

 

 

 

                                                                                                           

           

Posted in RMI(Remote Method Invoking) ]]> http://meherchilakalapudi.wordpress.com/2009/08/17/rmiremote-method-invoking/feed/ 0 meherchilakalapudi http://meherchilakalapudi.wordpress.com/2009/08/17/jspjava-server-pages/ http://meherchilakalapudi.wordpress.com/2009/08/17/jspjava-server-pages/#comments Mon, 17 Aug 2009 13:30:50 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=680 ]]>

 

 

 

JSP

                                [Java Server Page]

 

JavaServerPage is a simple,yet powerful technology for creating and maintaining dynamic webpages.A JSP is a HTML page with extension .jsp. we can include java executable code  in JSP pages.a JSP is nothing but a servlet after compilation.This enables the webapplication programmer to create dynamic content by using predifined components and by interacting with components using server side scripting . JSP can reuse javabeans and custom tag libraries for complex and dynamic functionality.

 

programmer tend to use JSPs most of the content sent to the client is fixed template data and only a small portion of the content is generated dynamically with java code.

 

Comparisions with CGI

 

• JSP can maintain state on the server between requests(since it can ue Servlet sessions)
•  Spawns a new thread for each request
•  Does not have to be loaded each time, once it has been initiated
•  Runs in a ready-loaded JVM as an extension to the web server

 

 

 

 

 

Comparing  with ASP:

 

	JSP	ASP
Platforms	All major web paltforms	Microsoft
Base Language	Java	Jscript or VBScript
Components	JSP tags,JavaBeans or Enterprise JavaBeans	COM/DCOM
Code Interpretation	Once	Each instance

 

 

Comparing with Servlets:

 

• While writing servlet programming logic is simple but output format is lengthy,

Where as in JSP we write logic and execute directly on the server

• In servlets we can’t separate dynamic content from java code

Where as in JSP clear separation of dynamic content from java code is possible

• In servlet we can’t generate complex HTML page,

     Where as in JSP we can directly include HTML

 

 

 

 

 

 

 

 

 

 

 

The Server component that executes the JSPs is referred as JSPContainer.

 

when JSP enabled sever receives the first request for a jsp, the jsp container translates that jsp into a java servlet that handles the current request and future requests to the jsp. If there are any errors in transalting into  the new servlet these errors result in translation-time errors.The jsp container places the java statements that implements the JSPs response in method _jspService() at translation time. If the new servlet compiles properly, the jspcontainer invokes method _jspService()  to process  the request .the jsp may respond directly to the request or invoke other webapplication components to assists the processing the request. Any errors that occur during the request processing are known as requesttime errors;

 

JSP Elements

 

• Scripting Elements
• Directives
• Actions

 

 

Scripting Elements

 

JSP scripting elements allow java code -variable or method declarations,scriplets, and expressions to be inserted into our JSP page.

 

 

 

 

1)Declarations:

       

        A declarations is block of java code in a JSP that is used to defiene class-wide variables and methods in generated servlet. Declarations are initialized when the jsp page is initialized, and have instance scope in the generated servlet,so that anything defined in a declarations is available throughout the JSP to other declarations,expressions,and code.Adeclaration block is enclosed between <%! and %> and do not write anything to the outputstream. the syntax is:

 

<%! Java variable and method declarations %>

Ex:

        <%! int no=10;

                String %>

 

2)Scriptlets:

 

        A scriptlet is a block of java code that is executed during the request procesing time, and is enclosed between <% and %> tags.It produces output for the client.All the code apperaring between the <% and  %> tags in the jsp gets put into the service() method of the servlet as is, in the order in which it appears.It’s syntax is:

 

        <% valid java code statements %>

        Ex:

                <% out.println(new java.util.Date().toString()); %>

 

 

 

 

 

3)Expressions:

 

        An expression is a shorthand notation for a scriplet that sends the value of a java expression back to the client.The expression is evaluated at HTTP request processing time, and the result is converted into string and displayed.An expression is enclosed between <%= and %> tags.It’s syntax is

        <%= java expression to be evaluated %>

        Ex:

                <%=”Java Expression” %>

 

JSPDirectives:

 

A JSP directive affects the overall structure of the servlet class.Directives begin with <%@ and end with %>, and the general syntax is:

 

<%@ directivename attribute=”value” attribute=”value” %>

 

There are three main directives that can be used in a JSP:

 

1)The page directive

2)The include directive

3)The taglib directive

 

The page directive:

 

        The page directive is used to define and manipulate a number of important attributes that affect the whole jsp page.A page contains any number of page directives in any order, any where in the  JSP page.The General syntax of page directive is:

<%@ page ATTRIBUTES %>

where the valid attributes are the following name value pairs

 

• language efines the scripting language to be used. default value is Java
• extends: The value is a fully qualified class name of the superclass that the generated class,into which this jsp page is translated, must extended.omitted by default
• import: Comma separated list of packages or classes, with the same meaning as import statements in java classes
• session: Specifies whether the page participates in an HTTP session.when true the implicit object named session is available and can be used to  access the current/new session for the page.

If false,the page does not participate in a session and the implicit session object is unavailable     .

• buffer: Specifies the buffering model for the output stream to the client. if the value is none, no buffering occurs and all output is written directly through to the ServletResponse by a PrintWriter.

if a buffer size is specified,out is buffered witha buffer size not less than that value.

• autoFlush: If true, the output buffer to the client is flushed qutomatically when it is full.   

If false, a runtime exception is raised to indicate a buffer overflow

• isThreadSafe: Defines the level of thread safety implemented in the page. if the value is true the JSPengine may send multiple client requests tothe page at the same time.

If the value is false then the JSP container queues up client requests sent to the page for processing, and process them one at a time, in the order in which they are recieved.

 

• errorPage: Defines a URL to another JSP page within the current webapplications,which is invoked if a checked or unchecked exception is thrown.The page implementation catches the instance   of the throwable object and passes it to the error page.
• isErrorPage: Indicates if the current JSP page is intended to another JSP page’s error page.
• contentType: Defines the character encoding for the JSP and the MIME type for the response of the JSP page.The default value of MIME  type is text/html.

       

The include directive:

 

The include directive instructs the container to include the contents of a resource in the current JSP, inserting it inline, in the JSP page.The specified file must be accessible and available to the JSP container.The syntax of the include directive is:

 

        <%@ include file=”Filename” %>

 

        Ex: <%@ include file=”/jsp-examples/hello.jsp” %>

       

the contents of the included file is parsed by the JSP only at translation time, that is when the JSP page is compilied into a servlet.

 

       

Implicit Objects:

 

• request: The request object represents the HttpServletRequest instance to access the incoming request.
• response: The response object is the HttpServletResponse instance that represents the servlets’s response to the request.
• pageContext: the pageContext provides a single point of access to many of the page attributes and is convenient place to put shared data within the page.
• session: The session object represents the session created for the requesting client.
• application: The application object represents the servlet context,obtained from the servlet configurations object.
• out: The out object is the object that writes into the outputstream to the client.
• config: The config object is the ServletConfig for this JSP page, and has page scope

 

 

Standard Actions(XMLTags):

 

JSPTags                                                           XMLTags

 

<%! declarations ; %>                                                                                                                  <jsp:declarations> 

                                                                                …….

                                                                                                                                                        </jsp:declarations>

 

<%= expressions %>                                                                                                                    <jsp:expression>

                                                                        …….

                                                                                                                                                        </jsp:expression>

 

 

 

 

 

<% scriplets %>                                  <jsp:scriplets >

                                                                        …….

                                                                                                                                                        </jsp:scriplets>

 

<%@ directive %>               <jsp:directive attribute=”value” />

       

<%@ include %>          <jsp:include page=”URL”   />

 

<% response.sendRedirect %>      <jsp:forward   page=”URL” />

                                <jsp:param name=”attribute” value=”value”>

                                                <jsp:plugin>

                                                <jsp:useBean>

                                                <jsp:setProperty>

                                                <jsp:getProperty>

 

 

 

 

 

 

<jsp:include>

 

        It is so convenient to be able to include the contents of othre files into a jsp page.This makes it easier to use standard HTML headers and footers or to share common code among multiple JSP pages.

Ex:

<jsp:include page=”url” />

 

 

 

 

<jsp:forward>

       

        This action is used to forward the  request to another JSP page.It has a single attribute i.e.,page which shared consists of a related URL.

Ex:

a)     <jsp:forward page=”url” />

 

b)     <jsp:forward page=”url” >

        <jsp:param value=”value” />

        </jsp:forward>

 

<jsp:useBean>

 

Syntax:

 

<jsp:useBean   id=”name”  scope=”page/request/session/application” />

 

id: This usually means instantiate an object of the class specified by class, and bind it to a varaible with the name specified by id.

 

class:It designates the full package name of the bean

 

scope:It indicates the context in which the bean should be made available

 

 

 

 

 

 

                                                                                                                 1.page: Bean is only available to the current page

                                                                                                                 2.request: Bean is only available for the current client request

                                                                                                                 3.session: Bean is available to all the pages during the life of the current

 

         HttpSession

                                                                                                                

 

4.application: Bean is available to all the pages that share the same servlet

                                                                                          context

 

<jsp:setProperty>

 

This action give values to properties of the beans

 

syntax:

                                                                                                                 <jsp:setProperty property=”propertyname”

                                                                                                                                  property=”propertyName”

   value=”propertyValue” />

 

<jsp:getProperty>                                                                                                                                                                                                                                           

 

This action retrieves the values of a bean property,converts it into a string and insert it into the output.

 

syntax:

                                                                                                                 <jsp:getProperty property=”propertyname”>

 

 

 

<jsp:plugin>

 

The <jsp:plugin> action provides easy support for including Java applets in JSP

Generated page. It is used to generate browser-specific HTML tags that results in the download of the Java Plugin-in software,if required ,followd by the execution of the applet or JavaBean component that is specified in the tag.

 

Syntax:

 <jsp:plugin type=”bean/applet” code=”objectcode”

 

codebase=”objectcodebase”

height=”height”

width=”width”>

<jsp:params>

     <jsp:param name=”paramname” value=”paramvalue” />

          <jsp:param name=”paramname” value=”paramvalue” />

        …….

</jsp:params>

<jsp:fallback >Alternative text to display</jsp:fallback>

</jsp:plugin>

 

                                                                                                  

                                                                                                  

                                                                                                                

 

 

 

                                                                                                                 

                                                                                                                                           property=”propertyof the bean” />

                                                                                                                                                                                                                                                       

 

 

 

       

 

 

 

 

 

 

Posted in Jsp(java server Pages) ]]> http://meherchilakalapudi.wordpress.com/2009/08/17/jspjava-server-pages/feed/ 0 meherchilakalapudi http://meherchilakalapudi.wordpress.com/2009/07/16/java-faqs/ http://meherchilakalapudi.wordpress.com/2009/07/16/java-faqs/#comments Thu, 16 Jul 2009 14:35:48 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=678 ]]>

FREQUENTLY ASKED QUESTIONS (JAVA) IN INTERVIEWS

for 1)What is OOPs?

Ans: Object oriented programming organizes a program around its data,i.e.,objects and a set of well defined interfaces to that data.An object-oriented program can be characterized as data controlling access to code.

2)what is the difference between Procedural and OOPs?

Ans: a) In procedural program, programming logic follows certain procedures and the instructions are executed one after another. In OOPs program, unit of program is object, which is nothing but combination of data and code.

b) In procedural program,data is exposed to the whole program whereas in OOPs program,it is accessible with in the object and which in turn assures the security of the code.

3)What are Encapsulation, Inheritance and Polymorphism?

Ans: Encapsulation is the mechanism that binds together code and data it manipulates and keeps both safe from outside interference and misuse.Inheritance is the process by which one object acquires the properties of another object.Polymorphism is the feature that allows one interface to be used general class actions.

4)What is the difference between Assignment and Initialization?

Ans: Assignment can be done as many times as desired whereas initialization can be done only once.

5)What are Class, Constructor and Primitive data types?

Ans: Class is a template for multiple objects with similar features and it is a blue print for objects. It defines a type of object according to the data the object can hold and the operations the object can perform. Constructor is a special kind of method that determines how an object is initialized when created.

Primitive data types are 8 types and they are:

byte, short, int, long

float, double

boolean

char

 

6)What is an Object and how do you allocate memory to it?

Ans: Object is an instance of a class and it is a software unit that combines a structured set of data with a set of  operations for inspecting and manipulating that data. When an object is created using new operator, memory is allocated to it.

7)What is the difference between constructor and method?  Ans: Constructor will be automatically invoked when an object is created whereas method has to be called explicitly.

8)What are methods and how are they defined?

Ans: Methods are functions that operate on instances of classes in which they are defined. Objects can communicate with each other using methods and can call methods in other classes.

Method definition has four parts.

they are name of the method,

 type of object or primitive type the method returns,

a list of parameters and the body of the method. A method’s signature is a combination of the first three parts mentioned above.

9)What is the use of bin and lib in JDK?

Ans: Bin contains all tools such as javac, appletviewer, awt tool, etc., whereas lib contains API and all packages.

10)What is casting?

Ans: Casting is used to convert the value of one type to another.

11)How many ways can an argument be passed to a subroutine and explain them?

Ans: An argument can be passed in two ways. They are passing by value and passing by reference.Passing by value: This method copies the value of an argument into the formal parameter of the subroutine.Passing by reference: In this method, a reference to an argument (not the value of the argument) is passed to the parameter.

12)What is the difference between an argument and a parameter?

Ans: While defining method, variables passed in the method are called parameters. While using those methods, values passed to those variables are called arguments.

13)What are different types of access modifiers?

Ans: public: Any thing declared as public can be accessed from anywhere.

private: Any thing declared as private can’t be seen outside of its class.

protected: Any thing declared as protected can be accessed by classes in the same package and subclasses in the other packages.

default modifier : Can be accessed only to classes in the same package.

14)What is final, finalize() and finally?

Ans: final : final keyword can be used for class, method and variables.A final class cannot be subclassed and it prevents other programmers from subclassing a secure class to invoke insecure methods.A final method can’ t be overriddenA final variable can’t change from its initialized value.finalize( ) : finalize( ) method is used just before an object is destroyed and can be called just prior to garbage collecollection   finally : finally, a key word used in exception handling, creates a block of code that will be executed after a try/catch block has completed and before the code following the try/catch block. The finally block will execute whether or not an exception is thrown. this For example, if a method opens a file upon exit, then you will not want the code that closes the file to be bypassed by the exception-handling mechanism. This finally keyword is designed to address

contingency.      

15)What is UNICODE?

Ans: Unicode is used for internal representation of characters and strings and it uses 16 bits to represent each other.

16)What is Garbage Collection and how to call it explicitly?

Ans: When an object is no longer referred to by any variable, java automatically reclaims memory used by that object. This is known as garbage collection.System.gc() method may be used to call it explicitly.

17)What is  finalize() method ?

Ans: finalize () method is used just before an object is destroyed and can be called just prior to garbage collection.

18)What are Transient and Volatile Modifiers?

Ans: Transient: The transient modifier applies to variables only and it is not stored as part of its object’s Persistent state. Transient variables are not serialized.Volatile: Volatile modifier applies to variables only and it tells the compiler that the variable modified by volatile can be changed unexpectedly by other parts of the program.

19)What is method overloading and method overriding?

Ans: Method overloading: When a method in a class having the same method name with different arguments is said to be method overloading.

Method overriding : When a method in a class having the same method name with same arguments is said to be method overriding.

20)What is difference between overloading and overriding?

Ans: a) In overloading, there is a relationship between methods available in the same class whereas in overriding, there is relationship between a superclass method and subclass method.

b)    Overloading does not block inheritance from the superclass whereas overriding blocks inheritance from the superclass.

c)     In overloading, separate methods share the same name whereas in overriding,subclass method replaces the superclass.

d)    Overloading must have different method signatures whereas overriding  must have same signature.

 

21) What is meant by Inheritance and what are its advantages?

Ans: Inheritance is the process of inheriting all the features from a class. The advantages of inheritance are reusability of code and accessibility of variables and methods of the super class by subclasses.

22)What is the difference between this() and super()?

Ans: this() can be used to invoke a constructor of the same class whereas super() can be used to invoke a super class constructor.

23)What is the difference between superclass and subclass?

Ans: A super class is a class that is inherited whereas sub class is a classthat does the inheriting.

24) What modifiers may be used with top-level class?

Ans: public, abstract and final can be used for top-level class.

25)What are inner class and anonymous class?

Ans: Inner class  : classes defined in other classes, including those defined in methods are called  inner classes. An inner class can have any accessibility including private.Anonymous class : Anonymous class is a class defined inside a method without a name and is instantiated and declared in the same place and cannot have explicit constructors.

26)What is a package?

Ans: A package is a collection of classes and interfaces that provides a high-level layer of access protection and name space management.

27)          What is a reflection package? 

Ans: java.lang.reflect package has the ability to analyze itself in runtime.

28)          What is interface and its use?

Ans:Interface is similar to a class which may contain method’s signature only but not bodies and  it is

a formal set of method and constant declarations that must be defined by the class that implements it.

Interfaces are useful for:

a)Declaring methods that one or more classes are expected to implement

b)Capturing similarities between unrelated classes without forcing a class relationship.

c)Determining an object’s programming interface without revealing the actual body of the class.

29)          What is an abstract class?

Ans: An abstract class is a class designed with implementation gaps for subclasses to fill in and is deliberately incomplete. 

30)          What is the difference between Integer and int?

Ans: a) Integer is a class defined in the java.lang package, whereas int is a primitive data type defined in the Java language itself. Java does not automatically convert from one to the other.

b)    Integer can be used as an argument for a method that requires an object, whereas int can be used for

calculations.

31)           What is a cloneable interface and how many methods does it contain?

Ans- It is not having any method because it is a TAGGED or MARKER interface.

32)          What is the difference between abstract class and interface?

Ans: a) All the methods declared inside an interface are abstract whereas abstract class must have at least one abstract method and others may be concrete or abstract.

b) In abstract class, key word abstract must be used for the methods whereas interface we need not use that keyword for the methods.

c)     Abstract class must have subclasses whereas interface can’t have subclasses.

 

33)          Can you have an inner class inside a method and what variables can you access?

Ans: Yes, we can have an inner class inside a method and final variables can be accessed.

34)          What is the difference between String and String Buffer?

Ans: a) String objects are constants and immutable whereasStringBuffer objects are not.

b) String class supports constant strings whereas StringBuffer class supports growable and modifiable strings.

35)          What is the difference between Array and vector?

Ans: Array is a set of related data type and static whereas vector is a growable array of objects and dynamic.

36)          What is the difference between exception and error?

Ans: The exception class defines mild error conditions that your program encounters.Ex: Arithmetic Exception, FilenotFound exception Exceptions can occur when try to open the file, which does not exist

• the network connection is disrupted
• operands being manipulated are out of prescribed ranges
• the class file you are interested in loading is missing

The error class defines serious error conditions that you should not  attempt to recover from. In most cases it is advisable to let the program terminate when such an error is encountered.

Ex: Running out of memory error, Stack overflow error.

37)          What is the difference between process and thread?

Ans: Process is a program in execution whereas thread is a separate path of execution in a program.

38)          What is multithreading and what are the methods for inter-thread communication and what is the class in which these methods are defined?

Ans: Multithreading is the mechanism in which more than one thread run independent of each other within the process.

wait (), notify () and notifyAll() methods can be used for inter-thread communication and these methods are in Object class.

wait( ) : When a thread executes a call to wait( ) method, it surrenders the object lock and enters into a waiting state.

notify( ) or notifyAll( ) : To remove a thread from the waiting state, some other thread must make a call to  notify( ) or notifyAll( ) method on the same object.

39)          What is the class and interface in java to create thread and which is the most advantageous method?

Ans: Thread class and Runnable interface can be used to create threads and using Runnable interface is the most advantageous method to create threads because we need not extend thread class here.

40)          What are the states associated in the thread?

Ans: Thread contains ready, running, waiting and dead states.

41)           What is synchronization?

Ans: Synchronization is the mechanism that ensures that only one thread is accessed the resources at a time.

42)          When you will synchronize a piece of your code?

Ans: When you expect your code will be accessed by different threads and these threads may change a particular data causing data corruption.

43)          What is deadlock?

Ans: When two threads are waiting each other and can’t precede the program is said to be deadlock.

44)          What is daemon thread and which method is used to create the daemon thread?

Ans: Daemon thread is a low priority thread which runs intermittently in the back ground doing the garbage collection operation for the java runtime system. setDaemon method is used to create a daemon thread.

45)          Are there any global variables in Java, which can be accessed by other part of your program?

Ans: No, it is not the main method in which you define variables. Global variables is not possible because concept of encapsulation is eliminated here.

46)What is an applet?

Ans: Applet is a dynamic and interactive program that runs inside a web page displayed by a java capable browser.

47)What is the difference between applications and applets?

Ans: a)Application must be run on local machine whereas applet needs no explicit installation on local machine.

b)Application must be run explicitly within a java-compatible virtual machine whereas applet loads and runs itself automatically in a java-enabled browser.

d)Application starts execution with its main method whereas applet starts execution with its init method.

e)Application can run with or without graphical user interface whereas applet must run within a graphical user interface.

48)How does applet recognize the height and  width?

Ans:Using getParameters()  method.

 

49)When do you use codebase in applet?

Ans:When the applet class file is not in the same directory, codebase is used.

50)What is the lifecycle of an applet?

Ans:init( ) method        -  Can be called when an applet is first loaded

         start( ) method      -  Can be called each time an applet is started

         paint( ) method     -  Can be called when the applet is minimized or maximized

         stop( ) method      -  Can be used when the browser moves off the applet’s page

destroy( ) method -  Can be called when the browser is finished with the applet

51)How do you set security in applets?

Ans: using  setSecurityManager() method

 

52)          What is an event and what are the models available for event handling?

Ans: An event is an event object that describes a state of change in a source. In other words, event occurs when an

action is generated, like pressing button, clicking mouse, selecting a list, etc.

There are two types of models for handling events and they are:

a)             event-inheritance model and b) event-delegation model

53)          What are the advantages of the event-delegation  model over the event-inheritance model?

Ans: The event-delegation model has two advantages over the event-inheritance model. They are:

a)It enables event handling by objects other than the ones that generate the events. This allows a clean separation between a component’s design and its use.

b)It performs much better in applications where many events are generated. This performance improvement is due to the fact that the event-delegation model does not have to be repeatedly process unhandled events as is the case of the event-inheritance.

54)What is source and listener ?

Ans: source : A source is an object that generates an event. This occurs when the internal state of that object changes in some way.

listener : A listener is an object that is notified when an event occurs. It has two major requirements. First, it must have been registered with one or more sources to receive notifications about specific types of events. Second, it must implement methods to receive and process these notifications. 

55)          What is adapter class?

Ans: An adapter class provides an empty implementation of all methods in an event listener interface. Adapter classes are useful when you want to receive and process only some of the events that are handled by a particular event listener interface. You can define a new class to act listener by extending one of the adapter classes and implementing only those events in which you are interested.For example, the MouseMotionAdapter class has two methods, mouseDragged( )and mouseMoved(). The

signatures of these empty are exactly as defined in the  MouseMotionListener interface. If you are interested in only mouse drag events, then you could simply extend MouseMotionAdapter and implement mouseDragged( ) .

56)What is meant by controls and what are different types of controls in AWT?

Ans: Controls are components that allow a user to interact with your application and the AWT supports the following types of controls:Labels, Push Buttons, Check Boxes, Choice Lists, Lists, Scrollbars, Text Components.These controls are subclasses of Component.

57)          What is the difference between choice and list?

Ans: A Choice is displayed in a compact form that requires you to pull it down to see the list of available choices and only one item may be selected from a choice.A List may be displayed in such a way that several list items are visible and it supports the selection of one or more list items.

58)          What is the difference between scrollbar and scrollpane?

Ans: A Scrollbar is a Component, but not a Container whereas Scrollpane is a Conatiner and handles its own events and perform its own scrolling.

59)          What is a layout manager and what are different types of layout managers available in java.awt?

Ans: A layout manager is an object that is used to organize components in a container. The different layouts are available are FlowLayout, BorderLayout, CardLayout, GridLayout and GridBagLayout,Null layout/Custom layout

60)          How are the elements of different layouts organized?

Ans: FlowLayout: The elements of a FlowLayout are organized in a top to bottom, left to right fashion. BorderLayout: The elements of a BorderLayout are organized at the borders (North, South, East and West) and the center of a container.

CardLayout: The elements of a CardLayout are stacked, on top of the other, like a deck of cards.GridLayout: The elements of a GridLayout are of equal size and are laid out using the square of a grid. GridBagLayout: The elements of a GridBagLayout are organized according to a grid. However, the elements are of different size and may occupy more than one row or column of the grid. In addition, the rows and columns may have different sizes.

61)           Which containers use a Border layout as their default layout?      

Ans:  Window, Frame and Dialog classes use a BorderLayout as their layout.

62)          Which containers use a Flow layout as their default layout?

Ans: Panel and Applet classes use the FlowLayout as their default layout.

63)          What are wrapper classes?

Ans: Wrapper classes are classes that allow primitive types to be accessed as objects.

64)          What are Vector, Hashtable, LinkedList and Enumeration?

Ans: Vector : The Vector class provides the capability to implement a growable array of objects.Hashtable : The Hashtable class implements a Hashtable data structure. A Hashtable indexes and stores objects in a dictionary using hash codes as the object’s keys. Hash codes are integer values that identify objects.

LinkedList: Removing or inserting elements in the middle of an array can be done using LinkedList. A

LinkedList stores each object in a separate link whereas an array stores object references in consecutive locations.

Enumeration: An object that implements the Enumeration interface generates a series of elements, one at a time. It has two methods, namely hasMoreElements( ) and nextElement( ). HasMoreElemnts( ) tests if this enumeration has more elements and nextElement method returns successive elements of the series.

65)          What is the difference between set and list?

Ans: Set stores elements in an unordered way but does not contain duplicate elements, whereas list stores elements in an ordered way but may contain duplicate elements.

66)          What is a stream and what are the types of Streams and classes of the Streams?

Ans: A Stream is an abstraction that either produces or consumes information. There are two types of Streams and they are:

Byte Streams: Provide a convenient means for handling input and output of bytes.Character  Streams:  Provide a convenient means for handling input & output of characters.Byte Streams classes: Are defined by using two abstract classes, namely InputStream and OutputStream.Character Streams classes: Are defined by using two abstract classes, namely Reader and Writer.

67)          What is the difference between Reader/Writer and InputStream/Output Stream?

Ans: The Reader/Writer class is character-oriented and the InputStream/OutputStream class is byte-oriented.

68)          What is an I/O filter?

Ans: An I/O filter is an object that reads from one stream and writes to another, usually altering the data in some way as it is passed from one stream to another.

69)          What is serialization and deserialization?

Ans: Serialization is the process of writing the state of an object to a byte stream.Deserialization is the process of restoring these objects.

70)          What is JDBC?

Ans: JDBC is a set of Java API for executing SQL statements. This API consists of a set of  classes and interfaces

to enable programs to write pure Java Database applications.

71)           What are drivers available?

Ans:  a)     JDBC-ODBC Bridge driver

       b)     Native API Partly-Java driver

c)     JDBC-Net Pure Java driver

d)    Native-Protocol Pure Java driver

72)          What is the difference between JDBC and ODBC?

Ans:  a)     ODBC is for Microsoft and JDBC is for Java applications.

b)            ODBC can’t be directly used with Java because it uses a C interface.

c)             ODBC  makes use of pointers which have been removed totally from Java.

d)    ODBC mixes simple and advanced features together and has complex options for simple queries. But JDBC is designed to keep things simple while allowing advanced capabilities when required.

e)     ODBC requires manual installation of the ODBC driver manager and driver on all client machines. JDBC drivers are written in Java and JDBC code is automatically installable, secure, and portable on all platforms.

f)     JDBC API is a natural Java interface and is built on ODBC. JDBC retains some of the basic features of ODBC.

73)          What are the types of JDBC Driver Models and explain them?

Ans: There are two types of  JDBC Driver Models and they are:

a)     Two tier model and b) Three tier modelTwo tier model: In this model, Java applications interact directly with the database. A JDBC driver is required to communicate with the particular database management system that is being accessed. SQL statements are sent to the database and the results are given to user. This model is referred to as client/server configuration where user is the client and the machine that has the database is called as the server.

Three tier model: A middle tier is introduced in this model. The functions of this model are:

a)     Collection of SQL statements from the client and handing it over to the database,

b)    Receiving results from database to the client and

c)     Maintaining control over accessing and updating of the above.

74)          What are the steps involved for making a connection with a database or how do you connect to a database?

Ans: a)    Loading the driver : To load the driver, Class.forName( ) method is used.

                     Class.forName(“sun.jdbc.odbc.JdbcOdbcDriver”);

                When the driver is loaded, it registers itself with the java.sql.DriverManager class as an available  database driver.

b)    Making a connection with database : To open a connection to a given database, DriverManager.getConnection( ) method is used.

Connection con = DriverManager.getConnection (“jdbc:odbc:somedb”, “user”,  “password”);

c)     Executing SQL statements : To execute a SQL query, java.sql.statements class is used.  

createStatement( ) method of Connection to obtain a new Statement object.

Statement stmt  = con.createStatement( );

A query that returns data can be executed using the executeQuery( ) method of  Statement. This method

executes the statement and returns a java.sql.ResultSet that encapsulates the retrieved data:

ResultSet rs = stmt.executeQuery(“SELECT * FROM some table”);

d)    Process the results : ResultSet returns one row at a time. Next( ) method of ResultSet  object can be called to move to the next row.  The getString( ) and getObject( ) methods are used for retrieving  column values:

while(rs.next( ) ) {

String event = rs.getString(“event”);

Object count = (Integer) rs.getObject(“count”);

75)          What type of driver did you use in project?

Ans: JDBC-ODBC Bridge driver (is a driver that uses native(C language) libraries and makes calls to an existing

ODBC driver to access a database engine).

76)          What are the types of statements in JDBC?

Ans: Statement                — To be used createStatement() method for executing single SQL statement

PreparedStatement  — To be used preparedStatement() method for executing same SQL statement over and

over

CallableStatement    — To be used prepareCall( ) method for multiple SQL statements over and over

 

77)          What is stored procedure?

Ans: Stored procedure is a group of SQL statements that forms a logical unit and performs a particular task. Stored Procedures are used to encapsulate a set of operations or queries to execute on database. Stored procedures can be compiled and executed with different parameters and results and may have any combination of input/output parameters.

78)          How to create and call stored procedures?

Ans:   To create stored procedures: Create procedure procedurename (specify in, out and in out parameters)

BEGIN

Any multiple SQL statement;

END;

To call stored procedures:

CallableStatement  csmt  = con.prepareCall(“{call procedure name(?,?)}”);

csmt.registerOutParameter(column no., data type);

csmt.setInt(column no., column name)

csmt.execute( ); 

        

——79)                What is servlet?

Ans: Servlets are modules that extend request/response-oriented servers, such as java-enabled web servers.For example, a servlet might be responsible for taking data in an HTML order-entry form and applying the business logic used to update a company’s order database.

80)          What are the classes and  interfaces for servlets?

Ans: There are two packages in servlets and they are javax.servlet and javax.servlet.http.

Javax.servlet contains:

               Interfaces                       Classes

               Servlet                            Generic Servlet

               ServletRequest               ServletInputStream

               ServletResponse             ServletOutputStream

               ServletConfig                 ServletException

               ServletContext                UnavailableException

SingleThreadModel

Javax.servlet.http contains:

               Interfaces                                 Classes

               HttpServletRequest                  Cookie

               HttpServletResponse                HttpServlet

               HttpSession                               HttpSessionBindingEvent

               HttpSessionContext                  HttpUtils

HttpSessionBindingListener

81)           What is the difference between an applet and a servlet?

Ans: a)       Servlets are to servers what applets are to browsers.

   b)    Applets must have graphical user interfaces whereas servlets have no graphical                                                                              user interfaces.

 

82)          What is the difference between doPost and doGet methods?

Ans: a) doGet() method is used to get information, while doPost( ) method is used for posting information.

b) doGet() requests can’t send large amount of information and is limited to 240-255 characters. However, doPost( )requests passes all of its data, of unlimited length.

c)     A doGet( ) request is appended to the request URL in a query string and this allows the exchange is visible to the client, whereas a doPost() request passes directly over the  socket connection as part of its HTTP request body and the exchange are invisible to the client.

83)          What is the life cycle of a servlet?

Ans: Each Servlet has the same life cycle:

a)     A server loads and initializes the servlet by init( ) method.

b)    The servlet  handles zero or more client’s requests through service( ) method.

c)     The server removes the servlet through destroy() method.        

84)          Who is loading the init() method of servlet?

Ans: Web server

85)          What are the different servers available for developing and deploying Servlets?

Ans:  a)    Java Web Server

      b)      JRun

g)     Apache Server

h)     Netscape Information Server

i)      Web Logic       

86)          How many ways can we track client  and what are they?

Ans: The servlet API provides two ways to track client state and they are:

a)             Using Session tracking and b) Using Cookies.

87)          What is session tracking and how do you track a user session in servlets?

Ans: Session tracking is a mechanism that servlets use to maintain state about a series requests from the same user across some period of time. The methods used for session tracking are:

a)     User Authentication – occurs when a web server restricts access to some of its resources to only those clients that log in using a recognized username and password

b)    Hidden form fields – fields are added to an HTML form that are not displayed in the client’s browser. When the form containing the fields is submitted, the fields are sent back to the server

c)     URL rewriting – every URL that the user clicks on is dynamically modified or rewritten to include extra information. The extra information can be in the form of extra path information, added parameters or some custom, server-specific URL change.

d)    Cookies – a bit of information that is sent by a web server to a browser and which can later be read back from that browser.

e)     HttpSession- places a limit on the number of sessions that can exist in memory. This limit is set in the session.maxresidents property

88)          What is Server-Side Includes (SSI)?

Ans: Server-Side Includes allows embedding servlets within HTML pages using a special servlet tag. In many servlets that support servlets, a page can be processed by the server to include output from servlets at certain points inside the HTML page. This is accomplished using a special internal SSINCLUDE, which processes the servlet tags. SSINCLUDE servlet will be invoked whenever a file with an. shtml extension is requested. So HTML files that include  server-side includes must be stored with an .shtml extension.

89)          What are cookies and how will you use them?

Ans: Cookies are a mechanism that a servlet uses to have a client hold a small amount of state-information associated with the user.

a)             Create a cookie with the Cookie constructor:

public Cookie(String name, String value)

b)    A servlet can send a cookie to the client by passing a Cookie object to the addCookie() method of HttpServletResponse:public void HttpServletResponse.addCookie(Cookie cookie)

c)     A servlet retrieves cookies by calling the getCookies() method of HttpServletRequest:

public Cookie[ ] HttpServletRequest.getCookie( ).

90)          Is it possible to communicate from an applet to servlet and how many ways and how?

Ans: Yes, there are three ways to communicate from an applet to servlet and they are:

a)     HTTP Communication(Text-based and object-based)

b)    Socket Communication

c)     RMI Communication

(You can say, by using URL object open the connection to server and get the InputStream from

URLConnection  object).

Steps involved for applet-servlet communication:

1)             Get the server URL.

URL url = new URL();

2)            Connect to the host

URLConnection Con = url.openConnection();

3)            Initialize the connection

Con.setUseCatches(false):

Con.setDoOutput(true);

Con.setDoInput(true);

4)            Data will be written to a byte array buffer so that we can tell the server the length of the data.

ByteArrayOutputStream byteout  = new ByteArrayOutputStream();

5)            Create the OutputStream to be used to write the data to the buffer.

DataOutputStream out = new DataOutputStream(byteout);

91)           What is connection pooling?

Ans: With servlets, opening a database connection is a major bottleneck because we are creating and tearing down a new connection for every page request and the time taken to create connection will be more. Creating a connection pool is an ideal approach for a complicated servlet. With a connection pool, we can duplicate only the resources we need to duplicate rather than the entire servlet. A connection pool can also  intelligently manage the size of the pool and make sure each connection remains valid. A number of connection pool packages are currently available. Some like DbConnectionBroker are freely available from Java Exchange Works by creating an object that dispenses connections and connection Ids on request.The ConnectionPool class maintains a Hastable, using Connection objects as keys and Boolean values as stored values. The Boolean value indicates whether a connection is in use or not. A program calls getConnection( ) method of the ConnectionPool for getting  Connection object it can use; it calls returnConnection( ) to give the connection back to the pool.

92)          Why should we go for interservlet communication?

Ans: Servlets running together in the same server communicate with each other in several ways.The three major reasons to use interservlet communication are:

a)     Direct servlet manipulation – allows to gain access to the other currently loaded servlets and perform certain tasks (through the ServletContext object)

b)    Servlet reuse – allows the servlet to reuse the public methods of another servlet.

c)     Servlet collaboration – requires to communicate with each other by sharing specific information (through method invocation)

93)          Is it possible to call servlet with parameters in the URL?

Ans: Yes. You can call a servlet with parameters in the syntax as (?Param1 = xxx || m2 = yyy).

94)          What is Servlet chaining?

Ans: Servlet chaining is a technique in which two or more servlets can cooperate in servicing a single request.In servlet chaining, one servlet’s output is piped to the next servlet’s input. This process continues until the last servlet is reached. Its output is then sent back to the client.

95)          How do servlets handle multiple simultaneous requests?

Ans: The server has multiple threads that are available to handle requests. When a request comes in, it is assigned to a thread, which calls a service method (for example: doGet(), doPost( ) and service( ) ) of the servlet. For this reason, a single servlet object can have its service methods called by many threads at once.

96)          What is the difference between TCP/IP and UDP?

Ans: TCP/IP is a two-way communication between the client and the server and it is a reliable and there is a confirmation regarding reaching the message to the destination. It is like a phone call. UDP is a one-way communication only between the client and the server and it is not a reliable and there is no confirmation regarding reaching the message to the destination. It is like a postal mail.

97)          What is Inet address?

Ans: Every computer connected to a network has an IP address. An IP address is a number that uniquely identifies each computer on the Net. An IP address is a 32-bit number.

Posted in Java_FAQ's ]]> http://meherchilakalapudi.wordpress.com/2009/07/16/java-faqs/feed/ 0 meherchilakalapudi http://meherchilakalapudi.wordpress.com/2009/06/04/save-earth-save-trees-save-water/ http://meherchilakalapudi.wordpress.com/2009/06/04/save-earth-save-trees-save-water/#comments Thu, 04 Jun 2009 13:52:08 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=676 ]]>

The Hydrosphere

The hydrosphere is often called the “water sphere” as it includes all the earth’s water found in streams, lakes, the soil, groundwater, and in the air. The hydrosphere interacts with, and is influenced by, all the other earth spheres. The water of the hydrosphere is distributed among several different stores found in the other spheres. Water is held in oceans, lakes and streams at the surface of the earth. Water is found in vapor, liquid and solid states in the atmosphere. The biosphere serves as an interface between the spheres enabling  water to move between the hydrosphere, lithosphere and atmosphere as is accomplished by plant transpiration. The hydrologic cycle traces the movement of water and energy between these various stores and spheres.

 

 

                        Figure TH.1 Earth Spheres/Systems

Distribution of water

The world’s oceans contain 97% of the water in the hydrosphere, most of which is salt water. Ice caps, like that found covering Antarctica, and glaciers that occupy high alpine locations, compose a little less than 2% of all water found on earth. Seemingly a small amount, the water stored as ice in glaciers would have a great impact on the environment if it were to melt into a liquid. One fear is that global warming will cause the melting and collapse of large ice sheets resulting in sea level rise. Rising sea levels could devastate coastal cities, displace millions of people, and wreak havoc on freshwater systems and habitats.

 

 

Figure TH.2 The largest store of water is the ocean which delivers water through evaporation each day

 

Water beneath the surface comprises the next largest store of water. Groundwater and soil water together make up about .5% of all water (by volume). There is a difference between ground water and soil water. Soil water is the water held in pore spaces between soil particles. Soil pore spaces usually are partially void of water most of the time but fill with water after a rain storm. Groundwater, on the other hand, is found where earth materials are saturated throughout the year. That is, the pore spaces are always occupied with water. Both soil and groundwater are very important sources of water. Soil water is available for plants to extract and use. Groundwater is an important source of water for irrigation and drinking water supplies.

Above the surface water is found stored in streams, rivers and lakes. One might expect that given the large rivers that flow across the earth and the huge numbers of lakes that this store would be rather large. Instead, streams, rivers and lakes only comprise .02% of all water in the earth system. In the atmosphere, only about .0001 % of the water in the hydrosphere is found.

 

The Hydrologic Cycle

 

 

 

        The Hydrologic Cycle involves the continuous circulation of water in the Earth-atmosphere system. Of the many processes involved in the hydrologic cycle, the most important are

• evaporation,
• transpiration,
• condensation,
• precipitation, and
• runoff

The hydrologic cycle consists of inflows, outflows, and storage. Inflows add water to the different parts of the hydrologic system, while outflows remove water. Storage is the retention of water by parts of the system. Because water movement is cyclical, an inflow for one part of the system is an outflow for another. Looking at an aquifer as an example, percolation of water into the ground is an inflow to the aquifer. Discharge of ground water from the aquifer to a stream is an outflow (also an inflow for the stream). Over time, if inflows to the aquifer are greater than its outflows, the amount of water stored in the aquifer will increase. Conversely, if the inflows to the aquifer are less than the outflows, the amount of water stored decreases. Inflows and outflows can occur naturally or result from human activity.

 

Evaporation
Evaporation is the change of state in a substance from a liquid to a gas. In meteorology, the substance we are concerned about the most is water. For evaporation to take place, energy is required. The energy can come from any source; the sun, the atmosphere, the earth, or objects on the earth such as humans.

Everyone has experienced evaporation personally. When the body heats up due to the air temperature or through exercise, the body sweats, secreting water onto the skin. The purpose is to cause the body to use its heat to evaporate the liquid, thereby removing heat and cooling the body. It is the same effect that can be seen when you step out of a shower or swimming pool. The coolness you feel is from the removing of bodily heat to evaporate the water on your skin.

Transpiration
Transpiration is the evaporation of water from plants through stomata. Stomata are small openings found on the underside of leaves that are connected to vascular plant tissues. In most plants, transpiration is a passive process largely controlled by the humidity of the atmosphere and the moisture content of the soil. Of the transpired water passing through a plant only 1% is used in the growth process of the plant. The remaining 99% is passed into the atmosphere.

Condensation
Condensation is the process whereby water vapor in the atmosphere is returned to its original liquid state. In the atmosphere, condensation may appear as clouds, fog, mist, dew or frost, depending upon the physical conditions of the atmosphere. Condensation is not a matter of one particular temperature but of a difference between two temperatures; the air temperature and the dewpoint temperature.

 

Precipitation
Precipitation is the result when the tiny condensation particles grow too large, through collision and coalesce, for the rising air to support, and thus fall to the earth.
Runoff
Runoff occurs when there is excessive precipitation and the ground is saturated (cannot absorb anymore water). This runoff flows into streams and rivers and eventually back into the sea.

Evaporation of this runoff into the atmosphere begins the hydrologic cycle over again. Some of the water perculates into the soil and into the ground water only to be drawn into plants again for transpiration to take place.

 

 

 

                                             lithosphere

 

The lithosphere^[1] is the solid shell of a rocky planet called earth. That means the crust and the upper part of the mantle which is joined to the crust (see picture on the right).

Under the lithosphere there is the asthenosphere, the weaker, hotter, and deeper part of the upper mantle.

 

 

The lithosphere is the surface layer of the fluid parts of the Earth’s convection system, therefore it thickens over time. It is broken up into pieces called plates (shown in picture on the left), which move independently relative to one another. This movement of lithospheric plates is described as plate tectonics.

The division of Earth’s outer layers into lithosphere and asthenosphere should not be confused with the chemical subdivision of the outer Earth into mantle, and crust. All crust is in the lithosphere, but lithosphere generally contains more mantle than crust.
There are two types of lithosphere:

• Oceanic lithosphere, which is associated with Oceanic crust
• Continental lithosphere, which is associated with Continental crust

Oceanic lithosphere is typically about 50-100 km thick (but beneath the mid-ocean ridges is no thicker than the crust). Continental lithosphere is thicker (about 150 km). It consists of about 50 km of crust and 100 km or more of uppermost mantle.

Oceanic lithosphere consists mainly of mafic crust and ultramafic mantle and is denser than continental lithosphere, for which the mantle is associated with crust made of felsic rocks. The crust is distinguished from the upper mantle by the change in chemical composition that takes place at the Moho discontinuity. Oceanic lithosphere thickens as it ages and moves away from the mid-ocean ridge. This thickening occurs by conductive cooling, which converts hot asthenosphere into lithospheric mantle, and causes the oceanic lithosphere to become increasingly dense with age. Oceanic lithosphere is less dense than asthenosphere for a few tens of millions of years, but after this becomes increasingly denser than asthenosphere. The gravitational instability of mature oceanic lithosphere has the effect that at subduction zones the oceanic lithosphere invariably sinks underneath the overriding lithosphere, which can be oceanic or continental. New oceanic lithosphere is constantly being produced at mid-ocean ridges and is recycled back to the mantle at subduction zones. As a result, oceanic lithosphere is much younger than continental lithosphere: the oldest oceanic lithosphere is about 170 million years old, while parts of the continental lithosphere are billions of years old.

Another distinguishing characteristic of the lithosphere is its flow properties. Under the influence of the low-intensity, long-term stresses that drive plate tectonic motions, the lithosphere responds essentially as a rigid shell and thus deforms primarily through brittle failure, whereas the asthenosphere (the layer of the mantle below the lithosphere) is heat-softened and accommodates strain through plastic deformation

 

 

Posted in Save Environment ]]> http://meherchilakalapudi.wordpress.com/2009/06/04/save-earth-save-trees-save-water/feed/ 0 meherchilakalapudi 1 2 3 4 http://meherchilakalapudi.wordpress.com/2009/05/30/know-about-biometric-authentication-system/ http://meherchilakalapudi.wordpress.com/2009/05/30/know-about-biometric-authentication-system/#comments Sat, 30 May 2009 14:06:28 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=674 ]]>

   

BIOMETRIC AUTHENTICATION SYSTEM

 

          

 

 

 

BIOMETRIC AUTHENTICATION SYSTEM

 

 

      A biometric authentication system provides automatic identification of an individual based on a unique feature or characteristic possessed by the individual. Iris recognition is regarded as the most reliable and accurate biometric authentication system available. Most commercial iris recognition systems use patented algorithms developed by Daugman, and these algorithms are able to produce perfect recognition rates. However, published results have usually been produced under favorable conditions, and there have been no independent trials of the technology. The paper involves the development of an iris recognition system in order to verify both the uniqueness of the human iris and also its performance as a biometric even at abnormal conditions. For determining the recognition performance of the system two digitized grayscale eye images are used. The iris recognition system consists of an automatic segmentation system that is based on the Hough transform, and is able to localize the circular iris and pupil region, occluding eyelids and eyelashes, and reflections. The extracted iris region is then normalized into a rectangular block with constant dimensions to account for imaging inconsistencies. Finally, the phase data from 1D Log-Gabor filters is extracted and quantised to encode the unique pattern of the iris into a bitwise biometric template. The Hamming distance is employed for classification of iris templates, and two templates are found to match if a test of statistical independence is failed. Based on Hamming distance the threshold is set between false acceptance and false rejection. Therefore, iris recognition is shown to be a reliable and accurate biometric technology.

INTRODUCTION

BIOMETRIC TECHNOLOGY:

    A biometric system provides automatic recognition of an individual based on some sort of unique feature or characteristic possessed by the individual. Biometric systems have been developed based on fingerprints, facial features, voice, hand geometry, handwriting, the retina, and the one presented in this thesis, the iris. Biometric systems work by first capturing a sample of the feature, such as recording a digital sound signal for voice recognition, or taking a digital colour image for face recognition. The sample is then transformed using some sort of mathematical function into a biometric template. The biometric template will provide a normalized, efficient and highly discriminating representation of the feature, which can then be objectively compared with other templates in order to determine identity. Most biometric systems allow two modes of operation. An enrolment mode for adding templates to a database, and an identification mode, where a template is created for an Individual and then a match is searched for in the database of pre-enrolled templates. A good biometric is characterized by use of a feature that is highly unique, therefore the chance of any two people having the same characteristic will be minimal stable, so that the feature does not change over time, and be easily captured in order to provide convenience to the user, and prevent misrepresentation of the feature.

THE HUMAN IRIS

The iris is a thin circular diaphragm, which lies between the cornea and the lens of the human eye. A front-on view of the iris is shown in Fig.1. The iris is perforated close to its centre by a circular aperture known as the pupil. The function of the iris is to control the amount of light entering through the pupil, and this is done by the sphincter and the dilator muscles, which adjust the size of the pupil. The average diameter of the iris is 12 mm, and the pupil size can vary from 10% to 80% of the iris diameter. The iris consists of a number of layers, the lowest being the epithelium layer, which contains dense pigmentation cells. The stromal layer lies above the epithelium layer, and contains blood vessels, pigment cells and the two iris muscles. The density of stromal pigmentation determines the colour of the iris. The externally visible surface of the multilayered iris contains two zones, which often differ in colour.

     They are an outer ciliary zone and an inner pupillary zone, and these two zones are divided by the collarette, which appears as a zigzag pattern. Formation of the iris begins during the third month of embryonic life. The unique pattern on the surface of the iris is formed during the first year of life, and pigmentation of the stroma takes place for the first few years. Formation of the unique patterns of the iris is random and not related to any genetic factors. The only characteristic that is dependent on genetics is the pigmentation of the iris, which determines its color. Due to the epigenetic nature of iris patterns, the two eyes of an individual contain completely independent iris patterns, and identical twins possess uncorrelated iris patterns, and identical twins possess uncorrelated iris patterns.

 

IRIS RECOGNITION

              

 

The iris is an externally visible, yet protected organ whose unique epigenetic pattern remains stable throughout adult life. This characteristic makes it very attractive for use as a biometric for identifying individuals. Image processing                                                                 

Fig.1: A Front-on-view on the human eye     techniques can be employed to extract the unique iris pattern from a digitized image of the eye, and encode it into  a  biometric template, which can be stored in a database. This biometric template contains an objective mathematical representation of the unique information stored in the iris, and allows comparisons to be made between templates. When a person is to be identified by an iris recognition system, his eye is first photographed, and then a template created forthe iris region. This template is then compared with the other templates stored in a data base until a matching template is found and the person is identified. If no match is found the person remains unidentified. The Daugman system has been tested under numerous studies, all reporting a zero failure rate. The Daugman system is claimed to be able to perfectly identify an individual, given millions of possibilities.   Fig: 2: Block Diagram of Iris Recognition System.

  However, there have been no independent trials of the technology, and source code for systems is not available. Also, there is a lack of publicly available datasets for testing and research, and the test results published have usually been produced using carefully imaged irises under favorable conditions.

 SEGMENTATION

     The first stage of iris recognition is to isolate the actual iris region in a digital eye image. Two circles can approximate the iris region, shown in Fig.3, one for the iris/sclera boundary and another, for the iris/pupil boundary (interior to the first). The eyelids and eyelashes normally occlude the upper and lower parts of the iris region. Also, specular reflections can occur within the iris region corrupting the iris pattern. A technique is required to isolate and exclude these artifacts as well as to locate the circular iris region. The success of segmentation depends on the imaging quality of eye images. Also, persons with darkly pigmented irises will present very low contrast between the pupil and iris region if imaged under natural light, making segmentation more difficult. The segmentation stage is critical to the success of an iris recognition system, since data that is falsely represented, as iris pattern data will corrupt the biometric templates generated, resulting in poor recognition rates. 

IMPLEMENTATION

Here the circular Hough transform was used for detecting the iris and pupil boundaries. This involves employing Canny edge detection to generate an edge map. Gradients were biased in the vertical direction for the outer iris/sclera boundary. Vertical and horizontal gradients were weighted equally for the inner iris/pupil boundary. The range of radius values to search for was set manually, values of the iris radius range from 90 to 150 pixels, while the pupil radius ranges from 28 to 75 pixels. In order to make the circle detection process more efficient and accurate, the Hough transform for the iris/sclera boundary was performed first, then the Hough transform for the iris/pupil boundary was performed within the iris region, instead of the whole eye region, since the pupil is always within the iris region. After this process was complete, six parameters are stored, the radius, and x & y center coordinates for both circles. A line is fit to the upper and lower eyelid using the linear Hough transform isolated eyelids. Then a second horizontal line is drawn, which intersects with the first line at the iris edge that is closest to the pupil. This process is illustrated in Fig.3 and is done for both the top and bottom eyelids. The second horizontal line allows maximum isolation of eyelid regions. Canny edge detection is used to create an edge map, and only horizontal gradient information is taken. The linear Hough transform is implemented using the MATLAB Radon transform, which is a form of the Hough transform. If the maximum in the Hough space is lower than a set threshold, then no line is fitted, since this corresponds to non-occluding eyelids.

Also, the lines are restricted to lie exterior to the pupil region, and interior to the iris region. A linear Hough transform has the advantage over its parabolic version, because it has lesser parameters to deduce, making the process lesser computationally demanding.

 

 

 

 

 

 

 

 

                           Fig.3 Stages of segmentation with eye image.

For isolating eyelashes, a simple thresholding technique was used, since analysis reveals that eyelashes are quite dark when compared to the rest of the eye image. Although, the eyelashes are quite dark compared with the surrounding eyelid region, areas of the iris region are equally dark due to the imaging conditions. Therefore thresholding to isolate eyelashes would also remove important iris region features, making this technique infeasible.

NORMALISATION

     Once the iris region is successfully segmented from an eye image, the next step is to transform the iris region so that it has fixed dimensions in order to allow comparisons. The dimensional inconsistencies between eye images are mainly due to the stretching of the iris caused by pupil dilation from varying levels of illumination. Other sources of inconsistency include, varying imaging distance, rotation of the camera, head tilt, and rotation of the eye within the eye socket. The normalisation process will produce iris regions, which have the same constant dimensions, so that two photographs of the same iris under different conditions will have characteristic features at the same spatial location.

IMPLEMENTATION

For normalisation of iris regions, a technique based on Daugman’s rubber sheet model is employed. The centre of the pupil is considered as the reference point, and radial vectors pass through the iris region, as shown in Fig. 4. A number of data points are selected along each radial line and this is defined as the radial resolution. The number of radial lines going around the iris region is defined as the angular resolution. Since the pupil can be non-concentric to the iris, a remapping formula is needed to rescale points depending on the angle around the circle.

This is given by

 

With 

 

 

where displacement of center of the pupil relative to center of the iris is given by Q_x, Q_y and r² the distance between the edge of the pupil and edge of the iris at an angle, q around the region, and r₁ is the radius of the iris. The remapping formula gives the radius of the iris region as a function of the angle q same number of points are chosen along each radial line, irrespective of how narrow or wide the radius is at a particular angle. Backtracking to find the Cartesian coordinates of data points from the radial and angular position in the normalized pattern creates the normalized pattern. From the iris region, normalization produces a 2D array with horizontal dimensions of angular resolution and vertical dimensions of radial resolution.

Another 2D array is created for marking reflections, eyelashes, and eyelids detected in the segmentation stage. In order to prevent non-iris region data from corrupting the normalized representation, data points that occur along the pupil border or the iris border are discarded. As in Daugman’s rubber sheet model, removing rotational inconsistencies is performed at the matching stage.

 

 

 

 

 

 

Fig.4. Outline of the Normalisation process.

FEATURE ENCODING AND MATCHING

    In order to provide accurate recognition of individuals, the most discriminating information present in an iris pattern must be extracted. Only the significant features of the iris must be encoded so that comparisons between templates can be made. The template that is generated in the feature encoding process will also need a corresponding matching metric, which gives a measure of similarity between two iris templates. This metric should give one range of values known as intra-class comparisons, when comparing templates generated from the same eye, and another range of values, known as inter-class comparisons when comparing templates created from different irises. These two cases should give distinct and separate values, so that a decision can be made with high confidence as to whether two templates are from the same iris, or from two different irises.

IMPLEMENTATION

  FEATURE ENCODING

Fig.5. Phase Quantisation

   Feature encoding is implemented by convolving the normalized iris pattern with 1D Log-Gabor wavelets. The 2D normalised pattern is broken up into a number of 1D signals, and then these 1D signals are convolved with 1D Gabor wavelets. The rows of the 2D normalized pattern are taken as the 1D signal in which each row corresponds to a circular ring on the iris region. Since maximum independence occurs in the angular direction, the angular direction is taken rather than the radial one, which corresponds to columns of the normalised pattern. The intensity values at known noise areas in the normalised pattern are set to the average intensity of surrounding pixels to prevent influence of noise in the output of the filtering. The output of filtering is then phase quantised to four levels using the Daugman method , with each filter producing two bits of data for each phasor. The output of phase quantisation is chosen to be a gray code, so that while going from one quadrant to another, only 1 bit changes. This will minimize the number of bits disagreeing, if say two intra-class patterns are slightly misaligned and thus will provide more accurate recognition. The feature encoding process is illustrated in Fig. 6. The encoding process produces a bitwise template containing a number of bits of information, and a corresponding noise mask, which corresponds to corrupt areas within the iris pattern, and marks bits in the template as corrupt. Since the phase information will be meaningless at regions where the amplitude is zero, these regions are also marked in the noise mask. The total number of bits in the template will be product of the angular resolution, the radial resolution, and the number of filters used. The number of filters, their center frequencies and parameters of the modulating Gaussian function in order to achieve the best recognition rate.

   Sample Iris Template:

 

 

 

Fig.6: An illustration of the feature encoding process.

MATCHING

  For matching, the Hamming distance is chosen as a metric for recognition, since bit-wise comparisons are necessary. The Hamming distance algorithm employed also incorporates noise masking, so that only significant bits are used in calculating the Hamming distance between two iris templates. Now when taking the Hamming distance, only those bits in the iris pattern that correspond to ‘0’ bits in noise masks of both iris patterns will be used in the calculation. The Hamming distance will be calculated using only the bits generated from the true iris region, and this modified Hamming distance formula is given as :

 

 

 

 

 

Where Xj and Yj are the two bit-wise templates to be compared, Xnj and Ynj are the corresponding noise masks for Xj and Yj, and N is the number of bits represented by each template. Although, in theory, two iris templates generated from the same iris will have a Hamming distance of 0, in practice this will not occur. Normalisation is not perfect, and also there will be some noise that goes undetected, so that some variation will be present when comparing two intra-class iris templates. In order to account for rotational inconsistencies, when the Hamming distance of two templates is calculated, one template is shifted left and right bitwise and a number of Hamming distance values are calculated from successive shifts. This bit-wise shifting in the horizontal direction corresponds to rotation of the original iris region by an angle given by the angular resolution used. If an angular resolution of 180 degrees is used, each shift will correspond to a rotation of 2 degrees in the iris region. This method is suggested by Daugman, and corrects misalignments in the normalised iris pattern caused by rotational differences during imaging. From the calculated Hamming distance values, only the lowest is taken, since this corresponds to the best match between two templates. The number of bits moved during each shift is given by two times the number of filters used, since each filter will generate two bits of information from one pixel of the normalised region. The actual number of shifts required to normalize rotational inconsistencies will be determined by the maximum angular difference between two images of the same eye. One shift is defined as one shift to the left, followed by one shift to the right. In this example one filter is used to encode the templates, so only two bits are moved during a shift. The lowest Hamming distance, in this case zero, is then used since this corresponds to the best match between the two templates.

 

 

 

 

 

CONCLUSION

 In this paper an iris recognition system has been presented using grayscale eye images in order to check the desired performance of iris recognition technology. Firstly, an automatic segmentation algorithm was presented, which would localise the iris region from an eye image and isolate eyelid, eyelash and reflection areas. Automatic segmentation was achieved through the use of the circular Hough transform for localizing the iris and pupil regions and the linear Hough transform for localising occluding eyelids.

Thresholding was also employed for isolating eyelashes and reflections. Next, the segmented iris region was normalised to eliminate dimensional inconsistencies between iris regions. This was achieved by implementing a version of Daugman’s rubber sheet model, where the iris is modelled as a flexible rubber sheet, which is unwrapped into a rectangular block with constant polar dimensions. Finally, features of the iris were encoded by convolving the normalised iris region with 1D Log-Gabor filters and phase quantising the output in order to produce a bitwise biometric template. The Hamming distance was chosen as a matching metric, which gave a measure of bits disagreed between two templates. A failure of statistical independence between two templates would result in a match i.e., the two templates were deemed to have been generated from the same iris if the Hamming distance produced was lower than a set Hamming distance.

Posted in KNOW ABOUT - BIOMETRIC AUTHENTICATION SYSTEM ]]> http://meherchilakalapudi.wordpress.com/2009/05/30/know-about-biometric-authentication-system/feed/ 0 meherchilakalapudi 1 2 3 4 5 6 7 8 http://meherchilakalapudi.wordpress.com/2009/05/26/pointer-and-files/ http://meherchilakalapudi.wordpress.com/2009/05/26/pointer-and-files/#comments Tue, 26 May 2009 13:37:28 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=652 ]]>

FUNCTIONS

———

  Function:

            It is a self contained block of statements and is used several

  times in a program but defined only once.

  library function:-

            The functions which are inbuilt with the compiler is called as

            library function.

         ex:- printf(),scanf(),etc…..

  User defined functions:-

            User can defined functions to do a task relavent to their program.

   such functions are called user defined functions.

 

       Any function has 3 things.

       1) Function declaration

       2) Function definition

       3) Function calling.

 

       In case of library functions the function declaration is in headers,

       definition is in C libraries and calling is in your source program.

 

       In case of user defined functions all the three things are in your

       source program.

Function declaration:-

              syn:-  returntype  func_name([arg_list]);

              ex:-  void sum(int,int);

Function definition:-

               syn:-  returntype func_name([arg_list])

                      {

                        body;

                      }

Function calling:-

               syn:-  func_name([arg_list]);

               ex:-  sum(a,b);

NOTE:-  The arguments which are given at the time of function declaration and

          function definion are called arguments (or) formal arguments.

        The arguments which are given at the time of function calling

          are known as parameters (or) actual arguments.

Void :-  Empty data type.

 

RULES FOR CREATING AND ACCESSING USER DEFINED FUNCTIONS:

——————————————————-

1.A function can be called by any no of times.

2.A function may or may not return a value.

3.A function may or may not receive arguments

4.If a function does not return any value,the function will be specified as

   void.

5.If a function returns a value only one value can be returned.

6.If a function returns a value the returning value must be returned with a

  statement “return”.

7.If a function returns a value the execution of return statement should be

  last.

8.A function returns an integer value by default;

9.If a function returns a value , the returning value should be match with

  the function return data type.

10.A function is defined after (or) before main function.

11.Before calling a function , the function declaration (or) definition

    is must and should.

12.If a function declaration is specified before the function call,The function

    definition can be specified any where in the program.

13.If a function definition is specified before the function call,then the

   function declaration is not necessary.

14.A function is executed when the function is call by its name.

15.The function defintion should not be terminated with semicolon.

 

RETURN:-

         Exits immediately from the corresponding executing function to the

         calling. return  Optionlly returning a value.

         syntax:-  return [<expression>];

 

Function prototypes (or) Function categories:-

———————————————-

   A Function depending on whether arguments are present (or) not and whether

a value is returning or not .They belong to one of the following categories.

  1)Function with no arguments and no return value

  2)Function with arguments and no return value.

  3)Function with arguments and return value.

  4)Function with no orguements and return value.

 

Recursive function:-

——————-

   Calling a function with in same function is known as recursion (or)

Recursive function.If we want to work with recursive function we must follow

the two conditions.

  1. Termination condition.

  2. Calling by itself.

 

POINTERS

——–

C provides the important feature of data manipulations with the address of

the variables,hence the execution time is very much reduced such concept is

possible with the special data type called Pointers.

Pointer:-

   A pointer is a variable which stores the address of anther variable.

Declaration:-

   Pointer declaration is similar to normal variable declaration but

   preceded by a *

   syn:- data type  *identifier;

   ex:- int *p;

Initialization:-

   datatype *identifier=address;

   ex:- int n;

        int *p=&n;

NOTE:-  Any type of pointer it allocates 2 bytes momory.Because it stores

address of memory location.

 

Void *:-

       It is a generic pointer.It refers the address of any type of variable

 and also it will assigned to any type of pointer.

 ex:-   int n=100;

        int *p=&n;

 

FUNCTIONS AND POINTERS

———————-

 Call by value:-

 ————–

       The process of passing a value to the function call is known as

       call by value.

Call by Reference:-

——————-

       The process of calling a function using pointers to pass the address

       of the variables is known as call by reference.

 

                 

 

 

POINTERS  AND  ARRAYS

———————

        When an array is declared,the compiler allocates a BASE address and

     sufficient amount of storage and contain all the elements of array

     in continuous memory allocation.

      The base address is the location of the first element (index 0 of

      the array).

      The compiler also defines the array name as a constant pointer pointed

      to the first element.

      suppose we declare an array ‘a’ as follows.

         int a[5]={1,2,3,4,5};

         suppose the base address of a is 1000 and assuming that each integer

         requires 2 bytes.Then the 5 elements will be stored as follows.

    Elements——>  a[0] a[1] a[2] a[3] a[4]

                     ————————

    values——->     1    2    3    4    5

                     ————————

    address——>   1000 1002 1004 1006 1008

                      |

                 base address

 

        Here ‘a’ is a pointer that points to the first element. so the value

        of a is 1000

           there fore  a=&a[0]=1000

        If we declare ‘p’ is an integer pointer,then we can make the pointer

        p to point to the array ‘a’ by the following assignment.

 

                int *p;

                p=&a[0]=a;

        Now we can access every value of a using p++(or)p+1 to move one

        element to another.

 

        The relationship between p and a is shown below.

        p+0 = &a[0]=1000

        p+1 = &a[1]=1002

        p+2 = &a[2]=1004

        p+3 = &a[3]=1006

        p+4 = &a[4]=1008

 

        When handling arrays instead of using array indexing,we can use pointer

        to access array elements.

        Note that *(p+k) gives the value of a[k].

        Pointer accessing method is much faster than array indexing.

 

DYNAMIC MEMORY ALLOCATION

————————-

       C language requires the no of elements in an array to  be specified

       at compile time.But we may not be able to do so always our initial

       judgement of size,if it is wrong,it make cause failure of the program

       (or) wastage of the memory space.In this situation we use Dynamic

       Memory allocation.

       Definition:-

                   The process of allocating memory at run time is known as

                   Dynamic memory allocation.

       Malloc():-              <alloc.h>

                It is a function which is used to allocating memory at

                 run time.

         syn:-    void *malloc(size_t size);

           size_t:-  unsigned integer. this is used for memory object sizes.

 

       Pointer variable=(type casting)malloc(memory size);

     Ex:-    int *p;

              p=(int *)malloc(sizeof(int));  //for 1 location

              p=(int *)malloc(n*sizeof(int));  //for n locations

 

       Calloc():-            <alloc.h>

                This is also used for allocating memory at run time.

        syn:-   void *calloc(size_t nitems, size_t size);

 

      NOTE:- Calloc allocates a block (n times*size)bytes and clears into 0.

      Realloc():- It reallocates Main memory.

         syn:- void *realloc(void *block, size_t size);

 

      free():- It deallocates the allocated memory.

        syn:-   void free(void *block);

 

                                    FILES

                                    —–

   C language permits the usage of limited input and output functions to read

and write data.

    These functions are used for only smaller volumes of data and it becomes

difficult to handle longer data volumes.Also the entire data is lost when the

program is over.

    To overcome these difficulties a flexible method is devoloped by employing

the concept of FILES to store,to read (0r) write data and to written them even

when program is over.

Definition:-

           A FILE is one which enable the user to read , write and store a

     group of a related data .  (or)

           A FILE is a collection of related data stored in a perticular area

     On the disk.

  C supports a no of functions to have the ability to perform the basic file

  operations which indicates.

           1) Naming a file

           2) Opening a file

           3) Reading data from a file

           4) Writing data into file

           5) Closing a file

 

FILE:-  It is a predefined structure and is used to declare a file pointer.

        Using this pointer we can perform all file operations.

 Declaration of file pointer:-

              FILE  *identifier;

              ex:-  FILE *fp;

TEXT FILES:-

            For reading and writing data in continuous blocks.

BINARY FILES:-

            For reading and writing data in orbitrary structured files.

fopen():-

         It opens a file stream.

         syn:- FILE *fopen(const char *filename,const char *mode);

file name:-  File that the functions open.

MODE:-

String ³ Description

***********************************************************************************

   r   ³ Open for reading only

   w   ³ Create for writing

       ³ If a file by that name already exists, it will be overwritten.

   a   ³ Append; open for writing at end of file, or create for

       ³ writing if the file does not exist.

   r+  ³ Open an existing file for update (reading and writing)

   w+  ³ Create a new file for update (reading and writing).

       ³ If a file by that name already exists, it will be overwritten.

   a+  ³ Open for append; open for update at the end of the file, or

       ³ create if the file does not exist.

 

 þ To specify that a given file is being opened or created in text mode,

   append “t” to the string (rt, w+t, etc.).

 þ To specify binary mode, append “b” to the string (wb, a+b, etc.).

 

 fclose():-    It closes a file stream

         syn:- int fclose(FILE *stream);

 fcloseall():- It closes all opened file streams.

         syn:- int fcloseall(void);

 

TEXT FILES

———-

Macros:-

——–

1)getc():- It is a macro that gets one character from a stream.

           syn:- int getc(FILE *stream);

2)putc():- It is a macro that out puts a character to a file stream.

           syn:- int putc(int c,FILE *stream);

Functions:-

———-

1)fgetc():- It is a function.It gets a character from a file stream.

           syn:- int fgetc(FILE *stream);

2)fputc():- It is a function.It outputs a character to a file stream.

           syn:- int fputc(int c,FILE *stream);

EOF:-  A constant indicating that end of file has been reached on a file.

      To get this character from keyboard press ctrl+z.

 

fgets():-

         It gets string from file stream.

         syn:-  char *fgets(char *s,int n,FILE *stream);

remarks:-

———

fgets reads characters from stream into the string s. It stops when it reads

either n – 1 characters or a newline character, whichever comes first.

 

fputs():-

          It outputs a string to file stream.

          syn:- int fputs(const char *s,FILE *stream);

Remarks:-

———-

fputs copies the null-terminated string s to the given output stream. It

does not append a newline character, and the terminating null character is

not copied.

 

feof():-

        It is a Macro that tests if end of file has been reached on a stream.

        syn:-  int  feof(FILE *stream);

returned value:-

—————

 ->  Returns non-zero if an end-of-file

     indicator was detected on the last input

     operation on the named stream.

 ->  Returns 0 if end-of-file has not been

     reached.

 

Rewind():-

            It  Repositions file pointer to stream’s beginning.

            syn:- void rewind(FILE *stream);

fscanf():-

          It scans and formats input from a file stream.

syn:- int fscanf (FILE *stream,const char *format [, address, ...]);

 

fprintf():-

           It sends formatted output to the file stream.

syn:-  int fprintf (FILE *stream, const char *format [, argument, ...]);

 

NOTE:-In fprintf (or) fscanf, between formats atleast one space is neccessary.

 

ftell():- It Returns the current file pointer.

   syntax:-  long ftell(FILE *stream);

 

fseek():- It repositions the file pointer of a stream.

  syntax:-  int fseek(FILE *stream,long offset,int whence);

Argument³ What It Is/Does

**************************************************************************

 stream ³ Stream whose file pointer fseek sets

 offset ³ Difference in bytes between whence (a file pointer position)

        ³ and new position. For text mode streams, offset should be 0 or

        ³ a value returned by ftell.

 whence ³ One of three they are

 

 Constant ³Value³ File location

*******************************************************************************

 SEEK_SET ³  0  ³ Seeks from beginning of file

 SEEK_CUR ³  1  ³ Seeks from current position

 SEEK_END ³  2  ³ Seeks from end of file

 

BINARY FILES

————

putw():-

——-

        It outputs an integer on a stream.

        syn:-  int putw(int w,FILE *stream);

getw():-

        It gets an integer from stream.

        syn:- int getw(FILE *stream);

 

fwrite():-

        It appends specified no of equal sized data items to an output file.

    syn:-size_t fwrite(const void *ptr, size_t size, size_t n, FILE*stream);

 

fread():-

       It reads a specified no of equal sized data items from an input stream

       into a block.

  syn:- size_t fread(void *ptr, size_t size, size_t n, FILE *stream);

 

Argument³ What It Is/Does

***********************************************************************************

ptr    ³ Points to a block into which data is read/write

 size   ³ Length of each item read/write, in bytes

 n      ³ Number of items read/write

 stream ³ Points to input/output stream

 

 remove():-

           It is a Macro that removes a file

           Declaration:  int remove(const char *filename);

 rename():-

           It Renames a file

           Declaration:  int rename(const char *oldname, const char *newname);

 

STORAGE CLASSES

—————

        By the declaration statement the memory is allocated temporarly for

all the variables.The size of momory varies with respect to the type of the

variable.The availability of variables for access depends on it’s declaration

time.the storage class specifiers are used to specify the life, scope of

variables with in the block , functions and the entire program.There are

four types of storage class specifiers.

        They are

        1. Automatic variables

        2. Static variables

        3. external variables (or) global variables

        4. Register variables.

 

1. Automatic variables :-

        These variables are declared inside the function block.

 

        storage : Main memory

        Default value : garbage value

        Scope  : Local to the block in which it is defined

        Life   : Till the control remaines with in the block in which it

                 it is defined

        Keyword : auto

NOTE:-

        If there is no storage class specifier before the declaration of any

variable inside a function block by default it takes auto storage class.

#include<stdio.h>

#include<conio.h>

void main()

{

        auto int a,b;

        clrscr();

        printf(“\na=%d”,a);

        printf(“\nb=%d”,b);

        getch();

}

o/p : a = garbage value

      b = garbage value

 

2. Static variables:-

        The memory of static variable remaines unchanged until the end of the

        program.

        storage : Main memory

        default value : zero

        scope   : Local to the block in which it is defined

        life    : The value of static variable persists between the different

                    function calls.i.e.., It can not re-initialize between

                    the different function calls.

        keyword  : static

 

#include<stdio.h>

#include<conio.h>

void main()

{

        static int a,b;

        clrscr();

        printf(“\na=%d”,a);

        printf(“\nb=%d”,b);

        getch();

}

o/p :   a = 0

        b = 0

 

#include<stdio.h>

#include<conio.h>

void main()

{

        void disp();

        int i;

        clrscr();

        for(i=1;i<=10;i++)

        {

                disp();

        }

        getch();

}

void disp()

{

        static int n=1;

        printf(“%5d”,n);

        n++;

}

 

o/p :-   1 2 3 4 5 6 7 8 9 10

 

NOTE :-

        If we can not use static storage class specifier in the above program

        o/p:  1 1 1 1 1 1 1 1 1 1

 

3. External variables (or) Global variables:-

        The variables that are both alive and active through out the entire

        program are known as external variables.

 

        storage : Main memory

        default value : zero

        scope   : global

        life    :as long as the execution of the program does not comes to end

        keyword : extern

 

#include<stdio.h>

#include<conio.h>

int a,b;

void main()

{

        clrscr();

        printf(“\na=%d”,a);

        printf(“\nb=%d”,b);

        getch();

}

o/p : a = 0

      b = 0

 

#include<stdio.h>

#include<conio.h>

void main()

{

        extern int n=100;

        clrscr();

        printf(“\n n=%d”,n);

        getch();

}

o/p : n = 100

 

4. Register variables: -

        We can use the register variables for frequently used variables to

        Improve faster execution of the program.

 

        storage :   CPU register

        default value : garbage value

        scope : Local to the block in which it is defined

        life  : Till the control remaines with in the block in which it is defined.

        keyword: register

 

Note : These variables are also declared inside the function block.

 

#include<stdio.h>

#include<conio.h>

void main()

{

        register int a,b;

        clrscr();

        printf(“\na =%d”,a);

        printf(“\nb =%d”,b);

        getch();

}

o/p : a = garbage value

      b = garbage value

 

Note :

 The register variables supports only integral data type ( int (or) char)

Posted in Pointers_&_Files ]]> http://meherchilakalapudi.wordpress.com/2009/05/26/pointer-and-files/feed/ 0 meherchilakalapudi http://meherchilakalapudi.wordpress.com/2009/05/22/knee-carewhile-doing-work-at-computers/ http://meherchilakalapudi.wordpress.com/2009/05/22/knee-carewhile-doing-work-at-computers/#comments Fri, 22 May 2009 06:41:32 +0000 meherchilakalapudi http://meherchilakalapudi.wordpress.com/?p=628 ]]>

1. INTRODUCTION

 

INTRODUCTION TO OSTEOARTHRITIS OF THE KNEE

There are many types of arthritis (rheumatoid, degenerative, post-traumatic, auto-immune induced, etc). The most common form is osteoarthritis. It is also known as degenerative joint disease (DJD). While the exact cause is unknown, there are known to be several possible causes including: injury, age, congenital predisposition and obesity.

                        

                           

‘Osteoarthritis is idiopathic, slowly progressive disease of the synovial joints occurring in life and characterized pathologically by focal degeneration of articular cartilage, bone thickening, osteophytes formation and joint deformity.’

The word arthritis means inflammation (swelling) of a joint. Osteoarthritis, also known as “wear and tear” arthritis is the most common type of arthritis. It is estimated that osteoarthritis affects nearly 55% of Indians over the age of 65.Danielsson in 1964 first reported a 40%incidence of osteoarthritis above the age of 50. ‘Osteo’ means ‘bone’ and ‘arthritis’ means ‘joint damage’. The joint affected are spine, hip ,hands, knee. Osteoarthritis places a tremendous physical and economic burden on the society because it is the most prevalent musculoskeletal disorder.  The knee is one of the most common joints affected by this disease.

A joint is where two bones meet. Most of our joints are designed to allow the bones to move in certain directions. The knee is the largest joint in the body, and also one of the most complicated because it has many important jobs to do. It must be strong enough to take our weight and must lock into position so we can stand upright. But it has to act as a hinge, too, so we can walk. It must also withstand extreme stresses, twists and turns, such as when we run or play sports. Osteoarthritis of the knee affects different people in different ways. Some people have a problem with only one knee, others with both knees As a result, it is not very helpful to compare the experience of one person with another, and we cannot predict the eventual outcome for any one individual with osteoarthritis.

Most types of treatment for osteoarthritis of the knee work best when started early, before there is a lot of “wear and tear” in the knee. Doctors and physiotherapists who deal with people who have osteoarthritis can help outline a treatment protocol. Every osteoarthritic knee is different, and there should be a team approach to treatment. Physiotherapy is an important part of the treatment program for people with osteoarthritis of the knee. A knowledgeable physiotherapist will be able to provide more information about each of these types of treatments. While it may not be necessary to attend regular physiotherapy sessions, it may be helpful to consult a physiotherapist for an appropriate strengthening program that can be done at home. Additionally, a physiotherapist can help educate about proper exercise techniques and appropriate activities of the knee joint.

 

 

 

ARTHROLOGY

It is a hinge type of a synovial joint. It is not a typical hinge variety because the transverse axis around which the movement takes place is not static but moves forward during extension and backwards during flexion. In additional to these movements there are some rotational movements also possible in the joint.

EPIDEMEOLOGY

There is a steady rise in the overall prevalence from the age of 30 such that by 65,80% of people show radiographic evidence of OA though only 25-30% have associated symptoms. Knee OA is prevalent in all racial groups but hip, hand and generalized OA are only prevalent in Caucasians. OA is more prevalent and more commonly associated with symptoms in women ,except at the hip where both genders are equally affected. Recognised occupational risks for OA include farming (OA hip),mining(OA knee) and professional football(OA knee).

 

2. KNEE ANATOMY

                                        

The knee joint is the largest joint in the body and the most complex .

BONES: The knee joint is made up of four bones, which are connected by muscles, ligaments, and tendons. The femur is the large bone in the thigh. The tibia is the large shin bone. The fibula is the smaller shin bone, located next to the tibia. The patella, otherwise known as the knee cap, is the small bone in the front of the knee. It slides up and down in a groove in the femur (the femoral groove) as the knee bends and straightens. The knee joint is formed by the condyles of the tibia and the condyles of the femur and the patella.

SYNOVIUM:

The joint is surrounded by a membrane which produces small amount of thick fluid called synovial fluid which helps to nourish the cartilage and keeps it slippery.

CAPSULE:

The fibrous capsule is very thin, fibrous sac covering the joint It is deficient anteriorly where it is replaced by the quadriceps femoris patella and ligamentum patella. Attachments: proximally: to the upper margins of the medial and lateral femoral condyles.Distally: to the margins of the tibial condyles. Medially: to the medial margins of the medial condylar surfaces. Laterally: to the lateral margins of the alcondyles.    

                                                  

                                       

ARTICULAR CARTILAGE:

There is another type of cartilage in the knee called articular cartilage. This cartilage is a smooth shiny material that covers the bones in the knee joint. There is articular cartilage anywhere that two bony surfaces come into contact with each other. In the knee, articular cartilage covers the ends of the femur, the femoral groove, the top of the tibia and the underside of the patella. Articular cartilage allows the knee bones to move easily as the knee bends and straightens.

LIGAMENTS:

Ligaments are like strong ropes that help connect bones and provide stability to joints. In the knee, there are four main ligaments.

On the inner (medial) aspect of the knee is the medial collateral ligament (MCL)Attachements: to the above to medial condyles of the femur and below to the medial margins and adjoining medal surfaces of tibia.

The outer (lateral) aspect of the knee is the lateral collateral ligament (LCL).Attachements: It is a rounded cord like structure.  attached above to the lateral condyle of the femur and below to the head of the fibula.

Transverse Ligament: it connects the anterior end of the medial and lateral semilunar cartilage.

Coronary Ligament: it consists of short ligamentous fibres which serve to connect the peripheral convex margins of the semilunar cartilages with the peripheral margins of the tibia.

Arcuate ligament: it is attached to the posterior aspect of the femur above the lateral femoral condyle and fibres are directed downwards and medially to merge with the capsule.

 

The other two main ligaments are found in the center of the knee. These paired ligaments are called the anterior cruciate ligament (ACL) and the posterior cruciate ligament (PCL). They are very strong bands which maintain the stability of the knee joint. Anterior cruciate ligament:it is attached above to the anterior part of the intercondylar area of tibia between the anterior ends of the semilunar cartilages and above to the posterior part of the medial surface of the lateral femoral condyles. Posterior Cruciate ligament:it is attached below  to the posterior part of the intercondylar area of tibia and above to the anterior part of the lateral surface of the medial condyles.

 

 

MENISCI:

 Two semilunar cartilages called menisci sit between the femur and the tibia. They are fibrocartilagenous discs which are shaped in the form of cresents. These structures act as “cushions” or “shock absorbers”. They also help provide stability to the knee. There is a medial meniscus and a lateral meniscus. When either meniscus is damaged it is often referred to as a “torn cartilage”.

Medial menisci: it is Cshaped and has 2 ends.the anterior horn is attached to the intercondylar area of the tibia and the posterior horn is attached to the posterior part of the intercondylar part of the tibia between the attachment of the lateral semilunar cartilage and posterior cruciate ligament.

Lateral menisci:it has 2 horns and the anterior horn is attached to the intercondylar area of tibia between the ACL and interondylar tubercle and the posterior horn is attached to the intercondylar area of the tibia. The strong quadriceps muscles on the front of the thigh attach to the top of the patella via the quadriceps tendon. This tendon covers the patella and continues down to form the “rope-like” patellar tendon. The patellar tendon in turn, attaches to the front of the tibia. The hamstring muscles on the back of the thigh attach to the tibia at the back of the knee. The quadriceps muscles are the main muscles that straighten the knee. The hamstring muscles are the main muscles that bend the knee.             

TENDONS: They connect the muscles to the bones. The strong quadriceps muscles on the front of the thigh attach to the top of the patella via the quadriceps tendon. Theis tendon covers the patella and continues down to form the rope like patellar tendon. this tendon in turn attaches to the tibia.

 

BURSA:

Finally, a bursa is a small fluid filled sac that decreases the friction between two tissues. Bursae also protect bony structures. There are many different bursae around the knee but the one that is most commonly injured is the bursa in front of the patella, the prepatellar bursa. Normally, a bursa has very little fluid in it but if it becomes irritated it can fill with fluid and become very large.

Anteriorly: suprapatellar bursa, prepatellar bursa, superficial  infrapatellar bursa ,deep infra patellar bursa.

 Laterally:  bursa between the lateral cruciate ligament and the popliteal tendon, popliteal bursa between the popliteal and the lateral condyle of the femur.

Medially: tibial intercondylar bursa, between the MCL and semimembraneous tendon

 Posterior: between lateral head of the gastroneimius and capsule., semimembraneous.                             

 

MUSCLESAROUND THE  KNEE JOINT

FLEXORS:

Seven muscles flex the knee.  The knee flexors are the semi membranosus, semi-tendinosus, biceps femoris, sartorius, gracilis, popliteus, and gastrocnemius muscles.

All of the knee flexors except for the short head of the biceps femoris and the popliteus are two – joint muscles.  As two joint muscles their ability to produce effective force can be influenced by the relative positions of the two joints over which they pass.  Four of the flexors (the popliteus, gracilis, semi-membranosus and semi tendinosus muscles) are considered to medially rotate the tibia on the fixed femur where as the biceps femoris is considered to be a lateral rotator of the tibia.

The semi-tendinosus, semi membranous and the bicep femoris muscles are known collectively as the hamstrings.  These muscles all orginate on the ischial tuberosity of the pelvis.  The semi membranosus, semi-tendinosus insert on the posteromedial and antero medial aspects of the tibia respectively.  The semi membranosus muscle has fibres that attach to the medial meniscus.  This attachment assist in knee flexion by facilitating  posterior motion of the medial meniscus during active knee flexion.

 

The Biceps femoris muscle has two heads, both of which insert on the lateral condyle of the tibia and the head of the fibula.  The biceps femoris tendon may be attached to iliotibial band and retinacular fibres of the lateral joint capsule.  The biceps femoris has a stabilizing role at the posterolateral aspect of the joint.  The short head of the biceps femoris does not cross the hip joint and therefore has a unique action at the knee joint.

The Gastrocnemius muscle arises from the posterior aspects of the medial and lateral condyles of the femur by two heads. It inserts into the calcaneus.  It is the only muscle at the knee that crosses the ankle and the knee.  The gastrocnemius becomes actively insufficient quite easily.  Rather than working to produce knee flexion the gastrocnemius appears to be effective in presenting knee joint hyper extension, it appears to be less a mobility muscle at the knee joint than a dynamic stabilizer.

 

The Sartorius muscle arises anteriorly from the anterior, superior, spine of the ilium and crosses the femur to insert into the anteromedial surface of the tibial shaft posterior to the tibial tuberosity although a flexor and medial rotator of the tibia, activity in the sartorius is more common with hip motion than the knee motion.

The Gracilis muscle arises from the inferior half of the symphysis pubis arch and inserts on the medial tibia by way of a common tendon with the sartorius and the semitendinosus muscle.  It is not only a hip joint  flexor and adductor but it can also flex the knee joint and produce slight medial rotation of the tibia.

 

 

The Popliteus is one joint knee flexor besides the short head of the biceps femoris is the relatively small popliteus muscle.  This muscle originates on the posterior aspect of the lateral femoral condyle and attaches on the medial aspect of the tibia. The popliteus muscle is a medial rotator of the tibia on the femur in an open kinematics chain, this muscle plays a very important role in initiating un-locking of the knee because it reverses the direction of the automatic rotation.

 

 

EXTENSORS:

 

The four extensors of the knee are collectively known as the quadriceps femoris muscle. 

 

1. Together the muscles of the quadriceps femoris extend the knee.

 

 

 

 

THE PATELLO FEMORAL JOINT

The ability of the patella to perform its function without restricting knee motion depends on its mobility.  The patella has the ability to slide on the femoral condyles while remaining seated between them.  In full knee extension, the patella sits on the anterior surface on the distal femur.  With knee flexion, the patella slides distally on the femoral condoles, seating itself between the femoral condyles .  In full flexion the patella sinks into the inter condylar notch.  The sagittal plane motion of the patella is called patellar flexion.  Knee extension reverses the sliding of the patella and brings it back to the patella surface of the femur.  This motion of the patella is refered to as patellar extension.

 

PATELLO FEMORAL ARTICULAR SURFACES

The triangularly shaped patella is distinguished by being the largest sesamoid bone in the body.  The articular surface of the patella is smaller than the femoral trochlear surface.  The posterior surface of the patella is covered by articular cartilage and divided by vertical ridge.  The patellar articulating surface of the femur is the inter condylar groove or femoral sulcus on the anterior aspects of the distal femur.

MUSCLE INVOLVEMENT IN KNEE FLEXION

 

1) Semi tendinosus.

 

Root value         :   L4, L5,S1,S2

Action         :   Flexes and medially rotates the knee joint.

 

 

 

2) Semi membranous

 

Root value :   L4, L5,S1,S2

Action       :   Flexes and medially rotates the knee joint.

 

 

3) Biceps femoris

 

Root value  :  Long head – L5,S1, S2, S3

Short head – L5,S1,S2

Action  :    The long and short head of the bicep femoris flex and laterally rotate the knee joint.

 

4) Popliteus

 

Root value   : L5, S1

Action  : Flexion and internal rotation of the knee joint.

 

 

MUSCLE INVOLVEMENT IN KNEE EXTENSION

1) Rectus Femoris

 

Root Value : L2, L3, L4.

Action     :   The quadriceps extends the knee joints.

 

 

2) Vastus Lateralis

 

Root Value  : L2, L4

 

3) Vastus medialis & Vastus intermedius

 

Root Value   :  L2,L4

Action     :   It draws articular capsule proximally.

 

3. BIOMECHANICS

 

KINEMATICS:  Is the area of bio – mechanics that includes description of motion without regard for the forces reducing the motion. 

KINETICS: 

It is the area of bio – mechanics concerned with the forced producing motion or maintaining equilibrium.

1. Consequently the femoral and tibial longitudinal axis normally forms and angle medially at the knee joint of 185⁰  – 190⁰ hence creating a physiological valgus angle at the knee. Although this might appear weight the lateral condyle more than the medial but this is not so.

 The mechanical axis of the lower extremity is the weight – bearing line from the center of the head of the femur to the center of the superior surface of the head of the talus.  This line normally passes through the center of the knee joint between the intercondylar tubercles.  Because the weight bearing lines follows the mechanical rather than anatomical axis, the weight bearing stresses on the knee joint in bilateral static stance are equally distributed between the medial  and lateral condoles.  If the medial tibiofemoral angle is greater than 195⁰ and abnormal condition called genu valgum ( knock knees ) exists.  This condition will increase the compressive forces on the lateral condyle while increasing the tensile stresses on the medial structures.  If the medial tibiofemoral angle is 180⁰ or less then resulting abnormality is called genuvarum ( bow leg ).  In this condition the compressor stresses on  the medial tibial condyle are increased whereas the tensile stresses are increased laterally.  In both cases constant overloading on the lateral or medial articular cartilage may result in damage to the cartilage.

KINEMATICS

SURFACE JOINT MOTION

The motion between the articular surfaces of a joint occurs between the tibial and femoral condyles and between the femoral condyles and the patella.

Tibiofemoral joint surface motion takes place in all three planes, simultaneously minimal in the transverse planes.

Patellofemoral joint takes place in two planes, simultaneously the frontal and transverse but is greater in the frontal plane.

TIBIO FEMORAL JOINT

In a normal knee this line is tangential to the surface of the tibia for each interval of motion from full extension to full flexion demonstrating that the femur is gliding  on the tibial condyles.  The surface motion of the joint is from 90 of flexion  to full extension in normal knees.

PATELLOFEMORAL JOINT

The surface motion of the patellofemoral joint in the frontal plane may also be described by means of instant center technique.  This joint also has a gliding motion.  From full extension to full flexion of the patella glides approximately 7 cm on the femoral condyles.  Both, the medial and lateral facets of the femur articulate with the patella from full extension to 90 of flexion.  Beyond 90 of flexion the patella rotates externally and only the medial femoral facets articulates with the patella, at full flexion the patella sinks into the inter condylar groove.

KINETICS

Static and dynamic analysis of the forces and moments acting on a joint.  Static is the study of the forces and moments acting on a body in equilibrium (body at rest or moving at a constant speed ).  Dynamic is the study of the moments and forces acting on a body in motion.  Kinetic analysis allows one to determine the magnitude of the moments and forces on a joint produced by body weight, muscle action, soft tissue resistance and externally applied load in any situation that produces excessive high moments and forces.

STATICS OF THE TIBIOFEMORAL JOINTS

The three main coplanar forces acting on a free body and to the main moments acting about the joint under consideration to the minimum magnitude of the forces and moments.  Those forces are designated as vectors, if four characteristics are known that are magnitude, direction, sense line of  applicability and point of application.  The tibio femoral joint of the ground reaction force equal to body weight.  The tensile force through the patella tendon exerted by the  quadraceps muscles and the joint reactions force on the tibial  plateau .

DYNAMICS OF THE TIBIOFEMORAL JOINTS

The main forces considered in dynamic analysis are those produced by body weights muscles, soft tissues and externally applied loads.  Friction forces which are negligible in a normal joint are not considered.  In dynamic analysis two factors in addition to those in static analysis must be taken in to account.  That is the acceleration and the mass moment of inertia of the body part.  The steps for calculating the minimum magnitudes of the forces acting on a joint at a particular instance in time during a dynamic activity are as follows :

 

 

1)         The anatomic structures involved in the production of forces are identified.

2)         The angular acceleration of the moving body parts is determined.

3)         The mass moment of inertia of the moving body part is determined

4)         The torque (moment) acting about the joint is calculated.

5)         The magnitude of the main muscle forced accelerating the body part is calculated

6)         The magnitude of the joint reaction force at a particular instance of time is calculated by static analysis.

 

STATIC AND DYNAMICS OF PATELLO FEMORAL JOINTS

The magnitude of the muscle forces acting on a joint directly affects the magnitude of the joint reaction force.  In general the greater the muscle force is, the greater is the joint reaction force. When the knee bends 90 this force is reached 2.5 – 3 times body weight with the knee flexed to the 90, during stair climbing and desent at a point when knee flexion reaches a maximum of 60 the peak value equals 3.3 of the body weight.

ARTHROKINEMATICS

FLEXION/EXTENSION

The large articular surfaces of the femur and the relatively small tibial condyle create problem as the femur begins to flex on the tibia . If the femoral condyles were permitted to roll posteriorly on the tibial condyles before much flexion had occurred. This would result in limitation of flexion or the femur would roll off the tibia.For the femoral condyles to continue to  roll with the increased flexion of the femur the condyles must simultaneously glide anteriorly on the tibial condyle to prevent them from rolling posteriorly off the tibial condyle.The first part of flexion of the femur from full extension(0-25) is primarily rolling of the femoral condyles on the tibia, bringing the contact of the femoral condyle posteriorly on the tibial condyle.as flexion continues the rolling is accompanied by a simultaneous glide just sufficient to create a nearly pure spin of the femur. Extension of the knee from flexion occurs initially as a rolling of the femoral condyles on the tibial condyles ,displacing the femoral condyles anteriorly back to the neutral position after the initial forward rolling, the femoral condyles glide posteriorly just enough to continue extension of the femur as an almost pure spin. The motion of the menisci with flexion and extension are an important components of the motions. The menisci reduce friction and absorb forces of the large femoral condyles on the small tibial condyle,The menisci must remain beneath the femoral condyles to continue their function. failure of the menisci result in limitation of joint motion.

 

 

LOCKING AND UNLOCKING

The incongurence of the femoral condyles result in a rolling and gliding of the condylar surfaces. The medial rotation of the femur that accompanies the final stages of knee extension is not voluntarily or produced by muscular forces it is reffered to as automatic or terminal rotation of the knee joint. It is also known as the locking mechanism or screw home mechanism of the knee. To initiate flexion the knee must first be unlocked that is medially rotated femur cannot flex in the saggital plane but must laterally rotate before flexion can proceed. A flexion force will automatically result in lateral rotation of the femur because the longer medial side will move before the shorter lateral side of the joint.

Q- angle: 

The pull of the quadriceps and the patellar ligament is commonly assessed by the Q angle of the knee. The Q angle is the angle formed between the line connecting the anterior superior iliac spine to the midpoint of the patella and a line connecting the tibial tubercle and the midpoint of the patella. An angle of 10-15 measured with the knee either in full extension or slightly flexed is normal .The increased angle among women has attributed to the fact that they have a wider pelvis , increased femoral anteversion and knee valgus. The Q angle is  measured with the knee in extension because excessive lateral forces may be a problem. A Q angle of 20⁰ or more is considered to be abnormal.

 

 4. PATHOLOGY:

A variety of mechanical , metabolic, genetic or constitutional insults may damage a synovial joint and trigger the need for repair. All the joints tissues (cartilage, bone, synovium , ligament, muscle)depend on each other for health and function. The OA process is metabolically active, involving new tissues production and remodeling of joint shape

ARTICULAR CARTILAGE

It is a firm rubbery protein material covering the end of a bone. It acts as a cushion or shock absorber between the bones. When articular cartilage breaks down, this cushion is lost, and the bones will grind together. This causes the development of symptoms such as pain, swelling, bone spur formation and decreased motion. Osteoarthritis commonly affects weight bearing joints such as the knee, but it may affect any joint. Cartilage changes in OA are highly characteristics.

There is enzymatic degradation of the major structural components aggrecan and collagen principally by aggrecanase collagenase and stromelysin. The chondrocytes increase their production of matrix components and divide to produce clones or nests of metabolically active chondrocytes. Although the turnover of aggrecan components is increased  the concentration of aggrecan eventually falls. The decrease in size of the hydrophilic aggrecan molecules increases the water concentration and swelling pressure in cartilage, further disrupting the retaining scaffolding of type 2 collagen and making the cartilage vulnerable to load bearing injury. There is eventual fissuring of the cartilage surface( fibrillation), development of deep vertical clefts, localized chondrocyte death and decrease in cartilage thickness.

Cartilage loss is focal rather than widespread and usually restricted to the maximum load bearing part of the joint. The changes in OA cartilage encourage deposition of calcium pyrophosphate and apatite crystals. Damage to the ‘articular’ or hyaline cartilage can be as a result of sporting trauma, but is more commonly due to repetitive stresses over a long period of time. This is usually associated with habitual overuse of a joint for occupational or sporting purposes. In most individuals, the signs and symptoms do not appear until middle age, but the disease process starts much earlier. The hyaline cartilage is normally ultra smooth to allow friction free movement, but early damage to the superficial layers produces a frayed appearance.

The joint cartilage layer in osteoarthritis becomes thinner and eventually disappears entirely so that only raw  bone ends come in contact with each other. The grating of raw bone ends in direct contact causes pain and stiffness in the knee joint. The contours of the bone ends enlarge and bone spurs forms at the periphery of the knee joint. Joint capsule enlarges and becomes thicker. The joint cartilage damage may be restricted only to one knee joint compartment, as in this picture, or the whole knee joint may be affected by osteoarthritic changes. The complete loss of articular cartilage exposes the bone to stresses. It then undergoes intense osteogenic response. This phenomenon spreads towards the joint margins where unimpeded by compressive forces and perhaps encouraged by traction of capsular attachements it produces osseous outgrowth that extend into the capsule.

BONE:

The bone immediately below the compromised cartilage increases its trabecular thickness. In some cases it reflects healed trabecular microfractures. Holes often develop possibly the result of small areas of osteonecrosis caused by the increased pressure in bones as the cartilage fails in its load transmitting function. At the margins of the joint there is fibrocartilage that then undergoes endochondral ossification to form osteophytes. Despite central and marginal new bone formation , with severe cartilage loss there may be attrition of bone as the two unprotected bone ends wear on each other. Such wear may ablate the trabeculae and lead to a smooth ,shiny, surface (eburnation) often with deep linear grooves. Bone remodeling and cartilage thinning slowly alters the shape or the OA joint increasing its surface. The bone surfaces become hard and polished as there is loss of protection from cartilage. In time, with more stress, this frayed surface evolves into cracks or ‘fissures’ in the articular cartilage, which can leave the underlying bone exposed. With continued wear the underlying bone takes on a polished appearance and eventually micro-fractures appear. The body’s natural reaction is to attempt to heal the micro-fractures by laying down more bone.

SYNOVIUM

The joint cartilage layer in osteoarthritis becomes thinner and eventually disappears entirely so that only raw bone ends come in contact with each other. The grating of raw bone ends in direct contact causes pain and stiffness in the knee joint. The contours of the bone ends enlarge and bone spurs forms at the periphery of the knee joint. Joint capsule enlarges and becomes thicker. The quantity of joint fluid increases (effusion). Distended joint capsule makes the movements in the knee joint painful. Small fragments of damaged joint cartilage float in the joint fluid and cause secondary inflammation of the synovium (the inner lining of the joint space) with swelling and pain in the knee joint. The joint cartilage damage may be restricted only to one knee joint compartment, as in this picture, or the whole knee joint may be affected by osteoarthritic changes. In knee joints with changes restricted to only one knee joint compartment, it is most often the inner (medial) compartment that is damaged by OA, whereas other compartments may have still healthy cartilage. The presence of knee effusion, increased warmth CPPD crystals deposition and obesity are suggested risk factors. The capsule undergoes fibrous degeneration and low grade chronic inflammatory changes. The outer (lateral) compartment has healthy joint cartilage (light blue) and meniscus (dark blue).

 

 

 

It is at the knee that the predisposition to calcium pyrophosphate dehydrate (CPPD) crystals deposition by OA is most evident. Such coexistent crystals deposition may result in a more overt inflammatory component (stiffness, effusions)and supergradedacute of synovitis.

 

 

CAPSULE

The capsule undergoes fibrous degeneration and low grade chronic inflammatory changes. The presence of knee effusion ,increased warmth CPPD crystals deposition deposition and obesity are suggested risk factors. The capsule undergoes fibrous degeneration and low grade chronic inflammatory changes. The bone surfaces become hard and polished as there is loss of protection from cartilage.  Articular cartilage: the complete loss of articular cartilage exposes the bone to stresses. It then undergoes intense osteogenic response. This phenomenon spreads towards the joint margins where unimpeded by compressive forces and perhaps encouraged by traction of capsular attachements it produces osseous outgrowth that extend into the capsule

Damage to the ‘articular’ or hyaline cartilage can be as a result of sporting trauma, but is more commonly due to repetitive stresses over a long period of time. This is usually associated with habitual overuse of a joint for occupational or sporting purposes. In most individuals, the signs and symptoms do not appear until middle age, but the disease process starts much earlier. The hyaline cartilage is normally ultra smooth to allow friction free movement, but early damage to the superficial layers produces a frayed appearance. In time, with more stress, this frayed surface evolves into cracks or ‘fissures’ in the articular cartilage, which can leave the underlying bone exposed.

With continued wear the underlying bone takes on a polished appearance and eventually micro-fractures appear. The body’s natural reaction is to attempt to heal the micro-fractures by laying down more bone. This leads to a thickening of the underlying layer of bone, which can be seen on x-ray. Another common feature on x-ray is the formation of bone cysts in the layer of bone beneath the joint surface, together with a flattening of the articular surfaces as the body attempts to spread the surface area over which forces are transmitted.

As well as changes in the articular cartilage and underlying bone, osteoarthritis produces soft tissue changes. As the joint becomes deformed by bony changes the ligaments which support the joint become stretched. There is also thickening and distension of the capsule that surrounds the joint and both of these features get progressively worse.

LIGAMENTS

The ligaments undergo laxity

 

MUSCLES

 

They undergo atrophy and wasting.

  ETIOLOGY

 

THERE ARE TWO FORMS  OF OSTEOARTHRITIS

THE IDIOPATHIC OA – as the name idiopathic says, no one knows for sure what causes the destruction of the joint cartilage

THE SECONDARY OA  - the damage to the cartilage in the knee joint has a known cause such as previous fracture through the joint surfaces or damage to the meniscus or ligaments of the knee joint.

 

ETIOLOGY:

No one knows for sure the exact cause of osteoarthritis. There are two types of osteoarthritis : primary and secondary

 

1)Age:

The process  starts in the second decade of life but the degenerative diseases     are not apparent until middle age by 55-65yrs.

2)Sex:

It is twice as common in women as in men over the age of 50.In men the order of the joint involved is hip, spine, ankle, shoulders, fingers. In women the order is knee, fingers, spine, hip, ankle, shoulders.

 

3)Hereditary:

 

Herberden’s nodes may be inherited  as a single autosomal gene, sex influnced to be dominant in females Genetic factors are very important. Genes may affect collagen, one of the main building blocks of cartilage, or the way the bone reacts and repairs itself, or even the inflammatory process.

 

 

4)Obesity:

            Due to lack of exercise it is very common in obese people. Due to defective lubricating mechanism and uneven nutrition of the articular cartilge.

 

5)Poor Posture

 

 

The Secondary causes are

 

1)Trauma:it is more important risk factor in men and may result in unilateral OA. Most knee OA particularly in women is bilateral and symmertrical.

  

2)  Dislocations

3)  Infections like TB of knee joint

 

 

6.CLINICAL FEATURES

OA knee targets the patella-femoral and the medial tibiofemoral compartments of the knee. The pain is localized to the anterior aspect or medial aspect of the knee and upper tibia. Patello-femoral pain is usually worse going up and down stairs or inclines. Posterior knee pain suggests a complicating popliteal cyst. Commonly functional difficulties are prolonged walking, rising, from the chair, getting in or out the car or bending to put on shoes and socks. As the disease progresses  patient finds it difficult  to walk, to do her or his daily activities of daily living, climb stairs, unable to kneel or squatt.

 

1)pain:

 

 Due to inflammation in response to joint irritation caused by interference by loose bodies, cartilagenous debris engulfed by synovium. This appears with exercise but can be ‘walked off’.  Pain becomes severe as varus deformity increases.The types of pain are aching: fleeting and constant, refered :passing down the limb,sharp stabbing: loose bodies, throbbing: mainly at night. Pain with changes in weather are possible. Pain may be related to OA directly through increased pressure in subchondral bone, trabecular fractures , capsular distension and low grade synovitis or bursitis.

Pain in OA may be attributed to incongruent articulations of joint surfaces, periosteal elevation secondary to bone proliferation at the joint margin, abnormal pressures on sub-chondral bone, trabecular micro fractures and distention of the joint capsules.  Many patients will also experience a secondary synovitis especially when the knee is involved.

2)crepitus:

 the joint is often felt to creak and grate and swell from time to time especially  after any activity involving flexed position. Eg: kneeling and squatting As the disease progresses  patient finds it difficult  to walk, to do her or his daily activities of daily living, climb stairs, unable to kneel or squatt.

 

3)there will be swelling and inflammation of synovium the patient feels stiffness, tightness and reddness around the joint. Stiffness loosens quickly with rest

 

 

4)tenderness is elicited all round the joint

 

5)muscle spasm is present.

 

 

6)A jerky asymmetric antalgic gait is present where there is less time weight bearing on the painful side.

 

7)Weakness and wasting of the quadriceps muscle

 

Joint line or periarticular tenderness.

 

 

9)Young onset OA:

 

Some patients present with typical symptoms and clinical signs of OA but are younger than the expected(<45years).In most of the cases they have OA at a single joint such as the knee and the explanation of previous trauma is apparent in their history.

 

 

10) Changes in the weather may make a difference in some people. All joints have nerve endings which are sensitive to pressure. The nerve endings may respond to the drop in atmospheric pressure which occurs before it rains.

 

CLINICAL SIGNS

1.  Restricted movements.

2.  Palpable sometimes audible, coarse crepitus.

3.  Boney swellings around the joint margins. Deformity usually without instability

4.  Joint line or periarticular tenderness.

5.  Muscle wasting or weakness.

6.  No or only mild synovitis (effusion, increased warmth)

OA does not have a bilateral, symmetrical presentation. A single joint or any combination of joints on one individual may be affected. OA is not a systemic disease and it is therefore not associated with systemic complaints such as generalized morning stiffness, fever or loss of appetite.  Individual with OA may experience some stiffness in particular joints upon awakening that is similar to the stiffness felt when mobilizing the same joints after inactivity during the day but this stiffness does not last long.

CREPITUS is a common clinical finding.

List of symptoms of Osteoarthritis

The list of symptoms mentioned in various sources for Osteoarthritis includes:

• Joint symptoms
  • Joint pain
  • Joint stiffness
  • Joint tenderness
  • Tenderness around joint
  • Joint pain worse with exercise
  • Stiffness
  • Difficulty moving
  • Slow movement
  • Morning stiffness
  • Joint stiffness improves with movement
  • No pain – not everyone gets pain in the joints.
  • Crunching feeling
  • Crunching sound
  • Joint grating sensations
  • Swollen joints
  • Joint locking
  • Joint symptoms worse in cold or damp weather
  • Joint symptoms worse after periods of inactivity
  • Other common symptoms:
    • Fever
    • Weight loss
    • Fatigue
    • Trouble breathing
    • Rash
    • Itch
    • Symptom area depends on the affected joint
    • Knee symptoms
      • Knee joint pain
      • Knee stiffness
      • Knee warmth
      • Knee instability

 

 

7. INVESTIGATIONS

The diagnosis and assessment of common OA are purely clinical.

1)X-RAY

A plain radiograph is the only useful but  non  essential  investigation. This may show one or more of the typical features of OA namely focal narrowing of joint space, marginal osteophyte, subchondral sclerosis, cysts, osteochondral loose bodies and deformity. Chondrocalcinosis may be an additional feature at the knee. The main use of a radiograph is to assess severity of structural changes, an issue if surgery is being considered. Standing AP radiograph are needed to assess tibio-femoral cartilage loss and a flexed skyline view is the best show the space between the bones narrowing as the cartilage thins, and changes in the bone such as spurs.

  

2) SYNOVIAL FLUID ASPIRATION

From the OA knees shows variable characteristics but is predominantly viscous with low turbidity. CPPD crystals may be identified up to 50%of knee OA fluids.

 

 

3) RADIO ISOTOPE BONE SCANS

 

It is performed for other reasons often show an incidental finding, discrete increased uptake in OA joints due to bone remodelling.

 

4) MAGNETIC RESONANCE IMAGING

Is also performed. MRI scan of the knee can be helpful. This shows the soft tissues (e.g. cartilage, tendons, muscles)  which cannot be seen on an x-ray. Sometimes, a MRI scan may be ordered to look at the knee more closely. A MRI scan is a special radiological test where magnetic waves are used to create pictures that look like slices of the knee. The MRI scan shows more than the bones of the knee. It can show the ligaments, articular cartilage, and menisci as well. The MRI scan is painless, and requires no needles or dye to be injected

 

         

                                     

        

8. MEDICAL MANAGEMENT

It consists of conservative means and surgical means.

CONSERVATIVE MEANS

Includes reduced activity, weight loss, supports and braces, and physiotherapy are important initial steps in managing patients with osteo-arthritis of the knee. If these approaches fail, more aggressive treatment is warranted.

1) REDUCED ACTIVITY

 Decreased physical activity can diminish symptoms or reduce the incidence of osteoarthritic flares in some patients. However, these advantages must be balanced against the negative aspects of decreased activity, including weight gain, diminished general fitness, and increased quadriceps weakness. It is preferable, and for many patients nonnegotiable, that they maintain or increase their level of physical activity. A pragmatic approach is to eliminate or decrease involvement in activities that are more likely to ignite an osteoarthritic flare, such as high-impact and pivoting sports. Safe exercise programs should be designed to be encouraged.Many people with osteoarthritis of the knee are overweight. Studies have shown that people who lose weight have fewer knee problems in the future improve function and conditioning levels.

2)WEIGHT LOSS

 

Weight loss can ameliorate the severity of symptoms, but it is less certain whether weight loss can slow the rate of disease progression. Unfortunately, weight loss can be difficult to achieve and maintain. Many patients have pain and functional disability that limit or preclude exercise. For these patients weight loss is exceptionally difficult to achieve. They may require intervention to improve their knee pain before they can exercise at adequate levels to achieve weight loss. Low-impact alternatives, such as cycling or water-based exercises, should than those who do not. Being overweight is also bad for your general health and increases the risk of heart disease, strokes and diabetes. So you should eat a balanced, healthy diet and keep your weight as close as possible to the ideal for your height and age. Swimming can be a very good way of exercising and keeping fit as it causes little pain. Water supports the body’s weight so that little force goes through the joints as you exercise. Also, warm water relaxes muscles and joints and is very soothing, allowing joints to move more freely. Prescribed exercises in a hydrotherapy pool can help get muscles and joints working better, without undue pain. Supervised swimming in natural spa waters is an ancient treatment.  It is the exercise that helps rather than any healing properties of the water itself!

                                                

                                            

 

4)SUPPORTS AND BRACES

 

Some patients experience subjective improvement with the use of an elastic support around the knee. This approach must be used cautiously, if at all, in patients with a history of deep venous thrombosis or lower limb edema. Although the efficacy of this approach is not well established, it may enhance stabilizing inputs from articular and periarticular sensory receptors onto the motoneurons of muscles acting across the knee. Bracing can be effective in patients who have relatively focused unicompartmental medial or lateral osteoarthritis. Thus, valgus bracing is often used for varus knees in which osteoarthritis primarily involves the medial compartment. Less frequently, varus bracing may be indicated for lateral-compartment osteoarthritis associated with a valgus deformity. Unfortunately, even when a brace is prescribed and found to be efficacious, many patients stop using it after a short time.

 

5)DRUGS

 

Painkillers (such as paracetamol) and anti-inflammatory and paprika (capsaicin) creams to rub into the knee can help pain and stiffness. Some people find them more helpful than others. Anti-inflammatory drugs (NSAIDs) help some people more than paracetamol but they can cause stomach ulcers. Some new types of NSAIDs called COX-2s have the same beneficial effects but with less risk of stomach problems. However, all NSAIDs may cause other side-effects such as rashes, headaches and wheeziness. Sometimes an injection of steroids may help, either into a tender spot around the knee or even into the joint itself. The effect can last for several months. Injections of hyaluronan (Synvisc, Hyalgan, Durolane) may also help by supplementing the joint’s natural synovial fluid, although not everyone benefits from this type of treatment.

     

·        NSAIDS ( NON-STEROIDAL ANTI- INFLMATORY DRUGS)

NSAIDS are a basic element in long term treatment having both analgesic and anti-inflamatory actions.  At lower doses the NSAID effect is analgesic, through the peripheral  inhibition of pro-inflamatory prostaglandin synthesis.  At higher doses, the effect is anti-inflamatory probably through both prostaglandin inhibition and alteration in macrophage and neutrophil function although NSAIDS provide symptomatic relief, they do not alter the underlined disease process.  The major serious and the most common side effects are gastro intestinal complaints ranging from nausea to gastro intestinal bleeding in ulcers.  Other possible side effects include dizziness headache doziness or tinnitus, kidney dysfunction and elevation of liver enzymes.

·        CAPSAICIN

In the past few years, topical capsaicin cream has been introduced as a symptomatic therapy for osteoarthritis. The neuropeptide substance P is present in A delta- and C-fiber sensory nerves. It has been implicated as a transmitter in articular pain pathways and is a putative contributor to joint inflammation. Application of capsaicin to A delta- and C-fibers depletes them of substance P, blocking subsequent intraneuronal axonal transport and synthesis of substance P.16 Thus, capsaicin can potentially ameliorate symptoms of osteoarthritis through 2 mechanisms: by attenuating transmission in articular pain pathways and by diminishing joint inflammation. Studies support the safety and efficacy of capsaicin for treating osteoarthritis of the knee.Unfortunately, people in warm climates may have difficulty using capsaicin. They may experience an extreme burning sensation when using capsaicin and exercising strenuously in a hot environment.

 

 

• ANTIDEPRESSANTS

                      The role, if any, of antidepressants such as amitriptyline in managing symptomatic osteoarthritis of the knee is unclear. Some patients with osteoarthritis of the knee exhibit chronic pain behavior.

·        CHONDROITIN AND GLUCOSAMINE

                          Chondroitin and glucosamine have become popular as putative symptomatic and disease-modifying therapies for osteoarthritis of the knee. It is hypothesized that these important constituent components of cartilage matrix can be ingested, enter plasma, cross the blood-synovial fluid barrier, and enter synovial fluid in sufficient concentrations to provide analgesia and promote cartilage healing. The mechanisms, if any, underlying these alleged therapeutic effects remain undiscovered. Some clinical trials data are starting to become available. Given their widespread use by patients, it is fortunate that these products appear to be safe, and they can be a useful adjunct, especially in patients who cannot tolerate NSAIDs

• VISCOSUPPLEMENTATION

                          Hyaluronan (HA) in both articular tissues and synovial fluid plays an important role in contributing to joint homeostasis and maintaining normal joint function. In osteoarthritis, the molecular weight and concentration of HA is diminished. This has led to the concept of visco supplementation, in which pathologic synovial fluid is aspirated and HA-based products are injected into the articular space in order to restore HA concentration and molecular weight to relatively normal levels. Therapeutic benefit is believed to occur by restoring the viselastic, antinociceptive, anti-inflammatory, and autoregulatory functions of HA in synovial fluid.

Clinical studies of HA-based products have demonstrated clinical benefit for about 70% of patients in a heterogenous osteoarthritis population. Duration of improvement ranges from months to years.