Welcome to the World of Linux

The for loop is used to process a list of items. The list can be supplied in the script. It could also be taken from the script arguments, from a file, or from the output of a command.

The syntax of the for construct is as follows:

for list
do
command_list
done

All the commands in the list are carried out for each item in the list.

The following script invokes the C compiler cc for all files with the extension of .c held in the current directory.


for filename in $(ls *.4gl)
do
cc $filename
done


A similar script could be written to compile several programs specified on the command line. The user can specify as many programs as he wants.


for filename in $*
do
cc $filename
done


$* expands to all the arguments given by the user. Each argument will count as a single item in the list.

This construct is very useful for working with commands that give more than one line of output. Note, however, that the loop will be carried out for each word of the output, rather than once for each line. Commands whose output consist of several columns separated by spaces will be treated as one list item for each column of each row.

To illustrate this, we will look at a script that takes the output from the ps command, and attempts to process it as a list. The output from ps looks something like this:

PID TTY TIME CMD
3218 pts/2 0:01 ksh
3207 pts/2 0:00 sh
3550 pts/2 0:00 ps

The script is as follows:


for process in $(ps)
do
echo $process
done


and gives the following output:

PID
TTY
TIME
CMD
3218
pts/2
0:01
ksh
3207
pts/2
0:00
sh
3579
pts/2
0:00
ps
3578
pts/2
0:00
ksh

If we wanted to carry out commands on each line rather than on each word, and ignore the column headings, we would have to use some ingenuity. The following script displays the PID and command from each line output from the ps command.


count1=0
count2=0
for process in $(ps)
do
count1=$(($count1 + 1))
if test $count1 -lt 5
then
continue
fi
count2=$(($count2 + 1))

if test $count2 -eq 1
then
pid=$process
fi
if test $count2 -eq 4
then
cmd=$process
echo $pid $cmd
count2=0
fi
done


The output would look like this:

3218 ksh
3207 sh
3609 ps
3608 ksh

The script uses two counters. The first, count1, is used to skip out the headings. The first four items in the list are headings, so if the counter is less than 5, it uses the continue command to skip all further commands for that item. The second counter is used to establish which column of the ps output is being processed. When it is column 1, it stores the PID in a variable. For column 4, it stores the command in a variable, echoes both variables to the user, and resets the count to zero.

free payday loanlow cost payday loancash advance payday loanapproval guaranteed loan paydaypayday loan applicationyahoo payday loanmoney tree payday loanloan money payday treepayday loan toronto,payday loan in toronto,loan payday torontoloan payday yahooinstant approval payday loan,approval instant loan paydaypayday loan paycheck advancecredit loan no payday,credit dollar loan no payday ten,no credit payday loanone hour payday loanno credit check payday loancalgary loan payday,calgary payday loanfast online payday loanfast loan online paydayadvance loan payday quickquick payday advance loanmilitary payday loanno fax cash advance,advance cash fax no,account advance cash fax no savingadvance cash company scams,cash advance company,advance advance cash company heirpayday payday loan cash advance loan,advance cash loan payday today,advance cash loan paydayadvance cash loan onlineace cash advancecash fast loan paydayfast faxless cash advance,fast cash advance,advance cash fast faxing nonational cash advancepayday us fast cash loan,fast cash payday loanadvance america cash company,advance america cash loan,cash advance americacash loan payday untiladvance cash loan loan paydayadvance america cashcash advance payday loan softwarebusiness cash advance loan,business cash advance,how to start a cash advance businessadvance cash loan payday softwareadvance cash cashing check greenville,greenville,advance cash cashing check durham,durham,check cash advancefirst american cash advancecash until payday loanadvance cash overnightcash advance servicecash loan payday tilfirst time cash advance,first choice cash advance,first cash advancescams on cash advance company,advance cash companyadvance cash now quick,quick cash advance,quick no fax cash advancecash in advancecash advance loan illinois,5 advance cash illinois loanadvance cash check credit noace advance america cash,advance america cash first,advance america cash advance

In common with most programming languages, Unix shell scripting caters for instructions to be carried out repetitively.

The while loop will repeat a list of commands as long as a given condition is true. All of the conditional statements used in the if…fi construct can be used here, including string and numeric comparisons, file checking, null checking and the exit status of a command. The syntax is

while condition
do
list of commands
done

As an example, let’s write a script named table that will take a number as a command line argument, and print its ‘times table’. The sample below shows the output we would expect from the script:


$ table 5
1 times 5 is 5
2 times 5 is 10
3 times 5 is 15
4 times 5 is 20
5 times 5 is 25
6 times 5 is 30
7 times 5 is 35
8 times 5 is 40
9 times 5 is 45
10 times 5 is 50
11 times 5 is 55
12 times 5 is 60


To achieve this, we will need a variable that will act as a counter. This will begin at 1 and be incremented within the loop. The loop will keep repeating while the counter is less than 13. Within the loop, we will multiply the command line argument $1 by the counter, display the result of this sum, and increment the counter.

The table script is shown below.


counter=1
while [ $counter -lt 13 ]
do
calc=$(( $counter * $1 ))
echo $counter times $1 is $calc
counter=$(( $counter + 1 ))
done


There are two additional commands that can be used within loops. The break command breaks out of the loop, even though the terminating condition has not yet been met. The continue command goes back to the beginning of the loop without executing any subsequent commands. If the condition controlling the loop is still true, the loop will be repeated.

A loop can be coded to repeat itself infinitely, using the construct while true. This can be used for repeated checking of some aspect of the system. Generally, in this kind of loop, the user is given the option to quit the loop, although he can simply terminate it using the interrupt key.

The following script will display the number of processes which the current user has running, giving him an option to list the processes. He can also opt to quit the script. A delay of 10 seconds is introduced into the loop by using the sleep command that pauses for a given number of seconds.


while true
do
sleep 10
set $(ps -u $LOGNAME | wc)
echo You have $1 processes running
echo Enter l to list them, q to quit or c to continue
read u_reply
if [ $u_reply = q ]
then
break
fi
if [ $u_reply = c ]
then
continue
fi
ps -u $LOGNAME
done


The command set $(ps –u $LOGNAME | wc) lists processes belonging to the current login name and pipes the output to the wc command. This returns the number of words, lines and bytes contained in the answer. This output is used to set the script arguments $1 etc. The first of these arguments, the number of lines returned by the ps command, will be the number of processes this user has running. This is displayed to the user.

If he chooses to continue without listing the processes, the continue command returns to the beginning of the loop. If he chooses to quit, the break command breaks out of the loop. Otherwise, the script will carry out he second ps command that lists all the processes.

The set command can be used to store a new set of values in the script arguments ($1, $2 etc.). This can be useful for separating the output from a command into separate components that can be worked with individually. The set command requires a string of arguments separated by spaces.

Suppose we have a script that runs at the end of each day, carrying out various tasks. On Fridays only, we want to remove a file named weekhist and create an empty one in its place.

The date command will tell us the day of the week, but it also gives a lot of other information that is not relevant to this task. The output from the date command looks like this:

Mon Sep 3 18:38:21 GMT 2007

We are only interested in the first column, which tells us the day of the week.

Placing the output from this command into the script arguments allows us to separate the output so that we can access the bit we want.

The command:

set $(date)

will have the following effect:

$1 becomes Mon
$2 becomes Sep
$3 becomes 3
$4 becomes 18:38:21
$5 becomes GMT
$6 becomes 2007

We can then check the value of $1 to see if it is a Friday. The relevant section of the script would be:


set $(date)
if [ $1 = Fri ]
then
rm weekhist
touch weekhist
fi


We could use other portions of the output from date for other purposes. Suppose that rather than deleting the weekhist file, we want instead to move it to a file named weekhist suffixed by today’s date.

We could change the script as follows:


set $(date)
if [ $1 = Fri ]
then
mv weekhist weekhist_$2_$3_$6
touch weekhist
fi


If run on Friday September 4 2007, this would have the effect of renaming the old weekhist file to weekhist_Sep_4_2007.

This technique can be used with any command that gives a single line of output consisting of columns separated by spaces. In a later article, we will look at how to deal with the output from commands that give many lines of output.

Scripts should run with as little user intervention as possible. There are several reasons for this. Firstly, the whole point of having a script is to reduce the amount of typing needed to run a job. Secondly, scripts should be able to be run by users with little or no technical knowledge. Thirdly, they are designed to make sure that nothing is left out or mistyped, thus reducing operator errors.

Many Unix commands and programs require input from the user. In an automated script, you would often want to provide this information from the script itself. We have seen in an earlier article that the standard input and output can be diverted to a file instead of the terminal by using output redirection (>), input redirection (<) and error redirection (2>).

Input can be redirected to come from the script itself by using the symbols <<. The text to be taken as input must be enclosed in exclamation marks (!). The following example illustrates how this is done.

cat > textfile <<!
This is some sample text
!

Again, this is a feature which can be very useful in installation scripts, which may need to set up initial files, customized scripts for running jobs etc.

Let’s look at an installation script that needs to schedule a day end job to be run at 6 PM daily. The cron utility is used for job scheduling. cron can take details of the job from a file, or they can be entered from the terminal. In the example below, the information is supplied within the installation script.

cron <<!
00 18 * * * /usr/progs/dayend

In a previous article, we looked at making commands within the script execute only if a given condition is true, using the if….else…fi construct. However, this construct can often become very clumsy when there are several possible conditions to be accounted for. Take the menu script example below:


clear
echo User Menu for $LOGNAME
echo Accounts......................1
echo Star Office...................2
echo Wages .........................3
echo Exit .............................4
echo
echo Please enter your choice 1,2,3 or 4
read menu_choice
if [ $menu_choice = 1 ]
then
accounts
elif [ $menu_choice = 2]
then
star
elif [ $menu_choice = 3 ]
then
wages
else
exit
fi
$0


Here we have four different conditions, all depending on the single variable $menu_choice. If we wanted to add more choices to the menu, it would become even more complex.

There is a better way of doing this, provided that all the conditions depend on the value of a single variable. This is to use the case construct.


clear
echo User Menu for $LOGNAME
echo Accounts......................1
echo Star Office...................2
echo Wages .........................3
echo Exit .............................4
echo
echo Please enter your choice 1,2,3 or 4
read menu_choice
case $menu_choice
in
1) accounts;;
2) star;;
3) wages;;
4) exit;;
*) echo $menu_choice is not a valid option;;
esac
$0


This new script does exactly the same as the old one, but it is much neater, and much easier to expand if more menu choices are added.

As with the if…fi construct, the syntax of the case…esac construct is very strict. It must begin with the word case followed by the name of the variable whose contents are to be examined. The word in must be placed on a separate line. The expected values must then be listed, each on a separate line followed immediately by the close bracket symbol. After the bracket, there is a space followed by one or more commands to be executed if the variable has this value. A double semi-colon terminates the list of commands.

The values to be matched may contain wild cards. Thus the *) matches to any value of menu_choice. Any value of menu_choice (e.g. 5 or 6) not previously matched will be seen as a match here. This then causes any value that is not between 1 and 4 to display the error message.

Normally, when you run a command from a script, the output from the command goes to the standard output: usually the user’s terminal. However, you may want to do further work with the output from the command, or use it to make decisions within the script.

Lets take a simple example. Suppose that you want to display the message, “Your current working directory is “ followed by the working directory. This can be done by inserting the output from the pwd command into the echo command, using a technique known as command substitution. There are two ways of doing this. The first is to enclose the name of the command in back quotes thus: `pwd`. The second is to enclose it in brackets and prefix it with the $ sign as follows: $(pwd).

So the above example could be entered into a script as follows:

echo Your current working directory is $(pwd)

This facility can be very useful in installation scripts, which may need to install different resources depending on system information such as the type of processor or operating system.

The uname command displays information about the current operating environment. Various switches can be used with it:

-a  Display all information
-s  Display the name of the operating system
-p  Display the type of processor
-r  Display the release number of the operating system.

The following is part of a script that uses the name of the operating system to decide which files to install.


ins_opsys=$(uname –s)
if [ $ins_opsys = SunOS ]
then
tar xvf sunfiles.tar
elif [ $ins_opsys = UnixWare ]
then
tar xvf unixwarefiles.tar
elif [ $ins_opsys = AIX ]
then
tar xvf aixfiles.tar
else
echo Operating system $ins_opsys not yet supported
fi


Command substitution becomes even more useful when combined with techniques covered in the next few articles. We will learn how to split the output from the command so that we can use selected parts of it, or reformat it. We will also learn how to deal with multi-line output.

Certain characters have special meaning to the shell. These are known as metacharacters. They are listed below.

Pattern Matching:

This set of metacharacters is used in pattern matching, also known as wild cards.

*    Match zero or more characters
?    match exactly one character
[]   Match a list or range of characters eg [AB], [A-N]
[!]  Match anything except the list or range of characters e.g. [!AB]

Redirection:

>    Redirect away from the standard output file
>>   Redirect away from the standard output and append to the target file
2>   Redirect away from the standard error file
<    redirect away from the standard input file
<<   Start of a Here document
|    Pipe the output of one command to the input of a second command

Miscellaneous

&    Run the command in the background
()   Execute command(s) in sub-shell
$    Substitute the value of a variable or argument
#    ignore text to end of line
;    New line
:    Null command

Literal

\    take the literal of the following character – this is used to include an actual metacharacter in a string, rather than have the shell try to interpret it e.g.\$
‘’   Interpret all characters enclosed in single quotes as a literal
“”   Interpret all characters enclosed in double quotes as a literal, except the dollar sign.

Literals are important, since without them, the shell will first interpret any metacharacters before working with the string. For example, the following command would produce an error:

echo Enter your name>

since the > would be interpreted as a redirection symbol. This can be avoided by placing the text in quotes:

echo ‘Enter your name>’

The term RAID stands for Redundant Array of Independent Disks. RAID technology allows you to group together the storage capacities of two or more disks, and manage them as a single storage unit. Users are unaware that this array is not a single disk. A RAID array is known as a virtual device, since the operating system views it as a single device when storing data.

RAID can either be controlled at the software or the hardware level. If it is hardware controlled, a RAID controller will be supplied. This is a plug-in card that is attached to the motherboard, and the disks are plugged directly into this controller. It controls all RAID activities, so that the motherboard, and the operating system, are not aware that it is anything other than a single disk device. This is a good option for large RAID arrays, since it tends to give better performance.

When using this option, RAID devices are configured into the operating system in exactly the same way as single-unit devices. There are no special configuration considerations.

If RAID is configured at the software level, partitions are grouped together into a virtual device and appear to higher layers of the operating system as a single unit. This gives good performance under Linux, and is a less expensive option.

Depending on how it is used, RAID can have several advantages, including increased performance, better fault tolerance and data integrity, and more efficient storage. RAID levels are a way of defining the actual type of RAID configuration.

In a large installation, it is sometimes difficult to keep control of user activities. Some user programs can use excessive resources, to the detriment of other users. This is particularly true in universities and colleges, where student’s activities can leave few resources for programs run by staff. Also some services automatically grab as many resources as they find available at run time, leaving few resources for subsequent programs. Resources include disk space, memory, CPU time and I/O requests.

It can be useful, therefore, to set limits for each user or group of users. The /etc/security directory holds files that can be used to limit user activities. The most important of these files is /etc/security/limits.conf. It holds entries in the following format:

<domain> <type> <item> <value>
where
<domain>   is the name of the user or group (groups must be prefixed with @)
<type>     is either ‘hard’ or ‘soft’
<item>     is one of the items listed below
<value>    is the value required

Items include:
core - limits the core file size (KB)
data - maximum data size (KB)
fsize - maximum filesize (KB)
memlock - maximum locked-in-memory address space (KB)
nofile - number of open files
rss - max resident memory set size (KB)
stack - stack size (KB)
cpu - maximum CPU time
nproc - maximum number of processes
as - address space limit
maxlogins - maximum number of logins for this user
priority - the priority used for user processes
locks - maximum number of file locks

When setting these limits, you should check the documentation of any software packages that you are running, to ensure that they are able to run within the limits that you have set.

Another useful file in this directory is /etc/security/access.conf, which is used to control which users are allowed to log in from which sources. For example, it is possible to restrict a user’s access to one particular terminal, or to prevent users from logging in via the network.

This file holds entries in the format:

<permission>:<user_or_group>:<origin>
where
<permission>     can be + or -
<user_or_group> A user name, group name, or ALL, or ALL except user or group name
<origin>    A TTY number, host name or domain name

In another article, we briefly looked at the find command, which searches for specified files. We will now look at more uses of this very powerful command.

It can be used to search for files using various criteria, including name, owner and time last changed. Additionally, it can execute a command or script for each of the matching files. It could therefore be used for many purposes, for example, to remove files not accessed since a given date, to recompile all programs modified since the time of the last compile, or to backup all the files belonging to a specific user.

The syntax of the command is:

find
where is the pathname to the directory where the search is to begin and
and specify the search criteria and any actions to be taken, as detailed below:

-name Find files whose name matches the given pattern. The pattern should be in quotes.
-print Display the path of matching files
-user Find files belonging to a specific user
-exec command {} \; Execute Unix/Linux command for each matching file.
-atime (+t,-t,t) Find files accessed more that +t days ago, less than -t or precisely t days ago.
-ctime (+t,-t,t) Find files changed more that +t days ago, less than -t or precisely t days ago.

-perm Find files with given permissions.
-type Locate files of a given type:
c=character device b=blocked device d=directory
p=pipe s=socket f=normal file l=symbolic link
-size nx Find files with size larger than n where x can be b=bytes;
c=characters; k=kilobytes w=2 byte word:

Examples:

find . –name myfile
Search for myfile under the current directory

find /usr/sam/cprogs –ctime –1 –exec cc {} \;

Compile all programs that have changed less than one day ago.