Basics of programming in the shell command shell. Where are command line systems found? Bulk and semi-automatic execution

Command interpreter

To provide a command line interface, the OS often uses command interpreters, which can be independent programming languages with their own syntax and distinctive functionality.

To operating rooms Windows systems 9x included command interpreter command.com, Windows NT includes UNIX command shells, popular csh, ksh and others.

As a rule, when low-level tuning the OS, the user has the opportunity to change the default shell.

Functions

The command interpreter executes commands in its language, given on the command line or coming from standard input or a specified .

Calls to system or application utilities, as well as control structures, are interpreted as commands. In addition, the shell is responsible for expanding filename patterns and for redirecting and binding utility I/O.

Together with a set of utilities, the shell is an operating environment, a full-fledged programming language and a powerful tool for solving both system and some application problems, especially automating frequently executed command sequences.

Standard command interpreter

Implementations and Availability

Alternatives

Along with standard ones, open operating systems also use alternative shells tcsh, differing in the syntax of control structures and the behavior of variables.

Some alternative OSs come with interpreters own languages command files (such as OS command file language Microsoft Windows NT, OS/2 language, etc.)

Some people prefer to use new interpreted languages, such as Python, to automate frequently executed command sequences.

Graphic shell

Graphic shells for OS Windows

The latest versions of Windows OS use the Windows Explorer integrated environment as their shell. Windows Explorer is a visual management environment that includes the Desktop, Start Menu, Taskbar, and file management functions. Early versions Windows OS 3.xx as graphical shell include a program manager.

Many third party developers offer alternative environments that can be used instead of the Explorer shell enabled by default by Microsoft to the Windows system.

Aston shell
BB4Win
BBlean
Cairo (Under Development)
Chroma
Emerge Desktop
Geoshell
Packard Bell Navigator
Program Manager
Secure Desktop
SharpE
Talisman Desktop
WinStep
Microsoft Bob

Simultaneous access to the system

Each computer running UNIX allows multiple users to log in and access the system simultaneously. Even if all users have only one monitor and one system keyboard, this feature is useful: having multiple users logged in at the same time allows you to work in turns without having to complete all started tasks and then resume them each time. It is also possible to register in the system several times under the same login name. This way, you can access the same resources and organize parallel work on several tasks.

A characteristic way for modern versions of UNIX to organize parallel work of users is virtual consoles. Virtual consoles are several programs executed in parallel by the operating system that provide the user with the opportunity to log into the system in text mode and access the command line (see Figure 3.2, “Virtual and Graphical Consoles”).

In the Linux operating system, switching between virtual consoles in text mode is done using a keyboard shortcut Alt-F1,Alt-F2 and so on. In this case, each virtual console is designated by a special name: “tty1”, “tty2”, etc. Each combination is associated with a corresponding virtual console number. Each of these consoles is associated with its own terminal, which is characterized by a device file with a corresponding name (for example, /dev/tty1).

Modern versions of UNIX provide users with graphical user interfaces that also provide command line functionality. This feature is achieved using graphic terminals– special programs that emulate a text terminal in a graphical window.

It is also possible to run multiple graphics subsystems, then switching between them will be carried out similarly to switching between virtual consoles - a key combination Ctrl-Alt-F1.

Figure 3.2. Virtual and graphical consoles

Each terminal device has its own possibilities on input and output of information. Examples of such capabilities are: the number of display colors, the ability to move the cursor and change the screen size, a set of control characters, etc. Terminals are divided by types: a set of capabilities regulated in a special configuration file. Examples of types are: tty (teletypewriter) or xterm (graphics terminal). The terminal type is explicitly specified in the terminal device name (for example, /dev/tty1). All terminal type settings are located in the /etc/termcap directory.

Operating systems Linux family, like any other OS, assumes the presence of an interaction interface between the components of the computer system and end user, i.e., the presence of a software layer that provides input of commands and parameters to obtain the desired results. Such program level got the name "shell" or at English language - shell.

What is a shell?

Command shell ( shell) provides interaction between the user and the Linux operating system environment. She is specialized software product, which ensures the execution of commands and obtaining the results of their execution, or, to put it very simply, a shell is a program that is designed to ensure the execution of other programs at the request of the user. An example of a shell could be, for example, a command interpreter command.com operating system MS DOS, or shell bash operating rooms Unix systems/Linux.

All shells have similar functions and properties, in accordance with their main purpose - to execute user commands and display the results of their execution:

Command line interpretation.

Access to commands and the results of their execution.

Variable support special characters and reserved words.

Processing of files, standard input and output operations.

Implementation special language shell programming.

For operating systems of the Unix / Linux family, it is possible to use several different shells that differ in properties and methods of interaction with the system. The most common shells are

sh- shell Bourne, classic shell for Unix OS

bash shell Bourne Again(GNU Bourne-Again SHell). Perhaps the most common this moment, a shell in the Linux OS environment.

ksh- shell Korn, designed as a shell development Bourne with command line history and the ability to edit commands.

csh- shell C, using the syntax of a popular programming language C

tcsh- shell version C with interactive command line editing.

Several different shells can be installed on the system, and each user can use their own default shell. All this, of course, is done automatically during the download and user registration process.

During the loading process of operating systems of the Linux family, after loading the system kernel, the system switches to interactive mode - the mode of interaction between the user and the operating system. On Linux, the first process launched during boot is the init program. init, which reads the contents of the configuration file /etc/inittab, determines the list and characteristics of terminals available in the system, and calls the program interactive login getty, which prompts you to enter your username. After entering the username and password, the program getty calls the program login, which checks the validity account, navigates to the user's home directory and passes control to the session startup program, which is usually the user's shell program, the specific flavor of which is determined by the contents of the file /etc/passwd for this account. For example:

user1:x:508:511::/home/user1:/bin/sh
interbase:x:510:511::/home/interbase:/bin/csh
apb:x:511:513:apb:/home/apb:/bin/bash

As can be seen from the contents of the file /etc/passwd, for the user user1 the shell will be launched sh(Bourne shell), for user interbase- shell csh(C shell) and for the user apb- shell bash(Bourne Again). After the shell starts, a command prompt is displayed on the screen (usually in the form of a dollar sign $ if the work is done in the context of an account regular user, or pound # , if the shell is used under a root user account ( root).

When exiting the shell, the system kernel returns control to the program init, which restarts the login process and displays a username prompt on the terminal. Exiting the shell can be done in one of two ways:

Through the team exit performed by the user

When a shell process receives a signal kill, sent by the kernel, for example when the system is rebooted.

Command line interpretation.

User input in response to a shell prompt is usually called command line or team. A Linux command is a character string of the command name and arguments, separated by spaces. Arguments provide the command with additional parameters that determine its behavior. Most often used as arguments options And names files and directories. For example, command line

ls -l file01 file02

Contains the ls command, the -l option, and two filenames file01 file02.

When using several options, they can be combined. For example, the following command options are identical:

Ls -l -d
ls -ld

Commands that are part of the shell are called built-in. Such commands include, for example, cd, if, case, etc. Naturally, built-in commands may differ for various options shells. In addition to the built-in commands, it is possible to use software modules, which are individual executable files, or files scripts or scenarios- ordinary text files, containing sequentially executed shell command lines. Some scripts (scripts) can be executed by Linux processes, such as the task scheduler cron. The task scheduler is typically designed to automatically perform system administration tasks on a schedule. Tasks cron are commands or scripts and are executed automatically, without any human intervention, and can be executed in the context of different user accounts. In the case where the scheduler task involves executing a script, the problem arises of choosing a shell that should be launched as a child process cron to process commands from a script file - after all, the shell can be any, and the syntax of a script, as a rule, requires the use of the specific shell for which it is written. To eliminate this problem, in Linux operating systems it is customary to indicate in the first line of the script the type of shell required for its execution, in the form:

#!/bin/bash- for shell bash

#!/bin/sh- for shell sh

Sign # is a sign of a comment and the characters following it are not interpreted as a command. This technique allows you to explicitly specify which shell should be used to process subsequent file contents. If the script does not contain an entry that explicitly defines the required shell, then the settings from the account in the context of which the script is executed will be used. In this case, it is possible that a script written for the shell, for example, tch will be passed to the shell for execution bash, which will make it impossible to execute.

When executing commands or scripts, use environment variables(in English - environment, the values of which characterize software environment, in which commands are executed. Such variables may contain General settings system, graphical or command shell parameters, paths executable files and so on. Environment variable values are set at the system level (for all users) and at the specific user level. To set environment variables at the system level, the contents of the files are used:

/etc/profile- sets variables only for command shells. Can run any scripts in shells compatible with the Bourne shell.

/etc/bash.bashrc- sets variables only for interactive shells. It also runs bash scripts.

/etc/environment- used by the PAM-env module. Only pairs can be specified in this file name=value.

Each of these files has its own application, so you should carefully choose the one that suits your purposes. For example, if you need to add a custom directory ~/bin into a variable PATH for all users, place the following code in one of system files environment initialization (/etc/profile or /etc/bash.bashrc):

# If the user ID is greater than or equal to 1000 and there is a ~/bin directory and it

#has not been previously added to the PATH variable,

# export ~/bin to $PATH.

If [[ $UID -ge 1000 && -d $HOME/bin && -z $(echo $PATH | grep -o $HOME/bin)

Export PATH=$HOME/bin:$(PATH)

Typically, on Linux operating systems, a user ID less than 1000 or less than 500 is used for service accounts. IN in this example, the environment variable will be set for all local users systems with an ID of 1000 or more.

If you need to change the environment for a specific user, modify the contents of the user's environment:

- ~/.bash_profile, ~/.bash_login and so on. - shell initialization files from the user's home directory.

- ~/.profile- user profile initialization file. Used by many shells to define environment variables.

~/.pam_environment- a custom analogue of the /etc/environment file, which is used by the PAM-env module.

For example, to add the user directory ~/bin to the search path of executable files specified by the variable PATH, for example, to a file ~/.profile put the line:

export PATH="$(PATH):/home/user/bin"

To set environment variables for graphic applications, the contents of the user graphical environment configuration files are used ~/.xinitrc

More often, environment variable values are set for the current user session. For example, to add a custom directory ~/bin in the search path for executable files:

export PATH=~/bin:$PATH

New variable value PATH will only last until the current user session ends.

To view the value of a variable, you can use the command echo $variable, For example:

echo $PATH

Currently, the most common shell, as mentioned above, is bash. This is caused primarily by the fact that the shell bash is sh- a compatible command shell with added useful features from Korn shells ( ksh) and C shell ( csh). Shell bash can run most scripts written under a shell programming language without any modification sh and in maximum degree trying to get closer to the standard POSIX, which led to many improvements, both for programming and interactive use. In modern implementation bash There is a command line editing mode, unlimited command history size, task management tools, the ability to use aliases, an extensive list of built-in commands, command shell functions, etc. Generally, bash best suits the needs of the average user, which has made it the most used in the Linux environment.

On startup bash without command line parameters, the shell starts in interactive mode, displaying a command prompt on the screen. An interactive shell typically reads data from the user's terminal and writes data to the same terminal, standard device The input device is the keyboard, and the standard output device is the display. The user enters commands on the keyboard, and the result of their execution is displayed on the display.

Command shell in UNIX is a command line interface in Unix-like operating systems, that is, it executes commands that are given by the user or that are read from files. Such files with shell commands are called shell scripts (scripts, programs). These scripts are not compiled, but rather interpreted by the shell. This means that the shell reads the script from start to finish, line by line, looking for the commands specified there and executing them; In contrast to this approach, the compiler converts the entire program into a form suitable for machine execution - the file with this code can then be used in a shell script. Feature shell language - many operations that are in traditional languages programming are built-in, performed by calling external programs.

One of the purposes of the shell is to provide user environment, which can be configured using configuration files.

Types of shells

UNIX provides several different shells to choose from:

sh, or Bourne shell ( Bourne Shell) one of the first shells to be used in UNIX-like environments. This is a basic shell, small and with a small set of capabilities. She is de facto standard shell, and is present on every UNIX system. On Linux, /bin/sh can be a symbolic link to bash. This is done to ensure compatibility with UNIX applications. bash, or new shell Borna ( Bourne Again Shell) standard Linux shell. Most Linux users have the default shell installed bash. In a manner bash- this is an add-on to sh, a set of add-ons and additional modules. Thus, the new Bourne shell is compatible with the regular Bourne shell: commands running in sh, will work in bash, but not necessarily the other way around. csh, or C-oriented shell ( C Shell) the syntax of this shell is similar to the C programming language. Typically, this shell is used by programmers. tcsh, or an improved C-oriented wrapper ( Turbo C Shell) superstructure over csh. ksh, or Korn shell ( Korn Shell) Was written in the early 1980s and featured the best features of the Bourne shell and csh at the time. It is an add-on for the Bourne shell and guarantees the execution of scripts written for the Bourne shell. POSIX standard is a Korn shell with limited capabilities.

The /etc/shells file contains information about the shells present on the system:

Mia: ~> cat /etc/shells /Bin/bash /Bin/sh /Bin/tcsh /Bin/csh

The user shell is installed in the /etc/passwd file. For example, user string mia which has a shell installed bash :

Mia: L2NOfqdlPrHwE: 504:504: Mia Maya:/home/mia:/bin/bash

Switching between shells

The user can launch another shell at any time. For example, if by default it works under the controls bash, and it’s more convenient for him to perform some actions under the shell csh, it launches this shell

Mia:~>csh

To return to control bash you need to type exit.

What is a shell and why is it needed?

Command shell in any unix-like systems, which includes GNU/Linux, is regular program, launched both in a text console (which is used less and less) and in graphical environment– in the terminal emulator window, available on any Linux system.

Its task is simple and obvious: accept a line (or lines) of input, produce them parsing and based on the results of this analysis, react accordingly - execute a command, run a program, display a diagnostic message, etc.

Almost all Linux distributions For users, the default command shell is bash (Bourne Again SHell is another Bourne command shell; Steve Bourne is the author of the first command shell in Unix - sh). In fact, it has become an unofficial standard, and improvements to it functionality continues continuously. There are other command shells - tcsh (C-shell version), ksh (Korn Shell), zsh, etc. – each has its own advantages and disadvantages, as well as its own fan groups. However, bash is more familiar to the general public with different levels preparation, that’s why I chose it. It is also worth noting that whatever capabilities the various shells have, they are all compatible with their ideological progenitor - the Bourn Shell (sh). In other words, a script written for sh will work correctly in any modern shell (the reverse is generally not true).

Benefits of the Command Line

The question may arise: why bother with the command line if there are convenient and beautiful graphical interfaces? There are many reasons for this. Firstly, not all operations are more convenient or faster to perform using GUI. Second, every program follows the fundamental principle of Unix systems: do a well-defined job and do it well. In other words, you always understand what happens when you run a particular utility (if something is not entirely clear, you should refer to the man manual). Thirdly, by mastering commands, trying their combinations and combinations of their parameters, the user studies the system, gaining valuable practical experience. You get access to powerful tools such as pipelines that allow you to organize a chain of commands for processing data, I/O redirection tools, and you can also program directly from the command shell. Perhaps it’s worth dwelling on programming in more detail, especially since many system scripts in Linux (for example, scripts for launching system services) are written for the shell.

Command shell as a programming language

So, the command shell can be considered as a programming language and as a software runtime at the same time. Of course, this language is not compiled, but interpreted. It allows the use of variables: system or own. The sequence of execution of program commands is changed using condition checking constructs and selecting the appropriate option: if-then-else and case. while loops, until and for allow you to automate repetitive actions. It is possible to combine groups of commands into logical blocks. You can even write real functions that pass parameters to them. Thus, all the signs and characteristics of a full-fledged programming language are available. Let's try to get double benefit from this - along with learning the basics of programming, we will automate our daily work.

Hello, World! Simple backup system

Everyone knows about the need for regular data backups, but users never have enough time for this boring operation. The solution is simple - organize automatic creation backup copies. This will be our first shell programming task.

#!/bin/bash # # Backup directories and files from the home directory # This batch script can be run automatically using cron # cd $HOME if [ ! -d archives ] then mkdir archives fi cur_date=`date +%Y%m%d%H%M` if [ $# -eq 0 ] ; then tar czf archive$(cur_date).tar.gz projects bin else tar czf archive$(cur_date).tar.gz $* fi if [ $? = 0 ] ; then mv archive$(cur_date).tar.gz $HOME/archives echo "$cur_date – Backup completed successfully." else echo "$cur_date - ERROR during backup." fi

Any command script (script is a script, as command shell programs are called) begins with an identifier line, in which the command interpreter is explicitly specified, indicating the full path to it. The full path is a sequential listing of all directories, starting from the root, that must be entered to get to the target file, and, of course, the name of this file. Recording the full path is extremely important to uniquely identify each file in the file system hierarchy.

Four lines of comments follow. As soon as the shell encounters the "#" character, it considers all subsequent characters to be comments and completely ignores them until the end current line. Therefore, you can start a comment not from the very beginning of the line, but accompany it with some command.

After the comments there is an empty line. It means nothing to the command shell, and no action is performed. In scripts, blank lines are usually inserted to make the code easier to read.

We finally got to the first “real” team. It allows you to change the directory (Change Directory), i.e. move from the current directory to another one passed to the command as an argument. In most cases, the target directory is specified explicitly, for example, cd /tmp or cd projects, but in our case the predefined system variable HOME is used - it contains the full path to the home directory of the current user under whose name the command script is being executed. This eliminates the need to make code changes every time we change users, because the command returns everyone to their personal directory. The dollar sign "$" in front of a variable name means that you need to extract the value contained in that variable and substitute it in place of its name on the command line. It should be especially noted that in a command language, letter case shells are important, i.e. HOME, Home and home are three different variables. By convention, uppercase letters denote the names of system variables: HOME, PATH, EDITOR, etc. This convention does not prevent users from creating their own variables with names from capital letters, but why complicate your life by violating generally accepted norms and rules? It is also not recommended to change the values of system variables unless absolutely necessary. In general, we follow a simple rule: we use system variables for read-only purposes, and if we need our own, we write its name in lowercase letters.

Our first command could be written more briefly:

cd ~

Here the "~" symbol also means the current user's home directory. Command line veterans put it even more succinctly:

The idea is that when the cd command is given no argument, it changes to the home directory.

Next up is the classic software design for checking conditions and making the appropriate decision. The general scheme is:

if<условие>then<одна или несколько команд>fi

The last word of the construction (if in reverse order) acts as a closing parenthesis, i.e. boundaries of the list of commands executed when the condition is true. The presence of fi is mandatory, even if there is only one team on the list.

To check a condition, as a rule, the test command or its alternative form of notation in square brackets is used. In other words, records

if [! -d archives ] if test ! -d archives

absolutely equivalent. I prefer square brackets because they more clearly define the boundaries of the condition being tested. Both the right and left parentheses must be separated from the condition by spaces.

The criteria for checking the condition are determined by various flags. The test command recognizes a very large list of them. In our example, the -d flag is used, which allows us to check whether the name specified after the flag actually matches existing directory(directory). The following flags are most often used when working with files:

F – does it exist regular file with a given name;

R – whether the specified file has the right to read from it;

W – whether the specified file has the right to write to it;

X – whether the specified file has the right to execute it;

S – whether the specified file has a non-zero size.

In our case, the condition is preceded by Exclamation point, denoting the operation of logical negation, so the meaning of the condition being tested becomes completely opposite. Let's try to write down the meaning of these commands in ordinary Russian:

if [! -d archives ] If the archives directory (in the current directory) does not exist, then start executing the command block: mkdir archives create the archives directory (in the current directory) fi end executing the command block.

As you can see, everything turned out to be not so complicated. With a little practice, you can easily read and create similar designs yourself. The directory creation command is so obvious that no further explanation is required.

IN next line we create our own local variable cur_date. In the vast majority of cases, variables are created by simply assigning a specific value, for example:

ten=10 string="This is a line of text"

But in our example, a little trick is used. Please note that after the equal sign - the assignment symbol - the command is written in back quotes. This form of notation allows you to assign to a variable not the string itself, but the result of its execution. Here is the output of the date command, which returns the current date and time in a format specified by a list of parameters:

%Y – current year in full form, i.e. of four digits (for example, 2009);

%m – number current month(for example, 09 – for September);

%d – current day number;

%H – current hour in 24-hour format;

%M – current minute.

Thus, if you run the command

cur_date=`date +%Y%m%d%H%M`

on the tenth of September 2009 at 22:45, then the variable cur_date will be assigned string value"200909102245". The purpose of this trick is to create a unique, non-repeating name for the archive file. If you intend to run several instances of the program within one minute, you can improve the uniqueness of the names by adding the current seconds. How? Study the date utility manual (man date) - there is nothing complicated about it.

Before we start creating an archive file, we need to determine which directories we will save in it. For greater flexibility, we can specify a set of directories to be archived by default, but provide the ability to replace this set with a list of directories passed as an argument to our command script. For this purpose, special command shell variables are used: $# – the number of parameters passed to the script and $* – all parameters passed, written in one line format.

if [ $# -eq 0 ] ; then

Checking the condition “if the number of passed parameters is zero”, then execute the following command. Note that the then keyword can be written on the condition line, separated from the conditional expression by a semicolon.

tar czf archive$(cur_date).tar.gz projects bin

The command to create an archive file and compress this file. The tar utility itself does not perform compression, but only collects all specified files and directories into a single tar file. The first flag is intended for this - c (create). Compression performs external program– here it is gzip, called by the second flag - z. If your system has more than effective program bzip2 compression, you can take advantage of it by changing the command as follows:

tar cjf archive$(cur_date).tar.bz2 projects bin

The third flag f indicates that what follows is the name of the archive file, so it is always the last one in the list of flags. Note that when substituting, the variable name is enclosed in curly braces. This is done to explicitly highlight a variable on the line surrounding it, thereby eliminating many potential problems. Extensions archive file are not assigned automatically; you fill in everything you need yourself. I've specified projects and bin as the default directories to archive, but you can write down the names of your most valuable directories here.

The else keyword opens an alternative branch of execution. The commands of this block begin to work if the condition check returns the result “false” (in our example: “the number of parameters passed is non-zero,” i.e. the user specified directory names). In this case the command will look like this:

tar czf archive$(cur_date).tar.gz $*

Here the default directories are replaced by a directory name string accepted externally. It is possible to accept and process each external parameter separately, but it is more convenient for us to pass the entire string.

At the end of the program, another check is performed. In Unix environments, all commands return a completion status code. If the command was successful, it returns code 0, otherwise the exit code will be non-zero. To check the success of the previous archiving command, we will use another special variable $?, which always contains the value of the completion code of the most recent command. If in the variable $? contains 0, i.e. The backup file was successfully created, then we move it to the archive directory:

mv archive$(cur_date).tar.gz $HOME/archives

and display the corresponding message:

echo "$cur_date – Backup completed successfully."

If the check shows that the completion code for the archiving operation is not equal to zero, then an error message is displayed:

echo "$cur_date - ERROR during backup."

This completes our command script.

To test the operation of our program, we need to save the source code described above in a file, for example, named bckp, and then, for convenience, make it executable:

chmod 750 bckp

and run:

./bckp

to create a backup of the default directories, and

./bckp docs progs works

to create a backup copy of the listed directories (specify the names of the directories that actually exist on your system, otherwise you will receive an error message).

You can place the bckp file in one of the directories specified in the system PATH variable. The most preferred locations are /usr/local/bin or $HOME/bin if you have them. After this you can run bckp as a system command.

How to automate scheduled backup operations

A few words about backup automation. For this purpose, the system serves cron scheduler, which reads work instructions from a special crontab file. To define such instructions, you need to create and edit your crontab file using the command:

crontab -e

Instructions are written in a strictly defined format (fields are separated by spaces):

minutes hours day_of_month month day_of_week command

One option for scheduling backup operations might look like this:

30 23 10,20,30 * * /usr/local/bin/bckp

This means that the backup script (you must provide the full path to this file) will run at 23:30 on the 10th, 20th and 30th of each month, regardless of the day of the week. (Asterisks indicate the entire permissible range of values, in in this case: every month - in the 4th field, any day of the week - in the 5th field)

If you prefer to summarize your results by week, and your system runs 24/7, then it makes sense to schedule backups during off-peak hours:

0 5 * * 3.5 /usr/local/bin/bckp

Here, backups will be created at 5:00 on Wednesdays and Fridays in each month (asterisk in the 4th field), regardless of the date (asterisk in the 3rd field).

You can read about all the intricacies of scheduling in the man 5 crontab manual.

Results and conclusions

The backup script discussed in this article has modest functional properties. But that was not his point the main task, but so that the reader understands what can be done on the command line, and not only copies and executes the proposed command file, but becomes interested in expanding its functions, and begins to explore the immense possibilities provided by command shells. And if someone, after reading this article, tries to improve the code given here, or writes own version, or implements his independent idea, then I will consider that the main goal has been achieved.