Expert Guide Shell scripts: Step-by-step guide to error handling and avoidance

How you can use error handling in shell scripts

The following text deals with a topic that is often neglected in practice: Error handling and avoidance in shell scripts.

Shell scripts are often used in Linux administration to automate smaller and larger, recurring tasks. Unfortunately, administrators are often less experienced on the field of programming, and bash has a drawback in that scripts will continue to run even after serious problems. In the worst case, scripts can do a lot of damage; think of a script that needs to fetch new data and store it to a location, but fails and overwrites existing files with empty files.

Step 1: Missing target directory

Let’s look at a small task: a script is to copy /etc/ regularly to /virtual/etc-backup, using a timestamp. A simple implementation follows:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup"
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE"

We get consecutive directories with names such as 2025-01-08 15:37:30 written to /virtual/etc-backup . There are several possible errors that aren’t addressed:

• The target directory does not exist or is no directory
• The target directory is not writable
• The source directory does not exist or files therein are not readable
• Other reasons

First, let’s check if the target directory exists:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup"
if [ ! -d "$BACKUP_DIR" ]; then
echo "Backup path $BACKUP_DIR does not exist or is not a directory."
exit 1
fi
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE"

With that, we catch a possible error, which is even more necessary should BACKUP_DIR be build from a user supplied option. Therefore, this error is checked for first. exit 1 not only ends the script, but also returns a value unequal to 0. This allows you to check whether the script has run without errors after it has been called..

Step 2: Server restarts or crashes during the copy process

Let’s look at another possible problem, which we are not able to catch by ourselves: while running, the server crashes, which results in an incomplete backup. One might argue that an incomplete backup is better than no backup, but I’d like to know if a copy is complete or not.

Therefore, I prefer to work with temporary files:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup/"
if [ ! -d "$BACKUP_DIR" ]; then
echo "Backup path $BACKUP_DIR does not exist or is not a directory.";
exit 1;
fi
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE.tmp"
if [ $? -ne 0 ]; then
echo "An error occurred while copying /etc to $BACKUP_DIR/$BACKUP_DATE.tmp.";
exit 1;
fi
mv "$BACKUP_DIR/$BACKUP_DATE.tmp" "$BACKUP_DIR/$BACKUP_DATE"

Now a copy is first created with the suffix .tmp . Should copying fail, the script ends, but the incomplete copy is still available and can be recognized as such.

Step 3: Compress directories

As a further improvement, let’s save space and use tar and gzip to compress a backup. We use error handling for every step:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup/"
if [ ! -d "$BACKUP_DIR" ]; then
echo "Backup path $BACKUP_DIR does not exist or is not a directory.";
exit 1;
fi
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE.tmp"
if [ $? -ne 0 ]; then
echo "An error occurred while copying /etc to $BACKUP_DIR/$BACKUP_DATE.tmp.";
exit 1;
fi
tar -cf "$BACKUP_DIR/$BACKUP_DATE.tmp.tar" "$BACKUP_DIR/$BACKUP_DATE.tmp"
if [ $? -ne 0 ]; then
echo "An error occurred while creating the tar archive $BACKUP_DIR/$BACKUP_DATE.tmp.tar.";
exit 1;
fi
gzip "$BACKUP_DIR/$BACKUP_DATE.tmp.tar"
if [ $? -ne 0 ]; then
echo "An error occurred while compressing the tar archive $BACKUP_DIR/$BACKUP_DATE.tmp.tar.";
exit 1;
fi
mv "$BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz" "$BACKUP_DIR/$BACKUP_DATE.tar.gz"
if [ $? -ne 0 ]; then
echo "An error occurred while renaming the tar archive $BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz to $BACKUP_DIR/$BACKUP_DATE.tar.gz.";
exit 1;
fi
# Cleanup of temporary files
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp"
if [ $? -ne 0 ]; then
echo "An error occurred while removing the temporary directory $BACKUP_DIR/$BACKUP_DATE.tmp.";
exit 1;
fi

Advantages

It is easy to understand the advantage of handling errors:

If even one command fails, the whole script will fail. However, the script got rather cumbersome to read and it can be discussed if such a detailed error handling is even necessary.

Especially, if we take into account that commands by themselves will also provide error messages. However, it is possible to use a ‘catchall’:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup/"
if [ ! -d "$BACKUP_DIR" ]; then
echo "Backup path $BACKUP_DIR does not exist or is not a directory.";
exit 1;
fi
trap 'echo "An error occurred, check output. Exiting..."; exit 1;' ERR
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE.tmp"
tar -cf "$BACKUP_DIR/$BACKUP_DATE.tmp.tar" "$BACKUP_DIR/$BACKUP_DATE.tmp"
gzip "$BACKUP_DIR/$BACKUP_DATE.tmp.tar"
mv "$BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz" "$BACKUP_DIR/$BACKUP_DATE.tar.gz"
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp"

trap catches the first error encountered ( ERR ) and exits the script. Therefore, we don’t have to evaluate every return value by itself, also, the script ends before calling commands that would make no more sense after previous commands have failed.

We can improve this with a separate function, which also allows us to catch SIGINT ( CTRL+C ).

If the latter happens, we clean up:

#!/bin/bash
BACKUP_DIR="/virtual/etc-backup/"
# Check if backup directory exists and is a directory.
if [ ! -d "$BACKUP_DIR" ]; then
echo "Backup path $BACKUP_DIR does not exist or is not a directory.";
exit 1;
fi
# Function to handle errors
handle_error() {
echo "An error occurred on line $1, check output. Exiting...";
exit 1;
}

# Function to handle SIGINT
handle_sigint() {
echo "SIGINT received, cleaning up and exiting...";
if [ -d "$BACKUP_DIR/$BACKUP_DATE.tmp" ]; then
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp"
fi
if [ -f "$BACKUP_DIR/$BACKUP_DATE.tmp.tar" ]; then
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp.tar"
fi
if [ -f "$BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz" ]; then
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz"
fi
exit 1;
}

# Trap errors and SIGINT
trap 'handle_error $LINENO' ERR
trap 'handle_sigint' SIGINT

# Backup
BACKUP_DATE=$(date +"%Y-%m-%d %H:%M:%S")
cp -a /etc "$BACKUP_DIR/$BACKUP_DATE.tmp"
tar cf "$BACKUP_DIR/$BACKUP_DATE.tmp.tar" "$BACKUP_DIR/$BACKUP_DATE.tmp"
# Simulate a long running process
echo "Sleeping for 10 seconds, press CTRL+C if you are impatient..."
sleep 10
echo "Waking up."
gzip "$BACKUP_DIR/$BACKUP_DATE.tmp.tar"
mv "$BACKUP_DIR/$BACKUP_DATE.tmp.tar.gz" "$BACKUP_DIR/$BACKUP_DATE.tar.gz"
rm -rf "$BACKUP_DIR/$BACKUP_DATE.tmp"

Conclusion

Using this example, we learned how a script can become more and more robust using error handling and how trap allows us to catch errors as well as POSIX signals (man signal to learn more). This keeps our script nice and tidy. We also discussed how to think ahead and plan for external factors like a system crash.