Essential Bash scripting

If you don’t have time to learn and master bash, worry not. I have collected some tips that will help you to be productive in less than 15 minutes.

Before we get started, always have set -u at the beginning of your script. This will force your script to error out when you refer to undefined variables. It has saved me a lot of time.

Variables

In shell, you define variable with assignment. You refer to the variable using $variable_name. Variables defined in shell are not visible to child processes. To make them visible, export the variable.

user_name=roy
export user_name

# Alternatively you can assign an export in the same line.
export user_name=roy

# You can use output from other commands to initialize variable.
user_name=$(whoami)

# Then you can refer to the variable like this.
grep $user_name /etc/passwd

These are the built-in variables that I frequently use.

$0 - Script name
$* - All command line arguments
$1 - First command line argument
$$ - Script process ID
$? - Exit status of the last command
$# - Number of arguments remaining in $*

Conditionals

Just keep this one rule in mind: always use [[ and ]].

-eq, -ne, -lt, -gt, -le, -ge - For numeric comparisons
==, !=, <, >, <=, >= - For string comparisons
|| - Logical OR
&& - Logical AND
"( " and " )" - For grouping. Note the space.

You may check for exit status of a subcommand often. Use the following idiom: use the command directly as if condition without [[ and ]]. If the command exits with zero status, it is treated as true value. Otherwise else part is executed.

if grep -q pattern input.txt ; then
   echo "Pattern found"
else
   echo "Pattern not found"
fi

Looping

for can be used to implement a loop that is over a list of values. while can be used to implement a loop using condition.

for x in 1 2 3 ; do
    echo $x
done

while [[ $# -ne 0 ]] ; do
    echo "$1"
    shift
done

Subcommands

Usually grave accent (`) is used for subcommands. Just remember this: DO NOT use `, use $(cmd) construct instead. It is nestable: you can write $(cmd1 $(cmd2)). And more readable.

for x in $(cat names.txt) ; do
   echo "Name: $x"
done

Redirection

You can redirect the standard output using > and redirect the standard error using 2> . If you want to redirect standard error to standard output: 2>&1.

grep pattern input.txt > matching_lines.txt
grep pattern missing_file.txt > matching_lines.txt 2>/dev/null
grep pattern input.txt missing_file.txt > matching_lines.txt 2>&1

Pipes

The most powerful feature you can learn is pipes in shell. You can send the output of one command as input to another command. You can arbitrarily chain commands. You use | symbol to create pipes.

grep pattern input.txt | tee matching_lines.txt

# Generate a histogram of the first column from a file
awk '{print $1;}' input.txt | sort | uniq -c

Temporary output file

When you want to store the output of a command in a temporary file and then pass the temporary file name as argument to another command, there is an easy way to do it: <(). When you use this construct, output of the command is stored in a file and the file name substituted.

# Create a tar file containing all the .conf files from current directory.
tar cvfz all_confs.tar.gz --files-from <(find . -name '*.conf')

Debugging

If you have set -x at the top of your script, every command will be printed as it is executed. Commands are printed after variable expansion. Hence it is extremely useful to trace and understand if the script is doing what you expect it to do.

Where to go from here

You must practice all the constructs you read in this post. That is the only way to gain proficiency. If you would like to learn more, here are a couple of references:

  1. Advanced Bash Scripting Guide
  2. Bash Cookbook recipes. This repository contains all programs from the Bash Cookbook.

links

social