Bash shell and scripting

From Knowledge Kitchen
Jump to navigation Jump to search


Brian Fox, creator of bash, the Bourne-Again Shell

Introduction

Bash logo

bash is the most commonly used shell for Unix/Linux.

  • written by Brian Fox at the behest of Richard Stallman and the GNU project.
  • it aimed to be a replacement for UNIX's Bourne shell
  • the name 'bash' is an acronym for the pun, 'Bourne-Again SHell'
  • Brian Fox also created the first-ever online banking platform and an online election platform

Bash commands can be entered individually into the command line, or saved into a file and executed as a batch.

  • When you run commands using the command line interface on a Mac using Terminal, on a Windows computer using Git Bash or WSL, or on a Linux machine, you are most likely using bash
  • Bash scripts saved into files are usually given the .sh extension

Documentation

Manual

Manual for the bash command

man bash

Get word count (more than 5,000 lines)

man bash | wc -l

Info

Info for bash

info bash

Get the word count (more than 9,500 lines)

info bash | wc -l

Online documentation

For even more details, view the online reference manual for bash:

https://www.gnu.org/software/bash/manual/bashref.html

Basic setup

Using shebang

The very first two characters in a bash script file must be shebang '#!'

Follow this with the path name to bash, e.g. (your path may differ)

#!/bin/bash

You can also automatically find the location of bash within the environment settings

#!/usr/bin/env bash

Anytime shebang is used, the kernel will pass the path to the script as the first argument to the script

File permissions

Make sure scripts have execute permissions to execute it directly

chmod u+x file.sh
./file.sh

If you don't have execute permission, but do have read permission, you can still run the file

bash file.sh

Built-in commands

Get a list of all built-ins in bash

enable

Get a list of keywords in bash

compgen -k

PATH variable

Contains your path directories, separated by colons

echo $PATH

Add a directory to the path:

PATH=$PATH:/usr/local/bin

Tracking time

The time command can find out how long some command takes to process. It reports

  • real time: However much time in minutes and seconds
  • user time: CPU time - how much time the process is directly being executed on the CPU (i.e. not sleeping or waiting for other processes)
  • system time: CPU time how much time the OS/kernel is doing processing in order to execute the process on the CPU
time <some other command>

e.g.

time find / -name core

Variables

Variable assignments must not have spaces around the = sign, otherwise the thing after the space is interpreted as a separate command:

some_variable_name="this is a string"

Read value of a variable with a $ in front of variable name

echo the value of some_variable_name is $some_variable_name

Remove a variable

unset some_variable_name

A copy of a variable defined within a script can be copied into your command-line environment, which makes it available to any other scripts you run

export some_variable_name

or

export some_variable_name=2

Functions can also be exported

export -f some_function_name

See what has been exported

export

Grouping

Parentheses

Commands grouped within parentheses are separate processes. Variables defined within the sub-process are not shared with the parent process.

a=1
(
a=2
)
echo $a
# prints 1

Braces

However, braces behave differently - these do not spawn a sub-process, so variables are shared:

a=1
{
a=2
}
echo $a
# prints 2

Startup settings

.bash_profile

Eecuted every time you log in

  • often used for setting environment variables

.bash_rc

Executed every time a bash shell is started

  • often used to set aliases and functions, since those are not normally exported from one shell to another like variables are

Aliases

Aliases allow you to set other names for common commands:

ll="ls -l"

To execute a command alias:

$ll

See all aliases set in the shell

alias

Unset an alias

unalias ll

Pipes

Pipes allow the output of one program to serve as input for another

List only .txt files in the current working directory:

ls -l | grep "\.txt$"

Swap all vowels in a file listing with 'u'

ls -l | sed -e "s/[aeio]/u/g"

Conditionals

Conditionals are expressed similarly to other programming languages, with some nuances:

  • a space must be included between the square brackets [ ... ] and the contents inside them
#!/bin/bash
if [ "foo" == "foo" ]; then
   echo expression evaluated as true
fi

An example including a variable.

#!/bin/bash
T1="foo"
T2="bar"
if [ "$T1" == "$T2" ]; then
    echo expression evaluated as true
else
    echo expression evaluated as false
fi


Loops

Like other programming languages, bash supports looping.

For loops

An example of a for loop that captures the output of the UNIX ls command and loops through each of the lines. This script could be made more useful if particular actions were taken on the various files, rather than simply echoing them out.

#!/bin/bash
for i in $( ls ); do
    echo item: $i
done

While loops

A classic counter example. Note that the less than sign < has special meaning, and so the bash syntax requires another way to do a 'less than' comparison. The same applies to greater than > operations.

#!/bin/bash 
COUNTER=0
while [  $COUNTER -lt 10 ]; do
	echo The counter is $COUNTER
	let COUNTER=COUNTER+1 
done

Functions

The following example includes a function with a local variable

#!/bin/bash
HELLO=Hello 
function hello {
        local HELLO=World
        echo $HELLO
}
echo $HELLO
hello
echo $HELLO

Another example, this time of a function with parameters, which are automatically stored in numbered local variables

#!/bin/bash 
function e {
    echo $1 
}  
e Hello
e World

Bash for automation

A basic understanding of bash scripts is often crucial to the proper setup of sophisticated software development automation.

In continuous integration / continuous delivery / continuous deployment, tests, builds, delivery, and/or deployment are automatically triggered when changes are committed to a repository. Some automated build systems that are triggered in such scenarios have their own custom scripting language if little customization is required. However, most continuous integration servers, such as Jenkins and Travis CI, can bet set up to execute a bash script that specifies the steps necessary to perform custom automated tasks.

An example of a bash script that could launch a unit test scripts when particular branches are modified, and pass some relevant Jenkins environmental variables to the tests:

#!/bin/bash
if [[ $BRANCH_NAME == "master" ]] || [[ $BRANCH_NAME == "master_dev" ]]
then
    ./runUnitTests.sh $REPOSITORY_NAME $BASE_BUILD_CORE $BRANCH_NAME $BUILD_NUMBER || echo "The npm may fail but the report exists"
fi

Making backups

An example use of bash to make a backup of a directory saved into a dated compressed file. Note the use of parentheses around the UNIX date command to capture its output and include it in the filename.

#!/bin/bash          
OF=/var/my-backup-$(date +%Y%m%d).tgz
tar -cZf $OF /home/me/


References


What links here