Variables, functions, environment
Your ~/bin and PATH
The PATH is an environment variable. It is a colon delimited list of directories that your shell searches through when you enter a command. Binaries are at /bin, /usr/bin, /usr/local/bin etc. The best place for your own is ~/bin.:
# add to .bashrc
export PATH="$PATH:$HOME/bin"
# after you have your script written, set +x bit and run it
chmod +x ~/bin/script_name.sh
script_name.sh
You can find where a program is using which
or type -a
, we recommend the later one:
type -a ls # a binary
type -a cd # builtin
Other options:
# +x bit and ./
chmod +x script.sh
./script.sh # that works if script.sh has #!/bin/bash as a first line
# with no x bit
bash script.sh # this will work even without #!/bin/bash
Extension is optional note that .sh extension is optional, script may have any name
Variables
In shell, variables define your environment. Common practice is that environmental vars are written IN CAPITAL: $HOME, $SHELL, $PATH, $PS1, $RANDOM. To list all defined variables printenv
. All variables can be used or even redefined. No error if you call an undefined var, it is just considered to be empty:
# assign a variable, note, no need for ; delimiter
var1=100 var2='some string'
# calling a variable is just putting a $ dollar sign in a front
echo "var1 is $var1"
# re-assign to another var
var3=$var1
# when appending a variable, it is considered to be a string
var+=<string>/<integer>
var1+=50 # var1 is now 10050
var2+=' more' # var2 is 'some string more'
# we come later to how to deal with the integers (Arithmetic Expanssions $(()) below)
There is no need to declare things in advance: there is flexible typing. In fact, you can access any variable, defined or not. However, you can still declare things to be of a certain type if you need to:
declare -r var=xyz # read-only
declare -i var # must be treated as an integer, 'man bash' for other declare options
BASH is smart enough to distinguish a variable inline without special quoting:
dir=$HOME/dir1 fname=file fext=xyz echo "$dir/$fname.$fext"
though if variable followed by a number or a letter, you have to explicitly separate it with the braces syntax:
echo ${dir}2/${file}abc.$fext
Built-in vars:
$? exit status of the last command
$$ current shell pid
$# number of input parameters
$0 running script name, full path
$FUNCNAME function name being executed, [ note: actually an array ${FUNCNAME[*]} ]
$1, $2 … input parameter one by one (function/script)
“$@” all input parameters as is in one line
f() { echo -e " number of input params: $#\n input params: $@\n shell process id: $$\n script name: $0\n function name: $FUNCNAME"; return 1; }; f arg1 arg2; echo "exit code: $?"
What if you assing a variable to a variable like:
var2='something'
var1=\$var2
echo $var1 # will return '$var2' literally
# BASH provides built-in 'eval' command that reads the string then re-evaluate it
# if variables etc found, they are given another chance to show themselves
eval echo $var1 # returns 'something'
In more realistic examples it is often used to compose a command string based on input parameters or some conditionals and then evaluate it at very end.
Magic of BASH variables
BASH provides wide abilities to work with the vars “on-the-fly” with
${var...}
like constructions. This lets you do simple text
processing easily. These are nice, but are easy to forget so you will
need to look them up when you need them.
Assign a $var with default value if not defined:
${var:=value}
Returns $var value or a default value if not defined:
${var:-value}
Print an error_message if var empty:
${var:?error_message}
Extract a substring:
${var:offset:length}
, examplevar=abcde; echo ${var:1:3}
returns ‘bcd’Variable’s length:
${#var}
Replace beginning part:
${var#prefix}
Replace trailing part:
${var%suffix}
Replace pattern with the string:
${var/pattern/string}
Modify the case of alphabetic characters:
${var,,}
for lower case or${var^^}
for upper case
# will print default_value, which can be a variable
var=''; echo ${var:-default_value}
var1=another_value; var=''; echo ${var:-$var1}
# assign the var if it is not defined
# note that we use ':' no operation command, to avoid BASH's 'command not found' errors
: ${var:=default_value}
# will print 'not defined' in both cases
var=''; echo ${var:?not defined}
var=''; err='not defined'; echo ${var:?$err}
# will return 8, that is a number of characters
var='abcdefgh'; echo ${#var}
# returns file.ext
var=26_file.ext; echo ${var#[0-9][0-9]_}
# returns archive.tar.gz out of full path
fpath=/home/user/archive.tar.gz; echo ${fpath##*/}
# returns path with no file name
echo ${fpath%/*}
# in both cases returns photo
var=photo.jpeg; echo ${var%.jpeg}
var=26_file.ext; echo ${var%.[a-z][a-z][a-z]}
# returns 'I hate you'
var='I love you'; echo ${var/love/hate}
# other options for substitutions
var=' some text ';
echo ${var/# /} # returns without the first space
echo ${var/% /} # without the last space
echo ${var// /} # without spaces at all
Except for the := the variable remains unchanged. If you want to redefine a variable:
var='I love you'; var=${var/love/hate}; echo $var # returns 'I hate you'
BASH allows indirect referencing, consider:
var1='Hello' var2=var1
echo $var2 # returns text 'var1'
echo ${!var2} # returns 'Hello' instead of 'var1'
To address special characters:
# replacing all tabs with the spaces in the var
var=${var//$'\t'/ }
Functions
Alias is a shortcut to a long command, while function is a piece of programming that has logic and can accept input parameters. Functions can be defined on-the-fly from the cli, or can go to a file. Let us set ~/bin/functions and collect everything useful there.:
# whoami alternative turned into function
me() {
$(id -un):$(id -gn)@$(hostname -s)
}
# turn check space usage into a function
spaceusage() {
du -hs * .[!.]* | sort -h
}
# in one line, note spaces and ; delimiters
myfunction() { command; command; }
# -or- in a full format
function myfunction { command; command; }
Read functions into the current shell environment and run them:
source ~/bin/functions
me
spaceusage
The function refers to passed arguments by their position (not by name), that is $1, $2, and so forth:
func_name arg1 arg2 arg3 # will become $1 $2 $3
# advanced version of spaceusage using BASH variables magic
spaceusage() {
du -hs ${1:-.}/* ${1:-.}/.[!.]* | sort -h;
}
Functions in BASH have return
but it only returns the exit code. Useful
in cases where you want to ‘exit’ the function and continue to the rest of the script.
By default functions’ variables are in the global space, once chaged in the function is
seen everywhere else. local
can be used to localize the vars. Compare:
var=2; f() { var=3; }; f; echo $var
var=2; f() { local var=3; }; f; echo $var
# get filename out of path
filename() {
local fpath=${1:?path is missing} && \
echo ${fpath##*/}
}
# filepath with no name
filepath() {
local fpath=${1:?path is missing} && \
echo ${fpath%/*}
}
If you happened to build a function in an alias way, redefining a command name while using that original command inside the function, you need to type command before the name of the command, like:
rm() { command rm -i "$@"; }
here you avoid internal loops (forkbombs).
Exporting a function with export -f function_name
lets you pass a function to a sub-shell,
by storing that function in a environment variable. Helpful when you want to use it within a
command substitution, or any other case that launches a subshell, like
find ... -exec bash -c 'function_name {}' \;
.
Exercise 2.2
Exercise
Add
spaceusage()
,filename()
,filepath()
,me()
to your ~/bin/functions and play with them. Note: here and later, we suggest that all newly created functions would go to ~/bin/functions file.Using
filename()
function make afilebasename()
so that function would output a filename with no extension. Likefilebasename path/to/archive.tar.gz
would return archive.Using
find
utility, implement a fast find (=*ff*) functionff word
. This function must return all the files and directories in the current folder which name contains <word>. Let it be case insensitive. Hint:find . -iname ...
(*) Make an advanced version of
ff()
that would accept a directory name to search at as a second argument ($2) and if it is missing then would use current. For a exampleff word path/to/
.(*)
:() { :|:&; };:
is a BASH fork-bomb [WARNING: Do not run it!]. Can you explain how it works it? & in this case sends process to the background.(*) On Triton write a function that
lfs find
all the dirs/files at $WRKDIR that do not belong to your group and fix the group ownership. Usefind ... | xargs
. Tip: on Triton at WRKDIR your username $USER and group name are the same. On any other filesystem,$(id -gn)
returns your group name. One can(*) Expand the function above to set group’s s-bit on all the $WRKDIR directories.