Many are used to Graphical User Interfaces (GUI) and these GUIs are in general very convenient. However, when manipulating text files and text in general, the Command Line Interface or CLI can be very powerful and effective as well. In this module we take a look at the command line interface.
A command line interface is sometimes called a shell. The term shell refers to the outer layer of the operating system that is exposed to the user and we’ve already discussed two types of shells, namely GUIs and CLIs. However, the name shell is not often used for GUIs but more for CLIs. Another often used name for the command line interface is “terminal”. A terminal in this context is a program in the GUI that shows the command line interface. In practice, the terms command line, shell and terminal are synonymous.
The command line uses a special program to interpret commands, the command line interpreter. There are many command line interpreters available such as the Bourne Shell (sh), the Bourne Again Shell (bash), csh, COMMAND.COM but we are going to use Bash, the Bourne Again Shell, one of the most widely used shells that is available on almost every computer system, even on Windows nowadays.
On Mac OS, Bash is already available; simply open the program “Terminal”.
For Windows, we open the “Control Panel” and choose “Programs” and then “Programs and Features”. Here we click on “Turn Windows features on or off” on the left side and turn on “Windows Subsystem for Linux”. This requires a reboot. After the reboot we install “Ubuntu 20.04 LTS” from the Microsoft Store and launch it.
During the initial launch, Ubuntu will install itself. It will ask for a default UNIX username that can be different from your Windows username. UNIX usernames are typically all in lower case without any signs, so just letters, examples are:
jamiedoejdoejaromilThe system will then ask for your password and confirmation.
Most likely, Ubuntu will want you to upgrade. To do so, run the following command that gives you superuser powers (superuser do) and requires you to type your password:
sudo apt updateThis command retrieves the new list of all available software packages that can be installed and upgraded. Then run the following command to actually upgrade:
sudo apt upgradePress Enter to confirm. After this has finished, we are
in a shell with Bash.
Linux installations have almost always Bash installed and often as default shell. Depending on your system open a terminal.
It is unclear whether we are actually are running Bash now. To make
sure that we are running Bash we can simply execute the command
bash. This starts the shell Bash from the shell you were
using. Note that this might have been Bash all along or perhaps a
different shell. After running bash you are certain that
you are using Bash as the command line interpreter.
[jamie@computer ~]$ bash [jamie@computer ~]$
The computer “prompts” the user to enter a command. The
prompt shows between brackets the current user
jamie, the hostname of the computer computer,
and the current directory ~ which refers to the user’s home
directory. The $ means that the user is prompted to give
input as a normal user without elevated privileges. The
root user is the account with elevated privileges, the
administrator account. As user root, the prompt sign is
typically #.
To verify that we are using Bash, we can run the program
env. This command will show you text with a set of variable
names and their values. One of those variables (called environment
variables) will be SHELL with probably the value
/bin/bash or perhaps /usr/bin/bash. If this is
not the case, then you are not using Bash as the command line
interpreter.
[jamie@computer ~]$ env SHELL=/bin/bash PWD=/home/jamie LOGNAME=jamie MOTD_SHOWN=pam HOME=/home/jamie LANG=en_US.UTF-8 TERM=xterm-256color USER=jamie XDG_SESSION_ID=14 LC_TIME=en_GB.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl MAIL=/var/spool/mail/jamie _=/usr/bin/env ... [jamie@computer ~]$
A very useful command is man with which you can show
manuals of commands. For example you can do man bash to
read documentation on Bash or even man man to read
documenation on man itself.
BASH(1) General Commands Manual BASH(1)
NAME
bash - GNU Bourne-Again SHell
SYNOPSIS
bash [options] [command_string | file]
COPYRIGHT
Bash is Copyright (C) 1989-2020 by the Free Software Foundation, Inc.
DESCRIPTION
Bash is an sh-compatible command language interpreter that executes
commands read from the standard input or from a file. Bash also incor‐
porates useful features from the Korn and C shells (ksh and csh).
Bash is intended to be a conformant implementation of the Shell and
Utilities portion of the IEEE POSIX specification (IEEE Standard
1003.1). Bash can be configured to be POSIX-conformant by default.
OPTIONS
All of the single-character shell options documented in the description
of the set builtin command, including -o, can be used as options when
Manual page bash(1) line 1 (press h for help or q to quit)
Another very useful command is pwd, show the current
working directory. The shell is always in a directory, the working
directory. Note that / is the root directory and
directory paths are separated by /, so
/home/jamie is the directory jamie in the
directory /home and home is in directory
/ (pronounced as root).
[jamie@computer ~]$ pwd /home/jamie [jamie@computer ~]$
To list the files and directories in the current working directory,
you can execute ls (for simply listing the files and
directories), ls -l (listing more metadata about each file)
and ls -al that also lists hidden files. Hidden files are
prefixed with a dot, so .bashrc would be a hidden resource
file for Bash.
[jamie@computer ~]$ ls -al total 28 drwx------ 2 jamie jamie 4096 Jan 30 18:06 . drwxr-xr-x 5 root root 4096 Jan 30 12:28 .. -rw------- 1 jamie jamie 3101 Jan 30 17:53 .bash_history -rw-r--r-- 1 jamie jamie 21 Jan 8 2022 .bash_logout -rw-r--r-- 1 jamie jamie 57 Jan 8 2022 .bash_profile -rw-r--r-- 1 jamie jamie 141 Jan 30 12:30 .bashrc_bak -rw-r--r-- 1 jamie jamie 3729 Feb 2 2022 .screenrc [jamie@computer ~]$
To make a new directory, we can execute mkdir new-dir
that creates a directory new-dir in the current working
directory. To go there, we can change the current working directory with
the command cd new-dir (the command cd with
argument new-dir. In a sense you now went deeper in the
directory tree. To go back up in the directory tree you can do
cd .. where .. essentially means one level
higher.
[jamie@computer ~]$ mkdir new-dir [jamie@computer ~]$ cd new-dir/ [jamie@computer new-dir]$ cd .. [jamie@computer ~]$
Note that in many cases, Bash can autocomplete commands or arguments
for you. So, for example, removing new-dir we can type
rmdir n<Tab> and if there is only one directory that
starts with n, Bash will immediately autocomplete this. If
there are more than one options, you can type an additional
Tab, so rmdir n<Tab><Tab> and Bash
will show you the list of options. You can type an additional character
to reduce the options until there is only one option that Bash will then
autocomlete.
Note that this works for commands as well, so most likely, you would be able to this:
rmd<Tab> n<Tab>to execute
rmdir new-dirCommand line tools typically print text: all the output of the command line tools we’ve seen so far is text. Typically the input to command line tools is text as well.
Since the env command may show many different
environment variables but we’re only interested in the
SHELL variable, we prefer to select from the output of
env only the line with SHELL in it. Since the
output of env is simply text, we can apply some text
processing tools to achieve just this. The tool grep takes
as input a so called regular expression and text and filters the text
based on this regular expression. The name grep stands for
[g]et [r]egular [e]x[p]ression. With the following command we “pipe” the
text output of env to serve as input for the command
grep where we use the regular expression “SHELL” to match
any line with this string in it:
env | grep SHELLYou can now inspect only the line we are interested in:
[jamie@computer ~]$ env | grep SHELL SHELL=/bin/bash [jamie@computer ~]$
In the previous module we talked about text files and binary files.
To verify that the output of the env is indeed only text we
can take a look at the very bits of that output. We first store the
output of the env command in a file by “piping” the output
of the env command directly to the file
output-env.txt with >:
env > output-env.txt[jamie@computer ~]$ env > output-env.txt [jamie@computer ~]$
If we execute ls we can see the file
output-env.txt is now listed. We can look at the contents
of the file as if it were executed by env with the command
cat (from concatenate, it can concatenate multiple files
into one, don’t forget autocompletion
cat o<Tab>):
cat output-env.txt[jamie@computer ~]$ ls output-env.txt [jamie@computer ~]$ cat output-env.txt SHELL=/bin/bash PWD=/home/jamie LOGNAME=jamie MOTD_SHOWN=pam HOME=/home/jamie LANG=en_US.UTF-8 TERM=xterm-256color USER=jamie XDG_SESSION_ID=14 LC_TIME=en_GB.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl MAIL=/var/spool/mail/jamie _=/usr/bin/env ... [jamie@computer ~]$
The program xxd takes as input a file and shows the
hexadecimal representation of the bytes together with the character if
the byte happens to represent a character. If the byte cannot be
interpreted as a character, it is shown as a dot.
The output of xxd applied on the file is
(xxd o<Tab>):
xxd output-env.txtThe output consists of three columns. At the left you can see the offset of the bytes in the file in hexadecimal notation. The middle column shows you the value of the bytes in hexadecimal. The right column shows you the character if the byte can be interpreted as such. Below you can see a possible output:
00000000: 5348 454c 4c3d 2f62 696e 2f62 6173 680a SHELL=/bin/bash. 00000010: 5057 443d 2f68 6f6d 652f 6a61 6d69 650a PWD=/home/jamie. 00000020: 4c4f 474e 414d 453d 6a61 6d69 650a 4d4f LOGNAME=jamie.MO 00000030: 5444 5f53 484f 574e 3d70 616d 0a48 4f4d TD_SHOWN=pam.HOM 00000040: 453d 2f68 6f6d 652f 6a61 6d69 650a 4c41 E=/home/jamie.LA 00000050: 4e47 3d65 6e5f 5553 2e55 5446 2d38 0a54 NG=en_US.UTF-8.T 00000060: 4552 4d3d 7874 6572 6d2d 3235 3663 6f6c ERM=xterm-256col 00000070: 6f72 0a55 5345 523d 6a61 6d69 650a 5844 or.USER=jamie.XD 00000080: 475f 5345 5353 494f 4e5f 4944 3d31 340a G_SESSION_ID=14. 00000090: 4c43 5f54 494d 453d 656e 5f47 422e 5554 LC_TIME=en_GB.UT
The first byte has value 0x53 and apparently this is the ASCII value for the character ‘S’. If we look up 0x53 ($5 * 16 + 3) = 83) in the ASCII Table, we can verify that this is indeed the value for character ‘S’. In the last position on the first line, we see the non-printable byte 0x0a (10). We can see in the ASCII Table that this byte represents a linefeed, or a newline in the ASCII encoding. This byte tells the program showing text to insert a new line for the following text.
We can contrast this text file with a binary file. The shell that you
are running, Bash, is an executable, a program. We can view the contents
of this program by means of xxd as well. In the above
output we can see that my shell is the file /bin/bash.
Since an executable is likely to be a large file we want to show only
the first part of the output of xxd. To accomplish this we
pipe the output of xxd as input to the command
head that shows only the first ten lines:
xxd /bin/bash | headThe output is:
[jamie@computer ~]$ xxd /bin/bash | head 00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000 .ELF............ 00000010: 0300 3e00 0100 0000 f018 0200 0000 0000 ..>............. 00000020: 4000 0000 0000 0000 1073 0e00 0000 0000 @........s...... 00000030: 0000 0000 4000 3800 0d00 4000 1a00 1900 ....@.8...@..... 00000040: 0600 0000 0400 0000 4000 0000 0000 0000 ........@....... 00000050: 4000 0000 0000 0000 4000 0000 0000 0000 @.......@....... 00000060: d802 0000 0000 0000 d802 0000 0000 0000 ................ 00000070: 0800 0000 0000 0000 0300 0000 0400 0000 ................ 00000080: 1803 0000 0000 0000 1803 0000 0000 0000 ................ 00000090: 1803 0000 0000 0000 1c00 0000 0000 0000 ................ [jamie@computer ~]$
We can see that the second to fourth byte are the characters “ELF” (denoting that this is an ELF binary format) and some bytes can be interpreted as characters but this is likely a coincedence.
We could execute cat /bin/bash | head but this will
likely give us gibberish and we rely on head encountering
10 coincedental newlines to stop the input. With reset we
can reset the terminal to normal in case it broke.
Besides this text processing capabilities, a large advantage of text
files is the fact that it is possible to conveniently show differences
between text files. Let’s create two files that are very similar. We
will use the command echo that simply echoes the text:
echo Is anyone there?[jamie@computer ~]$ echo Is anyone there? Is anyone there? [jamie@computer ~]$
We can redirect this to a file appending each line with
>> to the file:
echo Is anyone there? >> file1
echo Come here! >> file1
echo Come here! >> file1[jamie@computer ~]$ echo Is anyone there? >> file1 [jamie@computer ~]$ echo Come here! >> file1 [jamie@computer ~]$ echo Come here! >> file1 [jamie@computer ~]$
We then copy the file to file2:
cp file1 file2[jamie@computer ~]$ cp file1 file2 [jamie@computer ~]$
And add the following to file1:
echo This way, we must come together. >> file1[jamie@computer ~]$ echo This way, we must come together. >> file1 [jamie@computer ~]$
And the following to file2:
echo We must come together! >> file2[jamie@computer ~]$ echo We must come together! >> file2 [jamie@computer ~]$
We have now two files that are for a large part similar but different
in the last line. We can show this difference with the utility
diff:
diff --color file1 file2[jamie@computer ~]$ diff --color file1 file2 4c4 < This way, we must come together. --- > We must come together! [jamie@computer ~]$
You can see that the lines that are similar are ignored and the lines that are different are highlighted. The ability to visualize differences in lines of text is a very powerful feature of text files. It is often used in programming to highlight the difference in source code from an old version to a new version.
Visualizing differences in binary files is not possible in the same
manner. For example, typically, CAD files are not text based and to
highlight differences between two versions of the same CAD file would
need support from the CAD program to show the differences, whereas for
text files we can use the standard diff utility to
accomplish this.
To conclude this section, the fact that text files are easy to
analyze and process with standardized tools such as diff
and grep makes the command line and text files popular in
programming. Because these features, source code is often written in
text files and programmers use special programs called text
editors to process files.
Text editors are programs with powerful text processing features for text files. They are often used by programmers and there are many different versions with different capabilities. Two of the most popular and old text editors are Emacs and Vim (derived from the older, also still used version vi). Nowadays, Integrated Development Environments (IDEs) are also popular such as Visual Studio Code and Eclipse and these programs often provide many services for programming around the core of a text editor. Often these text editors are loosely based on the two “old” editors Vim and Emacs. Since Vim and Emacs are very powerful editors, the distinction between IDEs and text editor is sometimes not so clear.
For this primer, we will introduce the simple command-line text
editor nano, a text editor that is simple, lightweight and
almost always installed by default on Linux machines.
For example, we can open file1 from above in nano
with:
nano file1GNU nano 7.2 file1 Is anyone there? Come here! Come here! This way, we must come together. [ Read 4 lines ] ^G Help ^O Write Out ^W Where Is ^K Cut ^T Execute ^C Location ^X Exit ^R Read File ^\ Replace ^U Paste ^J Justify ^/ Go To Line
We can simply see the contents of the file and we see at the bottom
commands. For example, we can close nano with the command
^X, which means Ctrl-x. We will use this tool
in the next section that introduces us to Git.