An introduction to the CLI (Command Line Interface) and Bash on Ubuntu Linux
The default shell that is installed on Ubuntu Linux is bash. Alternatives exist, but they’re beyond the scope of this tutorial. Bash is available on almost all Linux distributions, so this tutorial will work on most Linux distributions as well.
What is a shell? Simply put, the shell is a program that takes your commands from the keyboard and gives them to the operating system to perform. In the old days, it was the only user interface available on a Unix computer. Nowadays, we have graphical user interfaces (GUIs) in addition to command line interfaces (CLIs) such as the shell. Bash is the most popular shell application for Linux, and is the default on Ubuntu and hundreds of other Linux distributions, Mac OS X, and soon Windows 10.
The basics are:
You type one or more command(s), hit enter, and it runs the command(s).
Use the up/down arrows to go through your bash history. Ctrl+P also works
Use Ctrl+R to search the history of commands used previously.
Hitting tab will autocomplete commands.
Instead of typing
cd ~/myfolder1/
you can just type
cd ~/my
and it’ll either autocomplete fully or if there are still more folders (like my234 and myfolder1), it’ll show you your options. You can enter multiple commands by separating them with “;” or “&&”. For clarity, we will focus only on single commands.
Bash and Linux in general is case sensitive. That means you can have new_folder, New_folder, NEW_FOLDER, and new_Folder all in the same directory. If the computer says a file or folder doesn’t exist, you should check to see if you forgot to capitalize something. This is another good use of tab complete!
The gist of it is that a command takes in arguments/parameters, so you enter your command, add a space, add your arguments. An argument CANNOT HAVE SPACES. A space means that the argument is done and you’re putting in another argument. To circumvent this, you have two options:
Escape all spaces by using a backslash (\). This is the foolproof method that will work every time. Tab autocomplete will put these in for you.
cd My\ Really\ Annoying\ Folder\ Name
Put the argument in quotations (Works 99% of the time)
cd “My Really Annoying Folder Name”
Here are the basic things that you’ll need to know how to do to get around.
Commonly used commands
The command
What it does
Examples
man
With man, you can retrieve the information in the manual about a command and display it as text output on your screen
man ls,
page-up/down or arrow keys to browse, q to quit, / to search
ls
Lists all the files and folders in the current directory.
Commonly launched with arguments -lsah for better output formatting.
ls Documents
text.txt
ls -lsah Documents
total 3
0 drwxr-xr-x+ 76 username group 2.5K Apr 19 18:52 .
0 drwxr-xr-x 6 root admin 204B Dec 24 01:01 ..
8 -r——– 1 username group 7B Nov 10 20:41 text.txt
cd
Changes directory. To go up a directory, its cd .. (two periods). cd with no parameters sends you to your home folder.
Using ~ will change you to your home directory and – will return to your previous working directory.
cd ..
cd my_folder
rm
Deletes files. Will delete folders recursively too with the -r option.
The -f option will forcefully remove files without warning
Common meme is telling people to rm -rf /, DON’T !
rm my_file.txt
rm -r my_folder
rm -rf my_folder #forcefully removes a directory and its contents
rm –no-preserve-root -rf / #This will recursively forcefully remove all files from your filesystem, don’t do it.
rmdir
Removes an empty directory (Note: Doesn’t work for non-empty directories)
rmdir test #Removes test directory
cp
Copies files and directories. Use -R for copying directories.
cp my_file.txt my_file_copy.txt
cp my_file.txt directory/my_file.txt
cp -R my_folder my_folder_copy
mv
Moves files and folders. Also the way to rename things in the command line.
mv my_file.txt this_subdirectory/my_file.txt
mv my_old_foldername my_new_foldername
pwd
Prints what folder you’re in. Sort of useful, but your shell should have the folder you’re in.
pwd
/home/username
!!
Re run the last command.
This can be combined with other commands.
pwd
/home/username/desktop/mydir
cd ..
/home/username/desktop
!!
/home/username
sudo
Runs a command as a different user, by default root. Does not work with cd.
mkdir folder
mkdir: cannot create directory ‘folder’: Permission denied
sudo mkdir folder
(You will be asked for a password and the permission issues should go away)
sudo !!
[sudo] password for user:
(this does the same thing as above)
mkdir
Makes a directory
mkdir folder
chmod
Changes file permissions (read, write, execute) + to add the permission to your user, – to remove.
chmod +x myprogram
./myprogram
chmod -x
chmod 777 filename.txt (Allow anyone to edit)
nano
The easiest command line text editor. No arguments opens it just like opening notepad.exe without opening with a file.
Type ctrl+o to save, and ctrl+x to exit.
nano mytextfile.txt
passwd
Change your password. (often has be run as root or be preceded by sudo )
passwd
Changing password for
(current) UNIX password:guest
Enter new UNIX password:hunter2
Retype new UNIX password:hunter2
cat
Concatenate two (or one) files or print everything in a file. Not recommended for viewing files, use less for that instead.
cat my_file.txt
head
Print the first 10 lines of a text file.
head my_file.txt
tail
Print the last n lines of a text file. Super useful for log files.
If it is invoked with the argument -f it lets you continuously view the file in real time.
tail -n 10 -f logfile.txt
less
Allows you to view a text file without editing it.
Can also view log files using +F (similar to tail -f)
less my_file.txt
less +F logfile.txt
grep
Allows you to use regular expressions to search through the output of a program or a file. Search for text in all files in a folder with the “-r” switch.
grep ‘Error:’ my_file.txt
grep -r `find me’ my_directory/
tar
Extract files from tar archives.
Common options:
tar -xzf : extract files from gzip compressed tar archive
tar -xjf : extract files from bzip2 compressed tar archive
tar -xzf system_backup_2016_04_07.tar.gz
Best way to remember tar -flags https://i.imgur.com/Vf0An8J.png
Modern versions are smart enough to detect the format, so you can use -xf or -cf (eXtract File, Create File)
touch
Create an empty file with the specified name if the file does not exist.
Will also update the file date of an existing file without modifying the content
touch foo.bar
top
Terminal-based GUI for viewing processes
top
ln
Used for creating links (shortcuts) in the filesystem.
In general always use -snf (trust me)
ln -snf /opt/foo /usr/bin/bar
/usr/bin/bar now links to /opt/foo, and it is transparent to the operating system.
screen
screen let’s you run multiple login sessions in the same terminal. Say you want to run a process, you can launch it through screen, detach it, and then later come back to the same process.
When inside a screen session, type CTRL+A, then CTRL+D to detach it (put it into the background).
When you later want to reattach to the screen session, type screen -r. If you only have one screen session, you will be brought right back, otherwise you will have to specify the session id
screen bash
screen -r
screen -ls
There are screens on:
767.ttys000.localhost (Detached)
844.ttys002.localhost (Detached)
2 Sockets in /var/some/folder/random/T/.screen.
screen -r 767.ttys000.localhost
whoami
Prints the currently logged in user.
whoami
john
whereis
Prints the location of a command.
whereis echo
/bin/echo
which
Prints the location of a command.
which echo
/usr/bin/echo
echo
Outputs text to the command line. Useful when writing shell scripts.
echo “hello world”
Hello world
kill
Attempt to terminate process
kill (process id)
*if ineffective, try kill -15 or kill -9 if 15 does not work (kill -9 will forcibly terminate almost any process)
killall
Will attempt to terminate a process.
killall firefox
file
Shows you the file type
file my_file.txt
my_file.txt: UTF-8 Unicode text
date
Show the current date in text form
date
Tue Apr 19 15:31:54 CDT 2016
ps
Display information about processes (different than top…)
ps -ef
can also search for processes:
ps -ef | grep firefox
apropos
Find commands that do a given task, Will return a list of commands that have the searched parameter in their man file.
Note: similar functionality to running “man -k command”.
apropos remove
Colrm (1) – remove colums from a file
Cut () – remove sections from each line of files
…
apropos concat
cat (1) – concatenate files and print on the standard output
cat (1p) – concatenate and print files
eval (1p) – construct command by concatenating arguments
…
alias
Very useful for creating custom shorcuts for commonly used programs or parameters
alias lcolor=’ls –color=auto’
Now lcolor is the same as ls –color=auto
But shorter
env
List environment variables / set environment variables
env
HOSTNAME=hostname.abcdefg.com
SHELL=/bin/bash
…etc
The filesystem in a nutshell
“Everything is a file”. In Linux, everything is treated as a file, even your devices.
Unlike Windows and DOS, Unix systems and Linux do not have drive names. Your thumb drive will be mounted as a “folder” on Linux – Instead of being F:\ on Windows, it would be /media//my_thumbdrive, assuming that is the device name on linux. On Ubuntu, drives are usually mounted in the /media folder under your username. All your shared folders in Virtualbox will show up there, but not under your username.
/home Where your user files are stored (equivalent of C:\Users)
/media Where drives are mounted (In Ubuntu)
/bin Where system executables are stored (almost the equivalent of system32)
/lib Where the system stores the library files (like the .dll’s in system32)
/usr/bin Where extra stuff you install is stored, like python (almost equivalent of C:\Program Files)
/etc Where configuration files for various programs and network services are stored
(Equivalent to where C:\Program Files\ApplicationName files are stored)
/dev Where all the systems devices can be found. Since everything is a file in Linux, every hardware device also has a corresponding file under /dev
/tmp Where temporary files are stored (This is wiped upon reboot, unless configured otherwise)
/var Variable files—files whose content is expected to continually change during normal operation of the system—such as logs, spool files, and temporary email files.
/proc Where you can query the hardware for information e.g. cat /proc/cpuinfo
Your home folder location can be referred to as ~. So instead of typing out /home/me, you can just type ~ , and it’ll resolve to be the same path.
And that’s about the extent you need to know, and probably more so to be honest. /home//bin is a good place to store any shell scripts or whatnot that you want to run from any folder. Which brings us to our next topic…
Running your own scripts
Earlier in the table of commands, I mentioned the chmod command. Files created by you will almost always be only read write permission level, which is great from a security standpoint. However, say you want to run a python script. You could do python myscript.py but that’s cumbersome. Instead, let’s do it the Linux Way™.
Header comment
The official name is the “shebang”, but in this guide I’ll call it the header comment – it’s a special type of comment you put at the top of a script in a Linux environment to tell the shell what program to execute your program with. For a python script, you do this:
#!/usr/bin/env python
If we were writing a perl script, we would do
#!/usr/bin/env perl
If you were to write a bash script we would do:
#!/usr/bin/env bash
What it’s telling the shell to do is to look in the environment settings of the linux install, figure out where python is installed, and then run the rest of the source code through the python executable. This method has the benefit of not only being extremely portable (python might not be installed in the same place on all linux systems), it’s also super easy to remember what to write each time if you deal with multiple programming languages.
Linux doesn’t care about file extensions
With the header comment written, we now don’t have to worry about putting .py at the end of the filename anymore. You could name it “mypythonscript.jpg” if you wanted to – the data inside it is the same, and Linux just looks for that header comment, so it really doesn’t matter. It’s super nice to have no extension though, especially if you run the script a lot. You’ll just have to have your python scripts organized in a separate folder if you’re going to start foregoing extensions – hard to tell file types apart with no extension. The ‘file’ command can be used to identify which type of file a file is if you prefer to have no extensions.
Running your script
The first thing you have to do is mark it as executable. To do so, we’re going to use our trusty chmod command. We want to mark it as executable, so we’re going to use the u+x argument to add executable permissions to the file – This tells Linux to change the file mode to executible by the user only.
chmod u+x mypythonscript.py
Now, our script is executable. You can verify this by running ls, and it will now be green. Green means an executable file. To run it, we’re going to do
./mypythonscript.py
This is a security measure put in place by Linux so that you can be sure you are executing the file within the current directory – imagine if someone placed a malicious executable named ls in a folder, and you ran ls, and instead of executing the one in /bin, it ran the malicious one in your current folder? It wouldn’t be good.
In Linux, a single period is your current directory, and two periods is the parent directory. So
./mypythonscript.py
is really
/home/my_username/my/full/path/name/mypythonscript.py
If you have a script you do want to run from anywhere, put it in your bin folder in your home folder (~/bin). If you want to make that script or executable available for all users on the system, place it in /usr/local/bin.
Cancelling a process in terminal
To stop a process, hit CTRL+D. This will exit out of the current program (if you’re in an interactive python process, or if there’s a program you want to force quit in general, just hit CTRL+D) If that doesn’t work, use CTRL+C, but that isn’t a nice way of stopping a program, and its not recommended. Use as a last resort.
Previously, it was mentioned you can run multiple commands by either separating them with “;” or “&&”. The difference is that if you separate with a semicolon, you will have to force stop each command; For example if I run the following:
./my_script.py; cp script_output backup/script_output; ./my_script2.py
If you force quit the instance of my_script.py, it will continue on to copy the script_output file, which you’d have to force quit as well, as well as the my_script2.py. A nice feature is that if my_script.py has an error and fails, it will not continue on to run the copy command or the second python script.
Alternatively, if you use && to run multiple commands:
./my_script.py && cp script_output backup/script_output && ./my_script2.py
Now, when you cancel any one of these processes, the remaining processes will also be cancelled. However, if my_script.py encounters an error, it will continue to run the second and third commands which may not be good – If cp ends up running the computer out of disk space and the second script generates more data, this could end badly. Choose the method of running multiple commands wisely. Personally I like using the && method better simply because I can force quit all of it.
To paste into the terminal, you must use CTRL+SHIFT+V. CTRL+V will not work to paste. Similarly, copying in the terminal must be CTRL+SHIFT+C.
Be careful when pasting commands into the terminal! If there’s a new line at the end of the command, it’ll automatically run the command!! (like if you hit enter after typing a command).
Multiline commands can be separated with \
Example:
cd \
/var/log
Is the same As “cd /var/log” only in two lines, useful for long commands.
Package manager
No linux tutorial would be complete without an introduction to the package manager. This is a unique feature of linux – it allows you to install, update, and remove any piece of software on your computer. The syntax is very simple too. All package management must be run as root, or with “sudo” before it.
Apt – Advanced Package Tool
Command
What it does
Example
sudo apt update
Updates the list of available software to install (if a security update for python got released yesterday, running update will let your computer know that)
sudo apt upgrade
Updates the installed software on your computer (applying that python security patch that it found out about through update)
sudo apt install
Allows you to install a package
sudo apt install python3
sudo apt remove
Allows you to remove a package
sudo apt remove vim
aptitude
Launches the synaptics package manager – good for searching for packages. Can be run without root, but you can’t install anything without running as root.
Q to quit.
sudo aptitude
Older tutorials will use apt-get, which has since been superseded by Apt. Apt-get will still work fine, but apt is the new standard, and it has some nice improvements.
To easily search for packages in the repository, you can use Synaptic Package Manager, or you can
just google “how to install x on ubuntu” and you’ll find the package name. You can try guessing a package name, but you might not always be successful.
Pip is also directly accessible through the command line for python. Just run
sudo -H pip install
Python virtual environment
On your own machine, you have full root access, but on a work machine you most definitely will not. Having a python virtual environment allows you to install as many python packages with pip as you want, without needing to run sudo pip install .
A good rule of thumb is to create a virtual environment for every project, separately, to separate dependencies from one environment to another.
An easy way to create virtual environments is to globally install a pip package called virtualenvwrapper.
To set up a virtual environment of python, create a directory in your home folder that you want the virtual environment to live in. Then run the following commands to create the virtual environment.
sudo pip install virtualenv
virtualenv pythonv
To make the python virtual environment your default python environment in your session, simply run
source /path/to/pythonv/bin/activate
Now when you run which python it will output /path/to/pythonv/bin/python.
Note that this is a temporary change – Closing the terminal or logging out will revert this change. Adding this to your ~/.bashrc file will execute it upon login, or you can set it as an alias in your bashrc.
Shell piping and redirecting IO
There are three types of pipe characters: <, >, and |. < is difficult to explain, so I left it out. Pipe/Redirect character What it does Examples >
Redirects output to a file
man -k search > man_output.txt
>>
Redirects output to file and appends
man -k find >> man_output.txt
|
Takes the output of process A and puts it as input into process B
cat man_output.txt | grep “fast”
&>
Redirect both standard output and error to same location.
cat file.txt &> output.txt
In Linux, you have two main pipes – Standard Output (stdout, all your normal output of things that are printed to the screen), and Standard Error (stderr, when something doesn’t work right – Error on line 2, ‘;’ expected)
stdout is numbered pipe 1, and stderr is numbered pipe 2. By default, > is actually 1> – It’s redirecting standard output. If we wanted to redirect standard error, we would do 2>. 2>> will append, just like >> does. If we want to redirect both error and output, we can do &>.
Copyleft 2016 /u/mcvittal. Licensed under the WTFPL – Do What The Fuck You Want To Public License.