Now that you have completed Assignment 1 and have a working sandbox Linux system installed, it’s time to cover the basics.
First, when I started using Linux, I got a boxed set of RedHat that included a nice cheat sheet sticker to go on a keyboard wrist rest. In lieu of that, I just saw this posted today. I recommend you keep it bookmarked and perhaps printed out for reference: Linux Command Shelf Cheat-Sheet. There’s a PDF for download on that site. I also recommend you somehow (RSS or Facebook) subscribe to NixCraft. I still regularly learn useful things from that blog.
Now, when taking care of a Linux (or other *NIX) system, you’re often going to need to download and extract file archives. Typically these are either singly-zipped files (
.bz2 from gzip and bzip2, respectively), or so-called “tarballs” (
.tar.bz2, etc). Tarballs are created with the
tar program, whose name comes from Tape ARchive. The additional suffixes indicate that the tarball has been compressed in some way. In other words, a simple
.tar file contains a bunch of files all rolled up into a single archive, but not compressed, while
.tgz for short) files have additionally been zipped up in some fashion. Incidentally,
tar is also still used to write files to tape, if that’s your thing.
The best way to get a file for download from the command-line (which I will focus on almost entirely in this series) is
wget. So, open up a terminal window. You should have a simple prompt waiting for your input. Check to see whether you have
wget by typing
which command searches through all of the directories on your
PATH, an environment variable that tells the shell where to find programs, for the command you list, in this case
wget. It’s a good way to check for the presence of a program on a system. If that doesn’t work, but you think it’s around somewhere, you can sometimes use
locate, but we’ll not cover that today.
Now, if you have
wget, you should get in return the full path to the program, something, like
/usr/bin/wget. If so, you can proceed to the next step where it says ‘Downloading a file with wget’.
If not, the utility will print nothing. I tried a made-up program called ‘snarfblat’:
brock@gamont:~$ which snarfblat brock@gamont:~$
Here’s a successful
which call for
brock@gamont:~$ which wget /usr/bin/wget brock@gamont:~$
If you don’t have it, try
sudo apt-get install wget on a Debian-based distribution or
sudo yum install wget on a RedHat-based distribution. If you’re not set up with
sudo, we will cover that later. For now become root with
su - followed by the root password, and then run the above commands without the
sudo at the beginning. Type
exit or press Ctrl-D to exit the root shell once you have installed it. Double-check that it’s now installed by calling
which wget again. If you get stuck, please email me or comment here so we can take care of it and you can continue.
Downloading a file with wget
For this tutorial, I’m going to have you download a sample tarball that I have created. It contains a bunch of directories and files with various names and contents. We will use it in this assignment and later assignments to practice command-line basics.
For now, go to a terminal (I recommend you make a directory for this like
mkdir ~/sywtbals, where
~/ is a shortcut for specifying your home directory. In my case that is
/home/brock. Then change to that directory:
wget to pull down the file.
It should look like this:
brock@water:~/sywtbals$ wget http://blog.brocktice.com/sywtbals/1.tgz --2013-05-05 17:30:33-- http://blog.brocktice.com/sywtbals/1.tgz Resolving blog.brocktice.com... 188.8.131.52 Connecting to blog.brocktice.com|184.108.40.206|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 215040 (210K) [application/x-gzip] Saving to: “1.tgz” 100%[======================================>] 215,040 259K/s in 0.8s 2013-05-05 17:30:34 (259 KB/s) - “1.tgz” saved [215040/215040] brock@water:~/sywtbals$
Give yourself a self-high-five if you noticed I changed machines in the middle of this post from gamont to water.
Now that you’ve got the file, let’s take a look at it with
ls, the file-listing tool:
brock@water:~/sywtbals$ ls -l total 212 -rw-r--r-- 1 brock brock 215040 May 5 13:40 1.tgz brock@water:~/sywtbals
I used the
-l for the ‘long’ listing that shows details about each file. This shows us a few interesting things. One is the permissions:
-rw-r--r-- brock brock
This tells us the permissions for the user, group, and everyone else. A file may only have one group and one user assigned to it. There’s SELinux (Security-Enhanced Linux), the NSA’s enhancements to make Linux permissions more fine-grained, but honestly I don’t understand SELinux well enough to do more than turn it off, so I’m not going to discuss it here.
So, in this case the user has read and write permissions, the group has read permissions, and everyone else also has read permissions. The user is brock and the group is brock. Note that even though the user and group names are the same they are totally separate. One represents a userid, and the other represents a groupid. Observe the use of the
id command to elaborate:
brock@water:~/sywtbals$ id uid=1000(brock) gid=1000(brock) groups=1000(brock),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),100(users),108(netdev),109(bluetooth),112(fuse),115(scanner),120(libvirt),1001(family) brock@water:~/sywtbals$
Here you can see that (and again, this is just a coincidence because I was the first user added to the system), the user id
brock is 1000, and the group id
brock is also 1000. However, the group id
brock could just as easily be 2005 or something.
Let’s say this file is somewhat secret and I don’t want anyone else (except
root of course) on the system to be able to read it. We’ll change it like this:
brock@water:~/sywtbals$ chmod go= 1.tgz brock@water:~/sywtbals$ ls -l total 212 -rw------- 1 brock brock 215040 May 5 13:40 1.tgz
What I did is to tell
chmod to set the group and other permissions to be equal to nothing. As a result, only the user has any permissions at all.
On the other hand, what if we want to allow anyone on the system to read and write the file?
brock@water:~/sywtbals$ chmod go+rw 1.tgz brock@water:~/sywtbals$ ls -l total 212 -rw-rw-rw- 1 brock brock 215040 May 5 13:40 1.tgz brock@water:~/sywtbals$
You can also use numerical codes to set permissions. All I ever remember about that offhand is that 0 means nothing and 7 is everything. I prefer the other notation, but you can look it up. A common 4-letter admonition you’ll see as a Linux sysadmin is RTFM — read the fucking manual.
Usually you can find the so-called
manfile for a program or configuration file like this:
This will show the output in the system pager, a program that pages through a text file. Historically this was normally a program called, sensibly,
more. Later, an improved program called
less replaced it. UNIX humor — less is more. Anyway, there are only four things you need to know to get around
less reasonably well. The q key quits. The spacebar moves to the next page. And the / key starts a search. The Esc key gets you back to the main mode. There’s a lot more you can do with less, but those four things will make up 99% of your
The most important thing you can do with the / search in many manpages as a beginner is to find the Examples section, if there is one. It will usually contain examples of what you’re trying to do. Of course you can also RTFM. Here’s me searching for the examples in the
chmod manpage. Note the
/examples at the end.
CHMOD(1) User Commands CHMOD(1) NAME chmod - change file mode bits SYNOPSIS chmod [OPTION]... MODE[,MODE]... FILE... chmod [OPTION]... OCTAL-MODE FILE... chmod [OPTION]... --reference=RFILE FILE... DESCRIPTION This manual page documents the GNU version of chmod. chmod changes the file mode bits of each given file according to mode, which can be either a symbolic representation of changes to make, or an octal number representing the bit pattern for the new mode bits. The format of a symbolic mode is [ugoa...][[+-=][perms...]...], where perms is either zero or more letters from the set rwxXst, or a single letter from the set ugo. Multiple symbolic modes can be given, sepa‐ rated by commas. A combination of the letters ugoa controls which users' access to the file will be changed: the user who owns it (u), other users in the /examples
Unfortunately, when I hit the Enter key after typing
/examples I was greeted with
Pattern not found (press RETURN). So I’ll have to properly RTFM. I press the spacebar to scroll down. (Arrow keys and page up/down can also be used to navigate.) Eventually I find this documentation on the octal numerical codes:
granted to users that are in neither of the two preceding categories (o). A numeric mode is from one to four octal digits (0-7), derived by adding up the bits with values 4, 2, and 1. Omitted digits are assumed to be leading zeros. The first digit selects the set user ID (4) and set group ID (2) and restricted deletion or sticky (1) attributes. The second digit selects permissions for the user who owns the file: read (4), write (2), and execute (1); the third selects permissions for other users in the file's group, with the same values; and the fourth for other users not in the file's group, with the same values. chmod never changes the permissions of symbolic links; the chmod system call cannot change their permissions. This is not a problem since the permissions of symbolic links are never used. However, for each sym‐ bolic link listed on the command line, chmod changes the permissions of the pointed-to file. In contrast, chmod ignores symbolic links encoun‐ tered during recursive directory traversals. SETUID AND SETGID BITS chmod clears the set-group-ID bit of a regular file if the file's group ID does not match the user's effective group ID or one of the user's supplementary group IDs, unless the user has appropriate privileges. Manual page chmod(1) line 44
There you go, the numerical codes are read = 4, write = 2, and execute = 1. Execute means that if it's a script or a program, it can be run, or if it's a directory, you can navigate (change directory or
cd) into it. Anyway, to set read/write for user and group, and nothing for everyone else, you'd use (4+2=6)(4+2=6)(0) or
So, now that we've beaten the permissions thing to death. Just a few other notes. In this output:
-rw------- 1 brock brock 215040 May 5 13:40 1.tgz
The last few bits are the file size (215040 bytes) and file modification time, ending with the filename. You can get a handy "human-readable" size using the
-h for human flag like so:
brock@water:~/sywtbals$ ls -lh 1.tgz -rw-rw-rw- 1 brock brock 210K May 5 13:40 1.tgz brock@water:~/sywtbals$
I usually use the
-h when doing a detailed listing. Because it rounds, however, sometimes you may want to omit the
-h to see if a file size matches, has changed, etc.
OK! Let's extract this sucker. We'll use a
tar command to extract. First, let's get a preview of the file contents. Nice people generally put everything to tar in a directory first, so that the directory extracts nicely and doesn't make a mess. Sometimes, (and occasionally even with good reason), people will put all the contents of the tar right at the top level, and extracting without knowing that will make a big mess. (I'll cover a quick tip to fix that when we get to pipes. Don't let me forget.).
To get a preview of the file contents, we use the
t flag to
brock@water:~/sywtbals$ tar tvf 1.tgz drwxrwxr-x brock/brock 0 2013-05-05 13:31 1/ drwxrwxr-x brock/brock 0 2013-05-05 13:30 1/sequential/ -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_85 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_34 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_60 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_43 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_30 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_64 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_11 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential/sequential_file_77 ... -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00093 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00049 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00013 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00068 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00018 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00055 -rw-rw-r-- brock/brock 12 2013-05-05 13:31 1/sequential_padded/sequential_file_00005 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00098 -rw-rw-r-- brock/brock 12 2013-05-05 13:31 1/sequential_padded/sequential_file_00001 -rw-rw-r-- brock/brock 13 2013-05-05 13:31 1/sequential_padded/sequential_file_00026 brock@water:~/sywtbals$
tar tfv 1.tgz can be broken down as follows:
tar is the program,
t tells it to test the file,
f tells it it's getting a filename for input (
1.tgz in this case), and
v means verbose, in that it prints out all the names of the files being extracted. You'll always need f to extract a file. The philosophies on v vary. Some people say using it causes errors to get lost in the shuffle and go unnoticed. Others like to see which files are being extracted. If you do a run like this with t first, then you can go ahead and do the actual extraction later without v and see any errors.
I'm a nice guy, so I put the contents in a directory called
1. (I'm also an idiot, this is Assignment #2, but I digress). So let's go ahead and extract the file.
brock@water:~/sywtbals$ tar xf 1.tgz brock@water:~/sywtbals$
In this case I omitted the v option, and changed from test mode to extract mode.
tar followed the UNIX convention of outputting no message at all if everything worked correctly.
Do a quick listing on the created directory just to double-check:
brock@water:~/sywtbals$ ls -l 1 total 8 drwxr-xr-x 2 brock brock 4096 May 5 13:30 sequential drwxr-xr-x 2 brock brock 4096 May 5 13:31 sequential_padded brock@water:~/sywtbals$
Looks good! Questions?
I have a bonus question.
tar figured out something about our
1.tgz file and took care of it without us asking. Do you know what that is?