Since uniq uniq works only on adjacent lines of its input, it is most often used in conjunction with works only on adjacent lines of its input, it is most often used in conjunction with sort sort.
Frequently used options -d Print only nonunique (repeating) lines.
-u Print only unique (nonrepeating) lines.
Examples Suppose file file contains the following: contains the following: b b a a c d c Issuing the command uniq uniq with no options: with no options: $uniqfile yields the following output: b a c d c Notice that the line with c c is repeated, since the duplicate lines were not adjacent in the input file. To eliminate duplicate lines regardless of where they appear in the input, use is repeated, since the duplicate lines were not adjacent in the input file. To eliminate duplicate lines regardless of where they appear in the input, use sort sort on the input first: on the input first: $sortfileuniq a b c d To print only lines that never repeat in the input, use the -u -u option: option: $sortfileuniq-u d To print only lines that do do repeat in the input, use the repeat in the input, use the -d -d option: option: $sortfileuniq-d a b c
Name wc Syntax wc[options][files]
Description Print counts of characters, words, and lines for files files. When multiple files are listed, statistics for each file output on a separate line with a c.u.mulative total output last.
Frequently used options -c Print the character count only.
-l Print the line count only.
-w Print the word count only.
Example 1 Show all counts and totals for file1 file1, file2 file2, and file3 file3: $wcfile[123]
Example 2 Count the number of lines in file1 file1: $wc-lfile1
Objective 3: Perform Basic File Management This section covers basic file and directory management, including filesystems, files and directories, standard file management commands, their recursive capabilities (where applicable), and wildcard patterns (also known as file globbing).
Filesystem Objects Nearly every operating system in history structures its collection of stored objects in a hierarchy hierarchy, which is a tree of objects containing other objects. This hierarchy allows a sane organization of objects and allows identically named objects to appear in multiple locations, an essential feature for multiuser systems such as Linux. Information about each object in the filesystem is stored in a table (which itself is part of the filesystem), and each object is numbered uniquely within that table. Although there are a few special object types on Linux systems, the two most common are directories directories and and files files.
Directories and files A directory is a container intended to hold objects such as files and other directories. A directory"s purpose is primarily for organization. A file, on the other hand, exists within the directory, and its purpose is to store raw data. At the top of all Linux filesystem hierarchies is a directory depicted simply by / /; this is known as the root root directory. Beneath directory. Beneath / / are named directories and files in an organized and well-defined tree. To describe these objects, you simply refer to them by name separated by the are named directories and files in an organized and well-defined tree. To describe these objects, you simply refer to them by name separated by the / / character. For example, the object character. For example, the object ls ls is an executable program stored in a directory called is an executable program stored in a directory called /bin /bin under the root directory; it is depicted simply as under the root directory; it is depicted simply as /bin/ls /bin/ls.
NoteDon"t confuse root root directory with the username directory with the username root root, which is separate and distinct. There"s also often a directory named /root /root for the root user. Keeping for the root user. Keeping / /, /root /root, and the root root user straight in a conversation can be a challenge. user straight in a conversation can be a challenge.
Inodes The identification information for a filesystem object is known as its inode inode. Inodes carry information about objects, such as where they are located on disk, their modification time, security settings, and so forth. Each Linux ext3 ext3 filesystem is created with a finite number of inodes that is calculated based on the size of the filesystem and other options that are given to filesystem is created with a finite number of inodes that is calculated based on the size of the filesystem and other options that are given to mke2fs mke2fs (the command used to create an ext2 or ext3 filesystem on a part.i.tion). Multiple objects in the filesystem can share the same inode; this concept is called (the command used to create an ext2 or ext3 filesystem on a part.i.tion). Multiple objects in the filesystem can share the same inode; this concept is called linking linking.
File and directory management commands Once a hierarchy is defined, there is a never-ending need to manage the objects in the filesystem. Objects are constantly created, read, modified, copied, moved, and deleted, so wisely managing the filesystem is one of the most important tasks of a system administrator. In this section, we discuss the basic command-line utilities used for file and directory management. There are GUI tools for this task, but the LPI Level 1 exams only test on command-line tools, and although GUI tools are sometimes more intuitive, a good system administrator should always be always be able to administer his or her system from the command line.
File-Naming Wildcards (File Globbing) When working with files on the command line, you"ll often run into situations in which you need to perform operations on many files at once. For example, if you are developing a C program, you may want to touch touch all of your all of your .c .c files in order to be sure to recompile them the next time you issue the files in order to be sure to recompile them the next time you issue the make make utility to build your program. There will also be times when you need to move or delete all the files in a directory or at least a selected group of files. At other times, filenames may be long or difficult to type, and you"ll want to find an abbreviated alternative to typing the filenames for each command you issue (see utility to build your program. There will also be times when you need to move or delete all the files in a directory or at least a selected group of files. At other times, filenames may be long or difficult to type, and you"ll want to find an abbreviated alternative to typing the filenames for each command you issue (see Table6-5 Table6-5).
To make these operations simpler, all sh.e.l.ls on Linux offer file-naming wildcards file-naming wildcards.
NoteWildcards are expanded by the sh.e.l.l, not by commands. When a command is entered with wildcards included, the sh.e.l.l first expands all the wildcards (and other types of expansion) and pa.s.ses the full result on to the command. This process is invisible to you.
Rather than explicitly specifying every file or typing long filenames, you can use wildcard characters wildcard characters in place of portions of the filenames, and the sh.e.l.l can usually do the work for you. For example, the sh.e.l.l expands in place of portions of the filenames, and the sh.e.l.l can usually do the work for you. For example, the sh.e.l.l expands *.txt *.txt to a list of all the files that end in to a list of all the files that end in .txt .txt. File wildcard constructs like this are called file globs file globs, and their use is awkwardly called globbing globbing. Using file globs to specify multiple files is certainly a convenience, and in many cases is required to get anything useful accomplished. Wildcards for sh.e.l.l globbing are listed in Table6-5 Table6-5.
Table6-5.Common file-naming wildcards
Wildcard Description *
Commonly thought to "match anything," it actually will match zero or more characters (which includes "nothing"!). For example, x* matches files or directories matches files or directories x x, xy xy, xyz xyz, x.txt x.txt, xy.txt xy.txt, xyz.c xyz.c, and so on.
Match exactly one character. For example, x? matches files or directories matches files or directories xx, xy, xz xx, xy, xz, but not x x and not and not xyz xyz. The specification x?? x?? matches matches xyz xyz, but not x x and and xy xy.
[characters]
Match any single character from among characters listed between the brackets. For example, listed between the brackets. For example, x[yz] x[yz] matches matches xy xy and and xz xz.
[!characters]
Match any single character other than characters listed between the brackets. For example, listed between the brackets. For example, x[!yz] x[!yz] matches matches xa xa and and x1 x1 but does not match but does not match xy xy or or xz xz.
[a-z]
Match any single character from among the range of characters listed between the brackets and indicated by the dash (the dash character is not matched). For example, x[0-9] matches matches x0 x0 and and x1 x1, but does not match xx xx. Note that to match both upper- and lowercase letters (Linux filenames are case-sensitive), you specify [a-zA-Z] [a-zA-Z]. Using x[a-zA-Z] x[a-zA-Z] matches matches xa xa and and xA xA.
[!a-z]
Match any single character from among the characters not in the range listed between the brackets.
{frag1,frag2,frag3,...} Create strings frag1 frag1, frag2 frag2, frag3 frag3, etc. For example, file_{one,two,three} file_{one,two,three} yields the strings yields the strings file_one file_one, file_two file_two, and file_three file_three. This is a special operator named brace expansion brace expansion that can be used to match filenames but isn"t specifically a file wildcard operator and does not examine directories for existing files to match. Instead, it will expand that can be used to match filenames but isn"t specifically a file wildcard operator and does not examine directories for existing files to match. Instead, it will expand any string any string.
For example, it can be used with echo echo to yield strings totally unrelated to existing filenames: to yield strings totally unrelated to existing filenames: $echostring_{a,b,c} string_astring_bstring_c
A few examples of the useful things you can do with wildcards follow: If you remember part of a filename but not the whole thing, use wildcards with the portion you remember to help find the file. For example, if you"re working in a directory with a large number of files and you know you"re looking for a file named for Linux, you may enter a command like this:$ls-l*inux*
When working with groups of related files, wildcards can be used to help separate the groups. For example, suppose you have a directory full of scripts you"ve written. Some are Perl scripts, for which you"ve used an extension of .pl .pl, and some are Python, which have a .py .py extension. You may wish to separate them into new, separate directories for the two languages like this: extension. You may wish to separate them into new, separate directories for the two languages like this:$mkdirperlpython $mv*.plperl $mv*.pypython Wildcards match directory names as well. Suppose you have a tree of directories starting with contracting contracting, where you"ve created a directory for each month (that is, contracting/january contracting/january, contracting/february contracting/february, through contracting/december contracting/december). In each of these directories are stored invoices, named simply invoice_ invoice_custa_01.txt, invoice_ invoice_custa_02.txt, invoice_custb_01.txt invoice_custb_01.txt, and so on, where custa custa and and custb custb are customer names of some form. To display all of the invoices, wildcards can be used: are customer names of some form. To display all of the invoices, wildcards can be used:$lscon*/*/inv*.txt The con* con* matches matches contracting contracting. The second * matches all directories under the contracting contracting directory ( directory (january through through december december). The last * matches all the customers and each invoice number for each customer.
See the bash bash manpages or info page for additional information on how manpages or info page for additional information on how bash bash handles expansions and on other expansion forms. handles expansions and on other expansion forms.
Name bzip2 Syntax bzip2[options][filenames...]
bunzip2[options][filenames...]
Description Compress or uncompress files using the Burrows-Wheeler block sorting text compression algorithm and Huffman coding. bzip2 bzip2 is generally considered one of the most efficient compression programs available for Linux systems. Files compressed with is generally considered one of the most efficient compression programs available for Linux systems. Files compressed with bzip2 bzip2 usually have the extension usually have the extension .bz2 .bz2.
Frequently used options -d Decompress a file. bzip2 d bzip2 d is the same as is the same as bunzip2 bunzip2.
-1 to to -9 -9 Set the block size to 100k, 200k, 300k...900k when compressing. This essentially means that -1 -1 compresses faster but leaves larger compressed files, whereas compresses faster but leaves larger compressed files, whereas -9 -9 compresses more slowly but results in smaller files. compresses more slowly but results in smaller files.
Example 1 Compress the file /etc/largefile /etc/largefile using the highest level of compression. It will be compressed and renamed using the highest level of compression. It will be compressed and renamed /etc/largefile.bz2 /etc/largefile.bz2: $bzip2-9/etc/largefile Example 2 Uncompress /etc/largefile.bz2 /etc/largefile.bz2. It will be uncompressed and renamed /etc/largefile /etc/largefile: $bunzip2/etc/largefile.bz2 or: $bzip2-d/etc/largefile.bz2
Name cp Syntax cp[options]file1file2 cp[options]filesdirectory Description In the first command form, copy file1 file1 to to file2 file2. If file2 file2 exists and you have appropriate privileges, it will be overwritten without warning (unless you use the exists and you have appropriate privileges, it will be overwritten without warning (unless you use the -i -i option). Both option). Both file1 file1 and and file2 file2 can be any valid filename, either fully qualified or in the local directory. In the second command form, copy can be any valid filename, either fully qualified or in the local directory. In the second command form, copy files files to to directory directory. Note that the presence of multiple files implies that you wish to copy the files to a directory. If directory directory doesn"t exist, an error message will be printed. This command form can get you in trouble if you attempt to copy a single file into a directory that doesn"t exist, as the command will be interpreted as the first form and you"ll end up with doesn"t exist, an error message will be printed. This command form can get you in trouble if you attempt to copy a single file into a directory that doesn"t exist, as the command will be interpreted as the first form and you"ll end up with file2 file2 instead of instead of directory directory.
Frequently used options -f Force an overwrite of existing files in the destination.
-i Prompt interactively interactively before overwriting destination files. It is common practice (and advised) to alias the before overwriting destination files. It is common practice (and advised) to alias the cp cp command to command to cp -i cp -i to prevent accidental overwrites. You may find that this is already done for you for the to prevent accidental overwrites. You may find that this is already done for you for the root root user on your Linux system. user on your Linux system.
-p Preserve all information, including owner, group, permissions, and timestamps. Without this option, the copied file or files will have the present date and time, default permissions, owner, and group. all information, including owner, group, permissions, and timestamps. Without this option, the copied file or files will have the present date and time, default permissions, owner, and group.
-r, -R -R Recursively copy directories. You may use either upper- or lowercase for this option. If copy directories. You may use either upper- or lowercase for this option. If file1 file1 is actually a directory instead of a file and the recursive option is specified, is actually a directory instead of a file and the recursive option is specified, file2 file2 will be a copy of the entire hierarchy under directory will be a copy of the entire hierarchy under directory file1 file1.
-v Display the name of each file verbosely before copying.
Example 1 Copy the messages file to the local directory (specified by . .): $cp/var/log/messages.
Example 2 Make an identical copy, including preservation of file attributes, of directory src src in new directory in new directory src2 src2: $cp-Rpsrcsrc2 Copy file1, file2, file5, file6 file1, file2, file5, file6, and file7 file7 from the local directory into your home directory (in from the local directory into your home directory (in bash bash): $cpfile1file2file[567]~On the ExamBe sure to know the difference between a file destination and a directory destination and how to force an overwrite of existing objects.
Name cpio Syntax cpioo[options][archive]
cpioi<>
cpiop[destination-directory]<>
Description cpio is used to create and extract archives, or copy files from one place to another. No compression is done natively on these archives; you must employ is used to create and extract archives, or copy files from one place to another. No compression is done natively on these archives; you must employ gzip gzip or or bzip2 bzip2 if you desire compression. if you desire compression.
Frequently used options -o Copy-out mode. This mode is used to create an archive.
-i Copy-in mode. This mode is used to copy files out of an archive.
-p Copy-pa.s.s mode. Don"t create an archive; just copy files from one directory tree to another.
Example 1 Create an archive that contains all the files in the current working directory: $lscpioov>/tmp/archive.cpio Notice that instead of pa.s.sing files to archive to cpio cpio on the command line, we had the on the command line, we had the ls ls command create a list of files for us, which we then send to the command create a list of files for us, which we then send to the cpio cpio command via standard input using the command via standard input using the(vertical bar) character. (vertical bar) character.
Example 2 Extract all the files from the archive we just created: $cpioiv
Name dd Syntax dd[options]
Description dd converts and copies files. It is one of the few commands in the Linux world that can operate directly on block devices, rather than requiring access through the filesystem layer. This is especially useful when performing backups of block devices, such as hard drive part.i.tions, CD-ROMs, or floppy disks. converts and copies files. It is one of the few commands in the Linux world that can operate directly on block devices, rather than requiring access through the filesystem layer. This is especially useful when performing backups of block devices, such as hard drive part.i.tions, CD-ROMs, or floppy disks.
Frequently used options -if=file Read from file file instead of standard input. instead of standard input.
-of=file Output to file file instead of standard output. instead of standard output.
-ibs=n Read n n bytes at a time. bytes at a time.
-obs=n Write n n bytes at a time. bytes at a time.
-conv=list Perform the conversions defined in list list.
Example 1 Create an image of the compact disc currently in the default CD drive (/dev/cdrom): $ddif=/dev/cdromof=/tmp/cd.img Example 2 Copy /tmp/file /tmp/file to to /tmp/file2 /tmp/file2, converting all characters to lowercase: $ddif=/tmp/fileof=/tmp/file2conv=lcase
Name file Syntax file[options][file]
Description file is designed to determine the kind of file being queried. Because Linux (and other Unix-like systems) don"t require filename extensions to determine the type of a file, the is designed to determine the kind of file being queried. Because Linux (and other Unix-like systems) don"t require filename extensions to determine the type of a file, the file file command is useful when you"re unsure what kind of file you"re dealing with. command is useful when you"re unsure what kind of file you"re dealing with. file file accomplishes this by performing three sets of tests on the file in question: filesystem tests, magic tests, and language tests. Filesystem tests involved examining the output of the "stat" system call. Magic tests are used to check for files with data in particular fixed formats. If neither of these tests results in a conclusive answer, a language test is performed to determine whether the file is some sort of text file. accomplishes this by performing three sets of tests on the file in question: filesystem tests, magic tests, and language tests. Filesystem tests involved examining the output of the "stat" system call. Magic tests are used to check for files with data in particular fixed formats. If neither of these tests results in a conclusive answer, a language test is performed to determine whether the file is some sort of text file.
Frequently used options -f namefile namefile Read the names of the files to be examined from namefile namefile (one per line) before the argument list. (one per line) before the argument list.
-z Try to look inside compressed files.
Example 1 Determine the file type of the currently running kernel: $file/boot/vmlinuz-2.6.27.29-170.2.78.fc10.i686 /boot/vmlinuz-2.6.27.29-170.2.78.fc10.i686:Linuxkernelx86bootexecutable RO-rootFS,root_ 0x902,swap_dev0x2,NormalVGA Example 2 Determine the file type of /etc/pa.s.swd /etc/pa.s.swd: $file/etc/pa.s.swd /etc/pa.s.swd:ASCIItext
Name find Syntax find[options][path...][expression]
Description find searches recursively through directory trees for files or directories that match certain characteristics. searches recursively through directory trees for files or directories that match certain characteristics. find find can then either print the file or directory that matches or perform other operations on the matches. can then either print the file or directory that matches or perform other operations on the matches.
Frequently used options -mount Do not recursively descend through directories on mounted filesystems. This prevents find find from doing a potentially very long search over an NFS-mounted share, for example. from doing a potentially very long search over an NFS-mounted share, for example.
-maxdepth X X Descend at most X X levels of directories below the command-line arguments. levels of directories below the command-line arguments. maxdepth 0 maxdepth 0 eliminates all recursion into subdirectories. eliminates all recursion into subdirectories.
Example 1 Find all files in /tmp /tmp that end in that end in .c .c and print them to standard out: and print them to standard out: $find/tmpname"*.c"
The expression "*.c" "*.c" means "all files that end in .c". This is an example of means "all files that end in .c". This is an example of file globbing file globbing and is explained in detail later in this Objective. and is explained in detail later in this Objective.
Example 2 Find files (and only files) in /tmp /tmp older than seven days. Do not recurse into subdirectories of older than seven days. Do not recurse into subdirectories of /tmp /tmp: $find/tmp-maxdepth1-typef-daystart-ctime+7 Example 3 Find files in /usr /usr that have the that have the setuid setuid permission bit set (mode 4000): permission bit set (mode 4000): $find/usr-perm-4000
Name gzip and gunzip Syntax gzip[options][filenames...]
gunzip[options][filenames...]
Description Compress or uncompress files using Lempel-Ziv coding. gzip gzip is one of the most common compression formats found on Linux systems, although it is starting to be replaced by the more efficient is one of the most common compression formats found on Linux systems, although it is starting to be replaced by the more efficient bzip2 bzip2. Files compressed with gzip gzip usually have the extension usually have the extension .gz .gz. Command-line options for gzip gzip are very similar to those for are very similar to those for bzip2 bzip2.
Frequently used options -d Decompress a file. gzip -d gzip -d is the same as is the same as gunzip gunzip.
-r Travel the directory structure recursively. If any of the filenames specified on the command line are directories, gzip gzip will descend into the directory and compress all the files it finds there (or decompress them in the case of will descend into the directory and compress all the files it finds there (or decompress them in the case of gunzip gunzip).
Example 1 Compress the file /etc/largefile /etc/largefile. It will be compressed and renamed /etc/largefile.gz /etc/largefile.gz: $gzip/etc/largefile Example 2 Uncompress /etc/largefile.gz /etc/largefile.gz. It will be uncompressed and renamed /etc/largefile /etc/largefile: $gunzip/etc/largefile.gz or: $gzip-d/etc/largefile.gz