The usage of the cat command is quite simple. For instance, if we had a file called cat_test.txt with the following contents:
here is a cat test
I am adding a tab on the next line
there is a tab here
throw some blank lines in here
See how that goes
Issuing:
cat cat_test.txt
Will output the contents of the file to standard out:
here is a cat test
I am adding a tab on the next line
there is a tab here
throw some blank lines in here
See how that goes
There are a few switches that allow you to control the output of the contents. The -n, or –number, will number each line of output:
cat -n cat_test.txt
Produces:
You can just number only the non-blank lines using the -b, or –number-nonblank, switch:
cat -b cat_test.txt
Produces:
The -s, or –squeeze-blank, switch will remove repeated empty lines of output. This switch will not remove all blank lines, but only the second or more blank lines in a row. So in the example file the blank line after the fourth line is retained; but lines 6, 7, and 8 are “squeezed” out:
cat -sn cat_test.txt
Produces:
To see the non-printing characters that are in a file use the -v, –show-nonprinting switch. The example file has been augmented with some dos non-printing characters. Catting the file now:
cat cat_test.txt
Produces
But:
cat -vn cat_test.txt
Shows the non-printing characters:
Note the ^M and ^V characters on lines 10 and 11 respectively.
The -E, or –show-ends, displays a “$” at the end of each line:
cat -E cat_test.txt
Produces:
To show tabs in a file use the -T, or –show-tabs, switch and tabs are replaced with “^I”:
cat -T cat_test.txt
Produces:
There are a few aggregate switches that combine options:
If no file is specified with the cat command standard input is used. Similarly, using a “-” will signify to use standard input instead of a file:
cat
Issued like this by itself the cat command will echo back what is entered through standard input once the “enter” or “return” key is pressed.
The cat command has no pager functionality so all content that the command displays will scroll off the screen. Using a pipe, “|”, output from the cat command can be passed to a pager like less or more:
cat httpd.log | less
The cat command is a handy utility for viewing the contents of a file or redirecting the contents of one file into another. One example, culled from the Linux from Scratch documentation is to use cat as a quick and dirty text editor:
cat > test.sh << “EOF”
This example cats all standard in to test.sh until a single line containing “EOF” is listed. At which point cat closes the file and exits. Using cat like this or with other commands like cut, head, and tail one can perform some pretty sophisticated text manipulation.
Bibliography
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
Four columns are displayed here:
In the example above there is only one users logged in and that user is showing as logged in 10 times. Note the second column, the terminal the user is logged in to. The first line shows tty1 which is a virtual terminal or virtual console. Way back in the early days of computing a keyboard and display corresponding to one instance of a terminal to give you access to the computer. A virtual terminal allows for more than one terminal connection to be used by a keyboard and monitor. The device files for a virtual terminal can be found in the /dev directory. In this case, /dev/tty1 the terminal dann is logged into. This is also the terminal that X is running from.
The remainging devices labeled pts/# are psuedo terminals. Like virtual terminals, psuedo terminals provide terminal-like access. In this case, the psudeo terminals are all running under an X-session, specifically X display :0.0. These are all Konsole sessions. But they could very well be ssh sessions:
In this screenshot there are 2 different user accounts logged on: root and dann. The root account is logged in locally but the two dann accounts are logged in from different hosts.
By default the who command gets the information about who is logged in from the /var/run/utmp, although utmp could be in a different location. The utmp file keeps track of users currently logged into the system along with other system related information since boot. You can point who to a different file to look at by specifying the path to the file after who:
who /var/log/wtmp
In this example who will print out the history of everyone who logged into the system. The /var/log/wtmp file stores historical login information.
Aside from options and file there is one more arguement that who will take and that is “am i”:
who am i
This will print out the username of the user who issues this command along with the terminal the user is running on, the time logged in and the host or X display logged in from. Essentially this is just the who information for the current user and currently user only. There is also a flag that will report this information:
who -m
To help you better recall what each column means, the -H, or –heading, switch will display a heading row at the start of the output defining each column:
The -l, or –login, switch will show all terminals that are waiting for a login. Most Linux systems define 6 virtual terminals be available at boot: tty1-6. Those virtual terminals that have not had anyone log in on yet will be displayed by this switch along and the user name will show LOGIN.
The -u switch will add an addtional column to the output after the time logged in showing how long the user has been idle. The value here is in the format HH:MM. The range is 24 hours so any value over 24 hours will report as “old” and a “.” signfies activity in the last minute.
The -w, -T, –mesg, –message, or –writeable, switches report the status of users ability to accept messages. One of three values will be reported in the column after the user’s name:
The ability to accept messages is set with the mesg command. Most system shave ths turned off by default so to turn it on the user must issues:
mesg y
Now messages from non-root users can be sent to the user using the write command.
The –lookup switch will attempt to canonicalize hostnames via a DNS lookup.
The -q, or –count, switch only reports the usernames of the users logged in and the total users logged in. No other standard information is reported.
The who command can display information other than who is logged in. The following list of switches can be passed to who:
Some of these values will not be reported on systems that have moved from init to Systemd or Upstart, so your mileage may vary on what is returned.
The final switch to report is -a, or –all, which is a combination of:
-b -d –login -p -r -t -T -u
This tells who to display the last system boot, list dead process, show any terminals waiting for LOGIN, list any active processes spawned by init, list the last system clock change, print the message status, and report the idle time for each user in addition to the regular who output.
The who command is a useful utility in a multi-user environment that will help you get a grasp on the users logged into your system. Beyond that it can display some other important information regarding the system status since last boot. Some of who’s functionality may be limited by migration from init to systemd or upstart, so be aware of these limitations. Regardless, who remains an important utlity for your toolbox.
Bibliography:
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
Virtual memory is a memory management technique that, to the system, provides more memory than is actually available. Instead of a one-to-one addressing of processes to physical memory, the Operating System maintains a virtual table space where “pages” of virtual memory are allocated to processes. These pages are mapped to a physical memory via a page table. This physical memory can be RAM or more slower allocated disk storage like a swap partition or swap file. To best maximize system memory, the Kernel attempts to allocate physical RAM to active processes while moving inactive process to virtual memory. That is, an inactive program may have its page moved to the swap partition. Moving data from RAM to virtual memory can be done by either paging or swapping. Swapping is when an entire process is transferred from physical memory to virtual memory. Paging is when portions of a process are written to virtual memory. A page-out is the writing of the “page” to virtual memory and a page-in is reading the “page” from virtual memory. Paging is normal behavior for a system, but excessive paging can cause problems, especially excessive page-outs. If the tasks assigned to your system exhaust the physical RAM for all active processes, the Kernel may need to push some active process pages out to virtual memory slowing the performance of the system significantly. Using a tool like vmstat can provide valuable insights into the health of your system and help ferret out performance problems.
Executing vmstat by itself prints out one a summary of information based upon the last reboot of the system. The information is presented in a table format:
The first line details the category of information detailed in the columns below:
This is the default reporting for vmstat. What gets reported and how can be altered with switches, which will be covered shortly. As stated calling vmstat by itself prints out a summary of system information from the last reboot. Two values can be passed to vmstat. The first it a delay in seconds and the second is a count, the number of cycles vmstat will run for. If you only specify the delay value vmstat, that is no count, vmstat will continue to report indefinetly. Specifying delay and/or cycles changes the reporting values from the last reboot to report the values reflecting the time between the last cycle. The value for delay and cycle must be a postive integer.
vmstat 2 5
This will cause vmstat to report statistics every 2 seconds for 5 intervals. The first report will reflect statistics from the last reboot. Each subsequent report will contain the statitics on the previous 2 seconds, the duration between cycles.
The -S, or –unit, switch changes the default output values from KB (1024 bytes) to one of the following:
Changing the unit will not effect the output of si, so, bi, and bo.
While vmstat is running it is smart enough to re-print the header if the values increment enough to cause the header to scroll off the screen. This behavior can be suppressed with the -n, –one-header, switch.
The -a, or –active, switch will show active and inactive memory information. This will introduce two new fields that replace the Memory Buffer and Cache statistics:
Memory
The -s, or –stats switch will output a table of event counts and statistics:
This display cannot repeat so neither a delay or cycle value will be accepted. The information reported is essentially the same information gathered from the default output just in a more user readable format. Note that some of the statistics are broken out in more detail in this format whereas they are added together in the default reporting. For instance, the default output combines both nice and non-nice user cpu time into just user time whereas the –stat view breaks these values out.
The -d, or –disk, reports disk statistics, replacing the default memory, system and cpu statistics:
The report details the following:
Disk information is reported on a per block device basis. Partition information can be listed with the -p, –partition, switch and specifying the device:
vmstat -p /dev/sda1
Again, the default output is replaced with a report on the partiton listed:
Like the standard report, the disk report has a summary view reported using the -D, or –disk-sum, switch:
vmstat -D
The report is a summary of the information for all disks connected to the system. Like the –stat view, the –disk-sum report will not accept the delay and/or cycle value.
The -f, or –forks, switch will display the total number of forks since the last reboot. This values includes to sum of the following totals:
The –forks switch does not accept delay or cycle values.
Vmstat has the ability to report on slabs using the -m, or –slabs, switch. A slab is a memory managment mechanism for allocating kernel objects and is used by the kernel in controlling cache. This switch can only be used by an account with elevated privileges:
vmstat -m
The output is a less verbose summary of the /proc/slabinfo file:
The –slabs switch does not accept delay or cycle values.
The vmstat tool is a handy utility for reporting on your systems virtual memory use and will give you some statistics on disk storage. This is a handy tool for helping uncover performance issues on a Linux system.
Bibliography
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
The Apache BenchMark tool is included with the Apache software but may be available in your distrobution under the Apache-Utils package. You do not need to have the Apache Web Server running on the machine you want to run AB against and you can run it against any HTTP server, not just Apache. Only run this tool against a web server that you have permission to test. Running this tool against any website you desire can get you into trouble. Finally, you should not run AB on the same system that you are load testing against.
The Apache BenchMark tool’s executable is ab. The only parameter required to be passed to ab is the webpage you want to bench mark:
ab http://www.thelinuxlink.net/index.php
This will execute AB against the web server hosting the domain www.thelinuxlink.net and target the page index.php. This test is not very stressful as it will only perform one request against the server. You can specify the number of request you want to make with the -n switch:
ab -n 500 http://www.thelinuxlink.net/index.php
This will send 500 requests to the web server one at a time. Again, not very stressful against the server you want to benchmark. To really begin to stress the system include the -c, option which defines the number of concurrent connections per request:
ab -n 500 -c 100 http://www.thelinuxlink.net/index.php
This will again perform 500 requests to the index.php but instead of one request at a time will process up to 100 requests concurrently. This does not mean that it will process 100 concurrent request 500 times. To achieve that test you would have to run this command:
ab -n 50000 -c 100 http://www.thelinuxlink.net/index.php
Before continuing on to some of the other switches ab will take we should probably analyze the output of the command:
Here is a break down of the sections:
As ab runs it will indicate progress after each 10% or 100 (minimum value to trigger) requests of the total requests are made. Once the total number of requests have completed the summary results are displayed. The first five lines display some information about the environment being tested:
The section of information details the test that was run:
The third section details the overall connection time results presenting the minimum values, mean value, standard deviation, median, and maximum value:
These values are in milliseconds. Again, note that there are 5 columns here:
The final bits of data report the “Percentage of requests served within a certain time.” These values are in milliseconds. In this example 50% of the requests completed in at the most 990 milliseconds. 90% of the requests completed in at the most 2320 milliseconds (about 2.3 seconds). All requests completed within 2654 milliseconds (2.6 seconds) which was the longest request time noted by the test.
Now that you have a basic understanding of how to interpret the results of the ab tool, let us return to covering some more of the switches.
The -k switch will turn on the KeepAlive feature. KeepAlive will keep the connection open between the webserver and the client to perform multiple requests within the same http session. The default for ab is to have this feature off and require a new connection for each request in the same session from the same client.
If you need to authenticate to the webserver to request the resource you can use the -A with like this:
-A username:password
The credentials are sent in base64 encoding. So be careful if you are not connecting to an secured channel.
You can run tests against https and if you need to specify the SSL/TLS protocol use the -f switch:
-f SSL2 | SSL3 | TLS1 | ALL
If you need to run your test through a proxy use the -X to define the proxy address and port:
-X proxy_ip:proxy_port
By default ab uses a GET request which returns the body of the document requested. You can switch to a HEAD request with the -i switch. A HEAD request will not return the body of the message, only the document headers, the meta-information.
You can sent POST and PUT information using the -p and -u switches respectively. Both of these switches require that a file with the POST or PUT contents be passed to the switch. Additionally, you must specify the content-type with the -T switch. The default content type is text/plain. For POST/PUT data you will probably want to set this to:
-T application/x-www-form-urlencoded
An example POST or PUT file might contain the following information:
name=dann&[email protected]&country=us
You can change the size of the TCP send and receive buffers with the -b switch and specifying a size in bytes:
You can run ab for a period of time as opposed to a number of request using the -t switch and specifying a number of seconds:
ab -t 10 http://www.thelinuxlink.net/index.php
This will run the benchmark for 10 seconds with the default concurrency of 1 request.
There are a few switches to control the output of the ab results:
The results of ab can be output in HTML table format with the -w switch:
ab -n 1000 -c 100 -w http://www.thelinuxlink.net/index.php
The output will be HTML code with the results after the percentage update in an HTML table. You can specify html formatting with the following switches:
You will probably want to redirect the output to a file otherwise the code will just go to standard out:
ab -n 1000 -c 100 -w http://www.thelinuxlink.net/index.php > abresults.html
The ab command will generate a comma separated value file with the -e switch:
ab -e ab_results.csv -n 1000 -c 100 http://www.thelinuxlink.net/index.php
This will still display the progress on standard out. The resulting CSV file contains two values: A percentage from 0 to 99 and the time it took to serve that percentage or requests in milliseconds.
The -g option will output the results in a tab separated format that can be imported in applications like gnuplot, IDL, Mathematica, or a spread-sheet application like Libre-SpreadSheet:
ab -e ab_results.tsv -n 1000 -c 100 http://www.thelinuxlink.net/index.php
The Apache Benchmark tool is a handy application to have in your tool kit to not only perform cursory load tests on a web server but to stress the server to help identify intermittent problems. For more information check out these links:
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
A file or directories attributes can be viewed with the lsattr command. This command by itself acts like a directory listing showing all the files in the current working directory and their attributes. Specifying a single file will show only the attributes of that file. The output will look something like this:
lsattr test.txt
Produces:
————-e– test.txt
To fill in the dashes, should these values be set:
suS-iadAcj-t-e– test.txt
The lsattr command has a few flags you can pass:
The version generation number is typically handled by the file system. This number is handy in networked file systems like NFS where the version number can be checked to see if a file has been changed or deleted by another user in situations where multiple users are accessing the file.
File system attributes can be manipulated by the chattr command. The command works similarly to chmod in that it requires that the change in attributes and a file or list of files be passed to it. Changing attributes can be done with:
Thus to add the append attribute to test.txt:
chattr +a test.txt
Remember, this attribute can only be added by a superuser account like root.
To remove the append attribute:
chattr -a test.txt
Using the = to set attributes will force only those attributes to be applied. If there are already attributes that are applied to the file those will be removed if not specified by in the chattr command. The caveat here is that only those attributes that can be set with the chattr will be altered. Trying to set or remove the h attribute will fail as this can only be set by the file system not with chattr.
There are a few flags that chattr accepts. The -R flag recursively sets the attributes for all files and sub-directories below the specified or current directory.
The -V flag tells chattr to be verbose on what it is doing and will also print the program version.
The -f switch will suppress error message.
The -v switch will allow you to set the version number of a file. It requires the version number to be passed. Unless you know what you are doing, do not use this flag.
This episode of Linux in the Shell discussed extended file attributes and how to view and set them with the lsattr and chattr command.
-
bibliography:
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
2 = 2
3 = 3
4 = 2 2
5 = 5
6 = 2 3
7 = 7
8 = 2 2 2
9 = 3 3
10 = 2 5
11 = 11
12 = 2 2 3
13 = 13
14 = 2 7
15 = 3 5
16 = 2 2 2 2
17 = 17
18 = 2 3 3
19 = 19
20 = 2 2 4
Basically, prime factoring involves dividing a number until all you have left is the prime factors whose product when multiplied together is the number you factored. The number 1 is not reported. The factor command, which is included in the coreutils package will factor any positive integer you provide:
factor 1382
Reports the following result:
1382: 2 691
There are only two options you can pass to factor and those are:
The primes command takes a starting and optional stopping number and will output all the prime numbers from the start option to the stop option. If you do not provide a stop option it will continue until 4294967295 is reached or you kill the program with ctrl-c:
primes 500 1500
Primes does not have any options to pass other than the start and stop values. You must provide a start value of 0 or a positive integer. The stop value is optional but if used must be greater than the start value.
Chances are the primes command is not installed on your system by default. Primes is part of the BSD games package and should be an easy install via your package manager.
The seq command will output a complete list of floating point numbers given a starting and stopping number. You can optionally include an incremental value:
seq 5 .5 10
This example would start at the number 5 and show numbers between 5 and 10 and increment each step by .5:
5.0
5.5
6.0
6.5
7.0
7.5
8.0
8.5
9.0
9.5
10.0
If a start number is not given seq defaults to 1. If you want to specify an increment value you must also specify a stop number. Otherwise it will treat the increment value as the stop number. If the stop number is greater than the start number the output will be null unless you provide a negative incremental value:
seq 10 1
Produces no output, but:
seq 10 -1 1
Produces a count down from 10 to 1 in 1 step decrements.
If you pass a value increments or decrements unevenly the sequence will end at the last possible number:
seq 1 .3 3
Produces the following output:
1.0
1.3
1.6
1.9
2.2
2.5
2.8
The seq command has a few options that it can take. The -s, or –separator=, will change the default separator between each number. The default is the newline (e.g.: \n):
seq -s, 1 10
Produces the output:
1,2,3,4,5,6,7,8,9,10
The -w, or –equal-width, will print all numbers with the same with padding with zeros where necessary:
seq -w 1 3 15
Produces:
01
04
07
10
13
The -f, or –format=, option allows you to specify the printf-style floating point conversion specification to display the sequence in. The accepted values are:
To change the output sequence to hexadecimal floating point:
seq -f %a 1 6
Produces:
0x8p-3
0x8p-2
0xcp-2
0x8p-1
0xap-1
0xcp-1
For more information about the output formats for seq consult the info page.
Before closing this episode there is one more BSD Games program to cover: arithmetic. The arithmetic program is a mathematics quiz that will ask you a series of math problems. To proceeded to the next problem you have to answer the current problem correctly. The game will continue indefinitely printing your score after every 20 questions. The default operation of arithmetic will use the number 0 through 10 and only addition and subtraction. There are switches to change this.
The -o switch allows you to specify the operators which can be one, more, or all of the following:
Examples:
arithmetic -o +* – only addition and multiplication problems
arithmetic -o / – only division problems
The default range is 10 and this can be changed with the -r switch. Note that for addition and multiplication the range applies to the addends, terms or factors. For subtraction and division the range applies to divisor and quotient or the subtrahend and the difference.
arithmetic -r 1000
This sets the the terms range from 0 to 1000 for addition and the sutrahend and difference range from 0 to 1000 for subtraction.
Bibliography
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
Like bc, if you execute the command units you will be placed into an interactive tool that asks you what you have:
In addition to being prompted units will also show you the currency exchanges rates from date and the number of supported units, prefixes, and non-linear units. To exit units press the – keys.
On most systems units stores the unit database in /usr/share/units/definitions.units. You can verify this by executing units with the -V, or –version, switch:
This file is plain text and can be read via a pager like less or more.
To begin converting between units simply enter the unit you have. For instance:
1 mile
You will be prompted for the unit you want:
feet
Units will then perform the conversion giving you two lines:
* 5280
/ 0.00018939394
The first line, preceeded by the “*” will give you the conversion values you requested. The second line, preceeded by the “/” indicates the inversion of the conversion. In this case, 1 mile equals 5280 feet and the inversion, 0.00018939394 miles equals one foot.
The capacity of units is very large and you can easily determine what conversions are applicable to a given unit by entering “?” for “you want:”. This will generate a list of applicable units you can convert to.
The functionality of units is quite simple but the implementation can be tricky. For instance, converting between fahrenheit and celsius is not as straight forward as meters or feet:
you have: tempfahrenheit(32)
you want: tempcelsius
The value returned is 0 degress celsius. You could also use:
you have: 0 degC
you want: degF
But the value returned is 0 degree fahrenheit. How is this so?
you have: tempfarenheit(212)
you want: tempcelsius
This returns a value of 100 temp but:
you have: 212 degF
you want: degC
Returns a values of 117.77778. That does not seem right. The expectation that presenting the units in these ways indicates they mean the same thing. That tempC and degC are going to return the value of the temperature conferted from fahrenheit. While the first unit conversion using tempcelsius does this, degC on the other hand is showing the temperature change in celsius given the temperature change in fahrenheit. That is, a 212 degree fahrenheit change in temperature is equivalent to 117.77778 degree celsius change in temperature. Understanding this is something you have to pick up reading the man page. The units definition page will not easily convey this information but it does provide a wealth of knowlege on the units and how they were derived.
Similar to temperature conversions, converting between megabytes, gigabytes, and kilobytes is not as evident as you would expect. While most versions of units have tabbed completion, if you attempt to complete meg you will not megabyte but just meg. This is equivalent to megabits and will not give you the value expected if you attempt to convert 15gb into kilobytes. Instead you must use the proper binary prefixes:
To convert 15 gb into kilobytes:
you have: 15 GiB
you want: KiB
Produces the value: 15728640
Again, if you get stumped in what you are looking for consult the definitions.units file. A quick search will probably turn up the answer.
Units has a few command line switches:
-q, –quiet, and –silent suppress the prompting of the interactive tool. You will not see the “you have:” and “you want:” prompts.
The –compact supress the printing of the “*” and “/” but still show both values.
-l, –one-line, will only show the first line of output and surpress the inverse of the conversion.
The preceeding three switches are combined in the -t, –terse, switch which is equivalent to –quiet, –compact, and –one-line. That is, no prompts and only the conversion line is shown.
You can execute the units command to perform conversions without having to go into interactive mode by providing the request on the command line like so:
units “15 feet” inches
Produces the output:
Currency exchange rates from 2012-10-24
* 180
/ 0.0055555556
You can use the -t switch to just get the conversion you are looking for:
180
The -v, or –verbose, switch will provide a bit more output than normal:
units -tv “15 feet” inches
Produces the output:
Currency echange rates from 2012-10-24
15 feet = 180 inches
This entry just touched the surface of the units command. Units is an incredibly powerful tool at your disposal. To learn more about the command consult the man and info pages and consider giving the definitions list a perusal.
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
echo $((512 * 1024))
That should return the value
524288
But,
echo $((4/3))
Returns the value of:
1
When it should be closer to:
1.333333
If you are going to use echo to perform simple arithmetic you need to follow the syntax:
echo $(( number symbol number))
Similarly the expr command can be used for simple arithmetic:
expr 4 + 7
Returns the result
11
But again, expr will not work well with decimals:
expr 4 \/ 3
Returns:
1
Note in this example the “/” had to be escaped.
To perform more complex mathematical operations from the command line use the bc tool. GNU bc is “an arbitrary precision calculator language” that will accept input from files, redirection or will run it’s own interactive session. Issuing bc by itself puts you into interactive mode:
screen shot 1
This is the same as executing bc with the -i, or –interactive, switch. At this point you can enter the statement:
4/3
Which will return
1
We seem to be in the same predicament as before, no decimals. This is because by default bc is operating with a scale of 0. Scale is a special variable in bc that determines the total number of decimal digits after the decimal point. To see the current value of scale while in bc execute:
scale
The default return should be
0
You can alter scale like so:
scale=3
Which sets scale to 3 decimal digits after the decimal point. So now:
4/3
Returns
1.333
To exit bc simply type quit.
There are a few other switches to the bc tool. The -q, –quiet, switch does not print the welcome message when entering into interactive mode. The -v, or –version, switch will show the version and copyright information for bc and then quit.
The -l, –mathlib, will load the standard math library when bc is executed. This sets the default scale to 20 and allows for additional math functions:
Aside from evoking bc interactively, bc can operate on redirection:
echo 4/3 | bc
Returns
1
Again, we are missing the decimal point. There are to ways to resolve this:
echo 4/3 | bc -l
echo “scale=3; 4/3″ | bc
Which return respecitvely
1.33333333333333333333
1.333
Recall that the -l switch loads the math library in and sets scale=20 by default, so there are 20 decimal digits after the decimal point. The second example we set the scale variable to 3 and thus returns 3 decimal digits after the decimal point.
There are three other special variables that alter the behaviour of bc:
Both obase and ibase are set to decimal value by default. To switch to output in binary:
obase=2
To switch to input in Octal:
ibase=8
Note that changing this values will set the ibase and obase for all subsequent statements. To restore back to defaults either set each back to decimal:
obase=10
ibase=10
Or quit and restart bc. Issuing these as a redirection to bc will only set those values for that session:
echo “obase=2; 5*5; obase=10; last” | bc
Will return:
11001
25
The standard order of operations applies to the bc tool and can be altered with the use of parenthesis:
echo “5+2/2″ |bc -l
echo “(5+2)/2″ |bc -l
Return
6
3.50000000000000000000
The bc tool has it’s own language called bc. You can do far more than just calculator functions. You can define your own functions with the define command. For example:
define litsconverter (a) {
obase=2;
print “binary value is “;a*1; print “\n”;
obase=8;
print “octal value is “; a*1; print “\n”;
obase=16;
print “hex value is “; a*1; print “\n”;
obase=10;
print “all finished (p.s.: Ignore the 0 it returns at the end of every function if a return is not used”;
}
Now running:
litsconverter(15)
Will convert 15 (or whatever number you put in there) to binary, octal, and hex values the returns obase to decimal. The litsconverter function will remain available until you quit bc. Also note, that if we did not set obase back to decimal it would have stayed at hexidecimal until changed later on or you exited bc.
There is a lot more functionality to bc than just listed here. Consult the man page to get started on how to create your own functions to unlock the power of bc.
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
time ls
is
real 0m0.007s
user 0m0.000s
sys 0m0.003s
The first line, real, is how much real time or “wall-clock” time the process took to execute. In this case, 0.007 seconds or micro-seconds. The second line, user, is how much user time the CPU used. User time is amount of time the CPU spends performing an action for a program that is not a system call. The third value, “sys” referred to as system CPU time, is the amount of time the CPU spends peforming system calls on behalf of the program. A system call is a request made to the Kernel by the program. Therefore, CPU time is the total of user time plus system time. CPU time will not necessarily equal real (wall-clock) time, and if it does not total CPU time would be less than wall-clock time. There can be many reasons for the discrepancy between wall-clock and CPU time including the process waiting for another process to complete. In the example above total CPU time is 0.003 seconds where as wall-clock time is 0.007s.
Most Linux distributions default to using the Bash shell. Bash has a built in time function that operates as listed above and is quite different than the actual time command. The functionality of the bash time command is very limited to the actual time command. The remainder of this article will focus on the actual time command as opposed to the built in bash time command.
Most distributions have the time command installed under /usr/bin. Executing:
which time
Should output the path to the time command:
/usr/bin/time
If no value is returned chances are you do not have the time command installed (I had this issue on Arch and had to install the time command by sudo pacman -S time, on Slackware it was already installed).
To use the time command as opposed to the bash built in command you must provide the full path to time:
/usr/bin/time ls
Note that the output is quite different:
0.00user 0.00system 0:00.00elapsed 50%CPU \
(0avgtext+0avgdata 784maxresident)k
0inputs+0outputs (0major+250minor)pagefaults 0swaps
There is a lot more information provided compared to the builtin Bash time command.
The first three items: 0.00user 0.00system 0:00.00elapsed should look familiar. These values are user CPU time, system CPU time, and wall-clock time respectively. To understand the remaining values better focus will be directed on how to actually format the output of the time command. This is done with the -f, or –format=, switch followed by the formatting strings. Similar to the date command, the formatting strings are in the form of “%” + variable:
Now that these values have been defined the output of:
/usr/bin/time ls
From above can be further explained:
0.00user 0.00system 0:00.00elapsed 50%CPU (0avgtext+0avgdata 784maxresident)k 0inputs+0outputs (0major+250minor)pagefaults 0swaps
In a nutshell:
%U %S %E %P (%X + %D %M)k
%I + %O (%F+$R)pagefaults %W(User CPU Time) (System CPU Time) (Wall-Clock Time) ( (Average size of shared text space) + (Average size of unsahred data are) (Maximum Resident Set Size of Process) )k
(Number of File System Inputs) + (Number of File System Outputs) ( (Number of Major Page Faults) + (Number of Minor Page Faults) pagefaults (Number of times process was swapped out of memeory)
As stated, formatting of the /usr/bin/time command can be done with -f, or –format=, switch followed by a string including the above variables:
/usr/bin/time -f “%e may not equal %S + %U” ls
The output of this may look something like:
0.04 may not equal 0.00 + 0.00
To emulate something like the output of the Bash built in time command you could execute:
/usr/bin/time -f “real\t%e \nuser\t%U \nsys\t%S” ls
The output may look like:
real 0.04
user 0.00
sys 0.00
Or you could pass the -p, –portability, switch which provides the same formatting:
real %e
user %U
sys %S
The output of time can be directed to a file with the -o, or –output=, switch and supplying the file name:
/usr/bin/time -o times.txt ls
This would redirect the output of the /usr/bin/command to the file times.txt. If you executed the same command again, it would overwrite the original times.txt file. If you want to preserve the original contents of the times.txt file use the -a, or –append, switch with the -o, –output=, switch:
/usr/bin/time -o times.txt -a ls
The final switch to discuss is the -v, or –verbose, switch which will output all the values possible that /usr/bin/time can report regarding a process.
/usr/bin/time -v tar cvf test.tar.gz lib
Produces the following output:
That is the time and /usr/bin/time commands in a nutshell. Remember, if you want the full feature set of the time command you must provide the full path to the time command. Otherwise you will more than likely use the built in time command of the shell you are running.
Video failed youtube processing, will be up shortly, use archive.org link below.
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!
locale -m LC_TIME
This will list the available locales configured for your system. Configuring of the locale values is dependent upon the distribution of Linux you are running. For instance, in Arch Linux you can set the locale by editing the /etc/locale.conf file. On a RedHat based system this value is store in /etc/sysconfig/i18n. On a Debian based system the default locale is stored in /etc/default/locale. Managing locale information is beyond the scope of this article so consult your distrobution’s documentation should you need to change your locale values or set a new default.
For the purpose of this article examples will be shown from the LC_TIME=en_US.UTF-8 value.
The date command issued by itself reports the date in the following format (Abbr stands for abbreviated):
[Abbr Day of Week] [Abr Month] [Month Day] /
[Hour:Minute:Second] [Timezone] [Year]
Sat Feb 9 11:48:51 EST 2013
%a %b %d %H:%M:%S %Z %Y
The last row is the relates the format controls to date corresponding to the equivalent values displayed on the second line. These will be covered shortly.
The -d, or –date=, parameter takes a string format for a date and returns a value as opposed to the current date. There are many different possible strings to pass for instance:
date -d “2 days”
will return the date string like above but 2 days from the current time:
Mon Feb 11 11:48:51 EST 2013
Issuing:
date -d “2 days ago”
Will return the date string above but from 2 days ago:
Thu Feb 7 11:48:51 EST 2013
Now realize that the actual time would be the time from 2 days prior or 2 days hence from when the date command was run and would not output the exact time string as the first entry. The example was given just an example based off the first date above.
Some of the values you can pass with the -d command are:
The -f, or –file=, switch operates the same way as the -d but will read a file where each line is a date value in the format of -d. For instance, if there was a file called datefile.txt with the following entries:
3 days ago
1 month
2 hours 9 minutes
Sun
Then:
date -f datetime.txt
Would read each line of the file and output the corresponding values:
Wed Feb 6 12:03:49 EST 2013
Sat Mar 9 12:03:49 EST 2013
Sat Feb 9 14:12:49 EST 2013
Sun Feb 10 00:00:00 EST 2013
The date command will allow you to view the last modification date of a file with the -r, or –refrence=, switch:
date -r datetime.txt
Returns the time the file was last modified:
Sat Feb 9 12:03:45 EST 2013
Before we look at the remaining swtiches to date, let us turn to how the output of date can be formatted differently. The syntax for fomatting the date ouput is as follows:
+%?%?%?
Where ? is a placeholder for a variable. Some of the more common variables are:
The output of most formats is to pad numeric fields with zeros. This can be altered with the following flags:
Appending a number between the % and variable will specify the width of the field:
date +%Y
date +%2Y
date +%8Y
The following values are returned:
2013
2013
00002013
Note that the second entry date +%2Y did not truncate the date to two characters. Field width will not limit the output but will pad the output.
If your locale supports it there are two more flags that can be applied to some of the format options:
To put some of this in perspecitve given our original date output: Sat Feb 9 11:48:51 EST 2013, the following lists the output returned by the specific formatting:
date +%Y = 2013
date +%Y%m%d = 20130209
date +%Y-%m-%d = 2013-02-09
date +%P = pm
date +%w = 6
date +%W = 05
date “+Today is %A have a good day” = Today is Saturday have a good day
Note that the third and last examples add extra content into the format. Any non-variable will be echoed back out in the string. If you want to include a “%” then issue like so: %%:
date “+%A is the 100%% best day”
There are other format tags for date:
These three values format the date and time according to their respective specifications. The first two, -R and -I show their defaults and require no further specification. –rfc-3339 requires that you specify a TIMESPEC. You can also specify TIMESPEC for –I using –iso-8601=TIMESPEC. TIMESPEC values between the –iso-8601 and –rfc-3339 differ:
The -u, or –utc or –universal, flag will output the date in Coordinated Universal Time
date
date -u
Shows the comparison:
Sat Feb 9 14:46:17 EST 2013
Sat Feb 9 19:46:19 UTC 2013
The date command can not only display the date or time but can also set the date and time with the -s, or –set, flag. This of course requires elevated priveleges:
date -s 2013-01-15
This would set the system date and time to:
2013-01-15 00:00:00 EST 2013
But:
date -s “2013-01-15 08:15:44″
Would set the date to:
2013-01-15 08:15:44 EST 2013″
The date command is a fantastic tool available for providing date stamps in your scripts. For example:
#!/bin/bash
if [ -n "$1" ] && [ -e "$1" ]
then
cp $1 $1.`date +%Y%m%d-%H%M%S`
fi
If you save this script as makebkup, make it executable, when called this script will take a file as input using the cp command make a backup using date to create a file name with a time stamp:
makebkup mypoem.txt
This would effectively run:
cp mypoem.txt mypoem.txt.20130209-150823
The date command is a powerful tool to have at your disposal. Check out the man or info pages for the full list of formatting variables.
If the video is not clear enough view it off the YouTube website and select size 2 or full screen. Or download the video in Ogg Theora format:
Thank you very much!