Bash 101

So are you new to Linux?  Are you the kinda person who's like "Awesome, Linux!" and you're also like "Awesome, Command Line Interface (CLI)!", but you're also like "I don't know how to use the command line effectively and desire to know more on this topic in order to advance my knowledge of the Unix world!"

If so, this article is for you.  It's really just to get you started.

This isn't really a technical guide.  This is a list of practical tips that, looking back, I wish I had seen somewhere when I started to make myself use the CLI on a regular basis.  Once you get used to the CLI, you start to use it a lot; on my Linux box, the only window I usually have open that isn't a terminal is Firefox.  Everything else - with a few exceptions - can, in my opinion, be done effectively from the command line.

So here's my list of tips for the CLI beginner:

1. Make use of auto-completion
This is a small tip, but useful if you don't already know it.  If you're using the bash shell, you can usually press [ Tab ] to do this.  In other words, say I have a directory I want to cd (change directory) to:

• user@localhost:~$ ls
  
• Directory_1 Directory_2 Directory_3 Other_Directory
  
• user@localhost:~$ cd Directory_2
  

Instead of taking the extra few seconds to type in "Directory_2", you can simply start the word and press tab.  In this case, suppose you type "Dire[ Tab ]".  As soon as you press [ Tab ], the shell will complete the word "Directory_" for you.  It won't, of course, put the number, because there's more than one choice.  If you had typed "Oth[ Tab ]" or even just "O[ Tab ]", it would completely fill the word "Other_Directory" for you.

This same tip will work if you are starting to type an executable name in your $PATH (the variable that knows where to find all of your executable files).  Suppose you wanted to run the program gnome-system-properties from the command line (which would actually launch a GUI, but can be useful anyway).  To speed things up, type a few letters and hit [ Tab ].  Bash will complete the string as far as it can until there is any uncertainty, which probably means up to 'gnome-'.  Perhaps you've forgotten the exact name of what you're trying to run?  Hit [ Tab ] again and you'll be given a list of possible completions.  Start typing again hit [ Tab ] to complete if you get lazy.

You will still, however, have to press [ Enter ].

P.S. - If this isn't working, you might not be using the bash shell (type set | grep SHELL or something reasonably equivalent to find out what you're using).  This doesn't mean all hope is lost.  If you're just in plain old sh, for example, you can use the wildcard character '*' to complete your phrase.  In the first example above, then, you could type 'cd Dire*2' to go to Directory_2.

2. Understand I/O Redirection
As many people will tell you, piping and I/O redirection is perhaps the most powerful part of the nature of the CLI.  Take this example:

Suppose you had to build a toy car.  You need someone to carve the right shape, someone to put on the wheels, someone to screw everything in, and someone to put it in the correct package.  In one case, you could give the car to the person who carves it, and then take it back.  You would put the car in a box and go find the person to put on the wheels.  When they were done, you would package it and take it to the person adding the screws.  Finally, you would put it back in a package and take it to the person who puts it in the package for marketing.

In another example, though, you could setup an assembly line and use a conveyor belt.  This removes your role as the messenger and removes the intermediate packaging.  This second example is analogous to the CLI.  Whereas in the GUI example one has to use a program, save the result, and then open a whole new program, the CLI-master can simply tell each program in line to pass on their result to the next program.  You might write, for example:

• user@localhost:~$ carver | wheeler | carpenter > correctBox
  

Here, the carver program sends it's output to the wheeler program which sends it to the carpenter program which then writes all of its output to the file correctBox.

As a more useful example, consider this command in my .bashrc file:

• alias dl = 'ls -laht | sed -n /^d/p'
  

After the next two tips, you'll understand what that does.

3. Use Stream Editors
More specifically, I mean that you should try to understand the power of commands like sed and grep.  Programs like these can help you get the data you want from other commands.  Say, for example, that I wanted to find all of the C source files in my ~/Code directory.  I might write something like:

• user@localhost:~$ ls Code | grep '\.c'
  

First of all, do you understand why simply grep .c doesn't give the same results in general?  If not, we'll get to that in a second.  Right now just realize that the period is a special symbol (a metacharacter).

The above command does this:  first, it uses the ls command to print out all of the regular files and folders in the directory.  The grep command then receives these filenames (just the names, not actually the files) as they are generated, and it selectively decides which ones to output.  In this case, it only prints out the strings which contain the substring '.c', which is more-likely-than-not a C source file.  Of course, if you happen to have a file called "a.b.c.d.e.txt", it will catch this as well.

For dealing with more advanced structured, we can use sed (among other tools); that, however, is a topic beyond the scope of this article.  As a basic primer, it can be said that sed usually operates on text using regular expressions, a powerful tool that the Perl hacker may already be familiar with.  This is what it does in the above example; for everything output by ls -laht (which, by the way, is a listing of the current directory in Long form with All files listed (as opposed to only non-hidden ones) in Human-readable file sizes, sorted by Time modified), sed only lets through lines that start with the letter 'd', which, using ls's long form, means that they are directories.  That's why I called this alias dl - it stands for directory listing (in my head, at least).

It should be noted that this is only a basic usage of sed, and in fact this exact same thing can be done with grep (ls -laht | grep ^d).  That '^' in both the sed and grep expressions is called a metacharacter.  Just like the period that we were searching for in the first grep example above, we would have to backslash-escape the character like this if we actually wanted to search for the character '^':

• user@localhost:~$ ls -laht | sed -n /\^d/p
  

In this case, '^' would no longer mean 'beginning of line'; instead, it tries to find the caret symbol within the inputted text.

4. Force yourself to use the CLI by trying to do everything with it until it gets to the point where you know where you want to be.
This is an interesting point that other people might not suggest, but I did this and it seemed to work.  If you really do have a desire to use the CLI but don't know how, just try forcing yourself to use it.  I did this by installing Fluxbox as my desktop environment and making the only two things in my menu 'firefox' and 'gnome-terminal'.  I use Alpine for my mail, VIM as my editor, Irssi as my IRC client, CenterIM as for Google Talk, and then I usually have one or two other terminals open just for kicks.  One probably has Nethack open.  The only non-terminal based program I use in Linux is Firefox, because the the Web really has evolved beyond the reasonably capacity of, say, Lynx (some might argue that point).  I also use VLC occasionally to watch movies, which obviously can't be done in the CLI.  I used to even use the CLI for playing music, but since my music started being centralized on my Mac, I usually just plug my speakers into my iPod.

So I suggest you do something similar - just keep Googling things like "best CLI programs" or "favorite terminal applications" until you know a reasonable amount about what's out there (and what's good) for the command line.  Make yourself use them for a month and see how you feel.  I decided not to go back (although I can't say that very well now that I have a Macbook, but it was true for a long time up until a few months ago).

Best of luck!

5. Links & Resources

Bash Beginner's Guide
Advanced Bash Scripting
Linux Shell Scripting Tutorial
CLI Apps
Favorite Command Line Programs

sunlight, graphite: recipe for meditative thought #1.

I'll be starting a series of meditative exercises for people to try if they so desire.  Here's the first one, which I did yesterday.

"Sunlight and Graphite"

(1) Make sure the room is comfortable to slightly cool.

(2) Prepare your favorite cozy beverage.  I picked black coffee, but various teas or cocoas would be alright as well.

(3) Find a hobby that is attention-intensive but not intellectually overbearing.  I picked sketching, which I think is a good suggestion for anybody.

(4) Sit in front of a window where the sunlight is coming in.  Open the window if it's comfortable outside.

(5) Put on some music, but nothing too fast or violent.

(6) Perform your activity in a meditative state.  The key is not to get frustrated.  Even if you're drawing something for the first time, you must take joy in the process of drawing and be unconcerned with the physical result.

(7) If you're using coffee or a similar drink, occasionally tip off your cup.  The sound of the pouring liquid is also meditative.

(8) Throughout the process, another key to getting the most out of it is opening the senses.  Feel the steam of your drink against your face contrasting with the cool room; listen to the graphite as your pencil streams across the paper; smell the air coming in through the window. Simply be aware.  Feel the rhythm of the music in your body.

(9) Ignore the clock.

finding the gcd using the euclidean algorithm

Remember the GCD (or GCF) from our younger math days?  The greatest common denominator (or factor) seemed pretty retarded.  We never really used it... especially once we had calculators to guess and check for us (or until we all got our TI-83/84/89's and had a function to do it).

But as I hack and slash my way through the dense, unforgiving jungle of Deskin's Abstract Algebra (you'll remember that Mark gave it to me at my birthday, if you were there), I'm finding more and more reasons to think that basic math is cool.  The book is very hard to get through, extremely exact and unforgivingly precise and verbose... but the subjects are things like counting, finding GCD's and LCM's, and expressing integers.

So anyway, I came across the "Euclidean Algorithm" the other day, which basically helps you calculate the GCD of two numbers.  I'll summarize it here.

So suppose two numbers X and Y, and say that we want to find their greatest common factor.  In order for me to show do a simultaneous example with real numbers, let's call say that they happen to equal the integer values 320 and 144, respectively.

Before we calculate the greatest common factor, I need to make a quick segue and discuss the Divisor Theorem.  Another formalization of 3rd grade math, the Divisor Theorem says that the following expression holds for all integers a, b, q, and r, such that b > a > r:

b = aq + r

In 3rd grade terms, b divided by a equals q remainder r.

So with that in mind, let's return to our original problem.  Suppose X > Y (and it is, as 320 > 144).  Then we can create the expression:

X = Y*q1 + r1   |   320 = 144*q1 + r1

Right?  Of course we can, because those are all integers.  So a little third grade division tells us that our numerical example resolves to:

320 = 144*2 + 32

Now, watch this:  we're going to forget X, make Y the largest number, and make the old remainder the new quotient.  Then we'll add a new remainder:

Y = r1*q2 + r2   |   144 = 32*q2 + r2

Now we just solve this in the same fashion:

144 = 32*4 + 16

We follow the same variable shifting pattern (try writing these equations in order on your paper and using arrows to show how the variables move right to left as you go down.  The arrows should end up being diagonal lines from the top-right to the bottom left.):

r1 = r2*q3 + r3   |   32 = 16*q3 + r3

If you understand the pattern we're following, then good!  You know how to use this algorithm.  If not, you're screwed, because we're basically done.  Note that in the above example q3 = 2 and r3 is exactly 0 (because 16 is a factor of 32).  This means we're done!  The number that q3 is multiplying (here, r2=16) is the greatest common factor of the original numbers.  That's it!

Basically, you would keep following this pattern until you got to evenly dividing numbers such that the new remainder is zero.  The greatest common factor of the original X and Y is exactly the number in the position where Y started out.  When I say exactly, what I'm trying to say is that... that's it!  It's not possibly some multiple of that; that is the answer.

More later on how every integer can be expressed in the form ax + by = c, where all those numbers are integers.  It has a lot to do with greatest common factors.

Final note:  the greatest common factor of two numbers A and B is often expressed as (A, B).  This should be easily distinguishable from an ordered pair by the context it's used in.

$ man punching

How do you throw a proper punch?  What's all this internal energy stuff they talk about in martial arts?  Here's the first half or so of an article I'm writing to explain.

  Directing Energy in Your Martial Arts

  ----       Matthew Daniels       ---- 

|=======================================|

If there is one thing that seems to frighten and temporarily alienate newcomers to the martial arts, it seems to be noise.  The kiai we often see in the Japanese arts is a prime example.  Often, a new student will feel out of place when asked to kiai, and may sometimes refuse to kiai for weeks into their training.  Even among those who do not refuse, it is not uncommon for the more arrogant students to snicker behind their instructor's back at the idea of a kiai.

But for the advanced student, the kiai is no longer only a noise; in fact, one might eventually feel that the kiai is not a noise at all, but happens to produce some involuntary sound as a consequence of an energetic explosion which drives the power behind their motions.

When a student begins to realize that a kiai is, indeed, not a noise at all, he has probably started to understand the importance of energy management in his chosen art.

I - and my instructors, before me - will sometimes ask students where their punch begins and ends.  To a white belt, this may seem a nonsensical question; the fist, of course, travels from the ribs to the desired target (the opponent's solar plexus, for instance).

Obviously, this is not the answer that an instructor is looking for.  The punch should start not from the ribs, but from the ground.  For some students, this will initially seem confusing, but it is certainly true.  The energy that drives your fist should start from the heel.

As a punch is executed, the heel should push against the ground.  Imagine that the path of your body between the heel and the knuckles is a large canoe resting against a riverbank (the ground).  From a standstill, can you simply begin to paddle and expect the canoe to move?

Well... actually, you can.  And sometimes, there's nothing wrong with this.  This is called punching from the shoulder.  But would it not be faster and more powerful to push off of from a sturdy riverbank, getting your momentum going before you begin to paddle?  This is called punching from the heel.

A convenient way to practice energy concepts like this one is to visualize the energy transferring through your body in real time, as you're punching.  The Chinese traditionally believe in the concept of chi, an energy-like substance which permeates the body and the world, providing a sort of life-force.  Fully appreciating the concepts of chi and the Chinese philosophies behind the different energies at work in the cosmos and inside the human body is far, far beyond the scope of this article.  For now, to illustrate my ideas about energy in your martial art, we will make the gross assumption that chi is a homogeneous, energetic fluid that can be directed and harnessed by the body.

So, in the previous example, can we draw energy like this from the ground?  Yes, we can; and you should, if you wish to generate a respectable amount of power in your attack.  If you visualize the energy transfers at work in your body, it can help you attain this energy channeling and put your whole body in sync with your attack.

Let's run a basic trace of this energy as it travels from the Earth and out of your fist.  For our purposes, the Earth is not moving; it provides an eternal riverbank to shove off of, and is a limitless source of energy.  Therefore, when appropriate, we should always try to use it to our advantage.  To do this effectively, we must root ourselves into the ground; this is one application of your stance training.  Try to imagine your feet reaching down into the ground, locking themselves in place by becoming part of the Earth.  These roots will help ou soak up the energy that will drive your fist.

As you shove off from the heel (without losing your root), the energy will being to travel up the legs.  As it reaches the waist, two things should happen simultaneously.  The energy should being to torque the waist and hips.  It turn, the waist and hips will accelerate the energy up the body and into the spine.

As the energy travels up the spine, a similar reaction will take place.  The energy will excite the spine, causing a wave through it and up the back.  Your chi should ride this wave up your body (along the fire path), and it will be thrown out of the spine and into the arm.  

perlmwdtut(0)

A tutorial for those new to Perl but not to Programming: Part 0

Introduction

When you don't have much to blog about, you start trying to think about things you already know.  Well... I know Perl, and I have some friends who would like to learn it, or at least be comfortable with it.  I'll present this tutorial in progressively more advanced parts, showingcasing a little at a time the high and low points of Perl and how to use it.

What is Perl?

Perl is lots of things to lots of people, but to me - and many other people - it's a swiss army knife language (if you will).  When you pull a bunch of random data out of an excel spreadsheet and you need to do some stuff with it real quick (like, maybe, remove invalid email addresses and then send a mass message), you don't compile a huge C application and you don't need to power up your Java VM.  You just write a few lines of Perl.

Perl has had other purposes, too.  Sometimes referred to as the glue that holds the web together, Perl has been used extensively throughout the years in CGI scripting, although it is my believe (but not quotable experience) that it is falling out of use with the evolution of the internet into a far mor

e interactive and m

ultimedia-oriented experience... although I also know that there is still plenty, plenty, plenty of Perl out there.

Perl is also great for just hacking things together and making things work.  If you had a round peg, a square hole, and a pocketknife, Perl would be that pocketknife.  Observe this comic from xkcd.com (released under Creative Commons; kudos to Randall Munroe for his amazing webcomic):

(Click for larger view.)

Anyway, that's about what Perl is.

Jumping Right Into It

No tutorial would be complete without...

#!/usr/bin/perl

use warnings;

use strict;

print "Hello, world!";

There's our familiar Hello, world! program.  The first three lines are unnecessary, generally.  You could write the program as simply print "Hello, world!";, execute perl and it would work just fine.  But if you're curious, here's what those first lines are all about.

Line 1: #!/usr/bin/perl

This line is often called a shebang, and is common to many Unix scripts.  In fact, if you go find a bash script on your computer, chances are that the first line is something like #!/bin/sh.  That's because when you run chmod +x script.pl to turn your script into an executable file, it looks as that first line to determine what needs to be called to run it.

Of course, if you're running Windows, none of this will make any sense.  If, for some reason, you are in a Windows system, this first line won't help you out much.  Just run your scripts using the perl syntax and everything will be okay.  Depending on your Perl interpreter (I know that ActivePerl works this way), you could also just double-click on your script in Explorer and it should run.  Be wary that if you're script isn't actually doing anything functional - for example, if it's just the Hello, World! script above - then you probably won't see anything happen besides a terminal appear for a few milliseconds and disappear again, because the terminal's process will die along with the Perl script.

But be warned that if you're running Windows, you probably won't be able to use a lot of the things in this tutorial.  Go use Visual Basic or something.

Anyway...

Line 2/3: use warnings; use strict;

The use of the warnings module alerts us when we do silly things.  It won't stop our program from running, but it might let me know if, say, I declared a variable I never used.  Sounds stupid?  Not really.  Perl is extremely nice to its user and tries to do exactly as it is told... which can be a problem.  Say I have the variable $rest = 5; and then later on I mistype it: for ($x=0; $x<$breast; $x++)....  Normally, an interpreter/compiler would catch this Freudian slip (perhaps you had some inappropriate browser windows open while coding), but Perl will do its best to do exactly as you say, so it'll instantiate $breast for you with the default value of 0.  This will not function correctly.  Using warnings will probably give you a little wake up call if you knew this variable was supposed to be in use.  This is only an example, though; the warnings module is extensive and highly useful for debugging.  The use strict flag makes you declare all variables with a some sort of scope modifier, which we'll talk about later, but it is important to note that, unlike many languages, a variable declared in any scope will, by default, become global.  This often causes trouble if not handled with care.

If you can't figure out line 4, you probably shouldn't be reading a tutorial for people who already have some programming experience.  In fact, you should probably understand line 4 whether you have programming experience or not.

Variables

Watch this:

$variable = 5;

$variable = "Hello, Janet!";

$variable = \$referenced_var

$variable = \&referenced_subroutie

$variable = -234.3

$variable = 0x0000003f;

$variable = "Some other data type!";

print $variable; # prints out "Some other data type!"... and yes, this is a valid program.

# By the way, this is a comment!

Perl has a what's called a dynamic typing system.  Several other languages have this as well (Ruby and Python, to name two) but it isn't typical of what you see in C, C++, or Java.  Whereas in those languages you must declare a variable's data type (either beforehand or on-the-fly), Perl variables can hold any type of data, any time... the only restriction is on memory (which, these days, is considerable).

Note: Perl 6 (which will be similar but also considerably different from Perl 5 and earlier) will have an exciting new hybrid typing system which allows you to assign a value if you want to.  For example, you could write the traditional, perlish: $variable = 5;, but you can also write the sometimes useful and more C-ish: int $variable = 5;, which will treat $variable just like C would.

More to come later!