Thursday, 22 January 2015

Unix - The vi Editor Tutorial

There are many ways to edit files in Unix and for me one of the best ways is using screen-oriented text editor vi. This editor enable you to edit lines in context with other lines in the file.
Now a days you would find an improved version of vi editor which is called VIM. Here VIM stands for ViIMproved.
The vi is generally considered the de facto standard in Unix editors because:
  • It's usually available on all the flavors of Unix system.
  • Its implementations are very similar across the board.
  • It requires very few resources.
  • It is more user friendly than any other editors like ed or ex.
You can use vi editor to edit an existing file or to create a new file from scratch. You can also use this editor to just read a text file.

Starting the vi Editor:

There are following way you can start using vi editor:
CommandDescription
vi filenameCreates a new file if it already does not exist, otherwise opens existing file.
vi -R filenameOpens an existing file in read only mode.
view filenameOpens an existing file in read only mode.
Following is the example to create a new file testfile if it already does not exist in the current working directory:
$vi testfile
As a result you would see a screen something like as follows:
|
~
~
~
~
~
~
~
~
~
~
~
~
"testfile" [New File]    
You will notice a tilde (~) on each line following the cursor. A tilde represents an unused line. If a line does not begin with a tilde and appears to be blank, there is a space, tab, newline, or some other nonviewable character present.
So now you have opened one file to start with. Before proceeding further let us understanding few minor but important concepts explained below.

Operation Modes:

While working with vi editor you would come across following two modes:
  1. Command mode: This mode enables you to perform administrative tasks such as saving files, executing commands, moving the cursor, cutting (yanking) and pasting lines or words, and finding and replacing. In this mode, whatever you type is interpreted as a command.
  2. Insert mode: This mode enables you to insert text into the file. Everything that's typed in this mode is interpreted as input and finally it is put in the file .
The vi always starts in command mode. To enter text, you must be in insert mode. To come in insert mode you simply type i. To get out of insert mode, press the Esc key, which will put you back into command mode.
Hint: If you are not sure which mode you are in, press the Esc key twice, and then you'll be in command mode. You open a file using vi editor and start type some characters and then come in command mode to understand the difference.

Getting Out of vi:

The command to quit out of vi is :q. Once in command mode, type colon, and 'q', followed by return. If your file has been modified in any way, the editor will warn you of this, and not let you quit. To ignore this message, the command to quit out of vi without saving is :q!. This lets you exit vi without saving any of the changes.
The command to save the contents of the editor is :w. You can combine the above command with the quit command, or :wq and return.
The easiest way to save your changes and exit out of vi is the ZZ command. When you are in command mode, type ZZ and it will do the equivalent of :wq.
You can specify a different file name to save to by specifying the name after the :w. For example, if you wanted to save the file you were working as another filename called filename2, you would type :w filename2 and return. Try it once.

Moving within a File:

To move around within a file without affecting your text, you must be in command mode (press Esc twice). Here are some of the commands you can use to move around one character at a time:
CommandDescription
kMoves the cursor up one line.
jMoves the cursor down one line.
hMoves the cursor to the left one character position.
lMoves the cursor to the right one character position.
There are following two important points to be noted:
  • The vi is case-sensitive, so you need to pay special attention to capitalization when using commands.
  • Most commands in vi can be prefaced by the number of times you want the action to occur. For example, 2j moves cursor two lines down the cursor location.
There are many other ways to move within a file in vi. Remember that you must be in command mode (press Esc twice). Here are some more commands you can use to move around the file:
CommandDescription
0 or |Positions cursor at beginning of line.
$Positions cursor at end of line.
wPositions cursor to the next word.
bPositions cursor to previous word.
(Positions cursor to beginning of current sentence.
)Positions cursor to beginning of next sentence.
EMove to the end of Blank delimited word
{Move a paragraph back
}Move a paragraph forward
[[Move a section back
]]Move a section forward
n|Moves to the column n in the current line
1GMove to the first line of the file
GMove to the last line of the file
nGMove to nth line of the file
:nMove to nth line of the file
fcMove forward to c
FcMove back to c
HMove to top of screen
nHMoves to nth line from the top of the screen
MMove to middle of screen
LMove to botton of screen
nLMoves to nth line from the bottom of the screen
:xColon followed by a number would position the cursor on line number represented by x

Control Commands:

There are following useful command which you can use along with Control Key:
CommandDescription
CTRL+dMove forward 1/2 screen
CTRL+dMove forward 1/2 screen
CTRL+fMove forward one full screen
CTRL+uMove backward 1/2 screen
CTRL+bMove backward one full screen
CTRL+eMoves screen up one line
CTRL+yMoves screen down one line
CTRL+uMoves screen up 1/2 page
CTRL+dMoves screen down 1/2 page
CTRL+bMoves screen up one page
CTRL+fMoves screen down one page
CTRL+IRedraws screen

Editing Files:

To edit the file, you need to be in the insert mode. There are many ways to enter insert mode from the command mode:
CommandDescription
iInserts text before current cursor location.
IInserts text at beginning of current line.
aInserts text after current cursor location.
AInserts text at end of current line.
oCreates a new line for text entry below cursor location.
OCreates a new line for text entry above cursor location.

Deleting Characters:

Here is the list of important commands which can be used to delete characters and lines in an opened file:
CommandDescription
xDeletes the character under the cursor location.
XDeletes the character before the cursor location.
dwDeletes from the current cursor location to the next word.
d^Deletes from current cursor position to the beginning of the line.
d$Deletes from current cursor position to the end of the line.
DDeletes from the cursor position to the end of the current line.
ddDeletes the line the cursor is on.
As mentioned above, most commands in vi can be prefaced by the number of times you want the action to occur. For example, 2x deletes two character under the cursor location and 2dd deletes two lines the cursor is on.
I would highly recommend to exercise all the above commands properly before proceeding further.

Change Commands:

You also have the capability to change characters, words, or lines in vi without deleting them. Here are the relevant commands:
CommandDescription
ccRemoves contents of the line, leaving you in insert mode.
cwChanges the word the cursor is on from the cursor to the lowercase w end of the word.
rReplaces the character under the cursor. vi returns to command mode after the replacement is entered.
ROverwrites multiple characters beginning with the character currently under the cursor. You must use Esc to stop the overwriting.
sReplaces the current character with the character you type. Afterward, you are left in insert mode.
SDeletes the line the cursor is on and replaces with new text. After the new text is entered, vi remains in insert mode.

Copy and Past Commands:

You can copy lines or words from one place and then you can past them at another place using following commands:
CommandDescription
yyCopies the current line.
ywCopies the current word from the character the lowercase w cursor is on until the end of the word.
pPuts the copied text after the cursor.
PPuts the yanked text before the cursor.

Advanced Commands:

There are some advanced commands that simplify day-to-day editing and allow for more efficient use of vi:
CommandDescription
JJoin the current line with the next one. A count joins that many lines.
<<Shifts the current line to the left by one shift width.
>>Shifts the current line to the right by one shift width.
~Switch the case of the character under the cursor.
^GPress CNTRL and G keys at the same time to show the current filename and the status.
URestore the current line to the state it was in before the cursor entered the line.
uUndo the last change to the file. Typing 'u' again will re-do the change.
JJoin the current line with the next one. A count joins that many lines.
:fDisplays current position in the file in % and file name, total number of file.
:f filenameRenames current file to filename.
:w filenameWrite to file filename.
:e filenameOpens another file with filename.
:cd dirnameChanges current working directory to dirname.
:e #Use to toggle between two opened files.
:nIn case you open multiple files using vi, use :n to go to next file in the series.
:pIn case you open multiple files using vi, use :p to go to previous file in the series.
:NIn case you open multiple files using vi, use :N to go to previous file in the series.
:r fileReads file and inserts it after current line
:nr fileReads file and inserts it after line n.

Word and Character Searching:

The vi editor has two kinds of searches: string and character. For a string search, the / and ? commands are used. When you start these commands, the command just typed will be shown on the bottom line, where you type the particular string to look for.
These two commands differ only in the direction where the search takes place:
  • The / command searches forwards (downwards) in the file.
  • The ? command searches backwards (upwards) in the file.
The n and N commands repeat the previous search command in the same or opposite direction, respectively. Some characters have special meanings while using in search command and preceded by a backslash (\) to be included as part of the search expression.
CharacterDescription
^Search at the beginning of the line. (Use at the beginning of a search expression.)
.Matches a single character.
*Matches zero or more of the previous character.
$End of the line (Use at the end of the search expression.)
[Starts a set of matching, or non-matching expressions.
<Put in an expression escaped with the backslash to find the ending or beginning of a word.
>See the '<' character description above.
The character search searches within one line to find a character entered after the command. The f and F commands search for a character on the current line only. f searches forwards and F searches backwards and the cursor moves to the position of the found character.
The t and T commands search for a character on the current line only, but for t, the cursor moves to the position before the character, and T searches the line backwards to the position after the character.

Set Commands:

You can change the look and feel of your vi screen using the following :set commands. To use these commands you have to come in command mode then type :set followed by any of the following options:
CommandDescription
:set icIgnores case when searching
:set aiSets autoindent
:set noaiTo unset autoindent.
:set nuDisplays lines with line numbers on the left side.
:set swSets the width of a software tabstop. For example you would set a shift width of 4 with this command: :set sw=4
:set wsIf wrapscan is set, if the word is not found at the bottom of the file, it will try to search for it at the beginning.
:set wmIf this option has a value greater than zero, the editor will automatically "word wrap". For example, to set the wrap margin to two characters, you would type this: :set wm=2
:set roChanges file type to "read only"
:set termPrints terminal type
:set bfDiscards control characters from input

Running Commands:

The vi has the capability to run commands from within the editor. To run a command, you only need to go into command mode and type :! command.
For example, if you want to check whether a file exists before you try to save your file to that filename, you can type :! ls and you will see the output of ls on the screen.
When you press any key (or the command's escape sequence), you are returned to your vi session.

Replacing Text:

The substitution command (:s/) enables you to quickly replace words or groups of words within your files. Here is the simple syntax:
:s/search/replace/g
The g stands for globally. The result of this command is that all occurrences on the cursor's line are changed.

IMPORTANT:

Here are the key points to your success with vi:
  • You must be in command mode to use commands. (Press Esc twice at any time to ensure that you are in command mode.)
  • You must be careful to use the proper case (capitalization) for all commands.
  • You must be in insert mode to enter text.

Unix - Network Communication Utilities

When you work in a distributed environment then you need to communicate with remote users and you also need to access remote Unix machines.
There are several Unix utilities which are especially useful for users computing in a networked, distributed environment. This tutorial lists few of them:

The ping Utility:

The ping command sends an echo request to a host available on the network. Using this command you can check if your remote host is responding well or not.
The ping command is useful for the following:
  • Tracking and isolating hardware and software problems.
  • Determining the status of the network and various foreign hosts.
  • Testing, measuring, and managing networks.

Syntax:

Following is the simple syntax to use ping command:
$ping hostname or ip-address
Above command would start printing a response after every second. To come out of the command you can terminate it by pressing CNTRL + C keys.

Example:

Following is the example to check the availability of a host available on the network:
$ping google.com
PING google.com (74.125.67.100) 56(84) bytes of data.
64 bytes from 74.125.67.100: icmp_seq=1 ttl=54 time=39.4 ms
64 bytes from 74.125.67.100: icmp_seq=2 ttl=54 time=39.9 ms
64 bytes from 74.125.67.100: icmp_seq=3 ttl=54 time=39.3 ms
64 bytes from 74.125.67.100: icmp_seq=4 ttl=54 time=39.1 ms
64 bytes from 74.125.67.100: icmp_seq=5 ttl=54 time=38.8 ms
--- google.com ping statistics ---
22 packets transmitted, 22 received, 0% packet loss, time 21017ms
rtt min/avg/max/mdev = 38.867/39.334/39.900/0.396 ms
$
If a host does not exist then it would behave something like this:
$ping giiiiiigle.com
ping: unknown host giiiiigle.com
$

The ftp Utility:

Here ftp stands for File Transfer Protocol. This utility helps you to upload and download your file from one computer to another computer.
The ftp utility has its own set of UNIX like commands which allow you to perform tasks such as:
  • Connect and login to a remote host.
  • Navigate directories.
  • List directory contents
  • Put and get files
  • Transfer files as ascii, ebcdic or binary

Syntax:

Following is the simple syntax to use ping command:
$ftp hostname or ip-address
Above command would prompt you for login ID and password. Once you are authenticated, you would have access on the home directory of the login account and you would be able to perform various commands.
Few of the useful commands are listed below:
CommandDescription
put filenameUpload filename from local machine to remote machine.
get filenameDownload filename from remote machine to local machine.
mput file listUpload more than one files from local machine to remote machine.
mget file listDownload more than one files from remote machine to local machine.
prompt offTurns prompt off, by default you would be prompted to upload or download movies using mput or mget commands.
prompt onTurns prompt on.
dirList all the files available in the current directory of remote machine.
cd dirnameChange directory to dirname on remote machine.
lcd dirnameChange directory to dirname on local machine.
quitLogout from the current login.
It should be noted that all the files would be downloaded or uploaded to or from current directories. If you want to upload your files in a particular directory then first you change to that directory and then upload required files.

Example:

Following is the example to show few commands:
$ftp amrood.com
Connected to amrood.com.
220 amrood.com FTP server (Ver 4.9 Thu Sep 2 20:35:07 CDT 2009)
Name (amrood.com:amrood): amrood
331 Password required for amrood.
Password:
230 User amrood logged in.
ftp> dir
200 PORT command successful.
150 Opening data connection for /bin/ls.
total 1464
drwxr-sr-x   3 amrood   group       1024 Mar 11 20:04 Mail
drwxr-sr-x   2 amrood   group       1536 Mar  3 18:07 Misc
drwxr-sr-x   5 amrood   group        512 Dec  7 10:59 OldStuff
drwxr-sr-x   2 amrood   group       1024 Mar 11 15:24 bin
drwxr-sr-x   5 amrood   group       3072 Mar 13 16:10 mpl
-rw-r--r--   1 amrood   group     209671 Mar 15 10:57 myfile.out
drwxr-sr-x   3 amrood   group        512 Jan  5 13:32 public
drwxr-sr-x   3 amrood   group        512 Feb 10 10:17 pvm3
226 Transfer complete.
ftp> cd mpl
250 CWD command successful.
ftp> dir
200 PORT command successful.
150 Opening data connection for /bin/ls.
total 7320
-rw-r--r--   1 amrood   group       1630 Aug  8 1994  dboard.f
-rw-r-----   1 amrood   group       4340 Jul 17 1994  vttest.c
-rwxr-xr-x   1 amrood   group     525574 Feb 15 11:52 wave_shift
-rw-r--r--   1 amrood   group       1648 Aug  5 1994  wide.list
-rwxr-xr-x   1 amrood   group       4019 Feb 14 16:26 fix.c
226 Transfer complete.
ftp> get wave_shift
200 PORT command successful.
150 Opening data connection for wave_shift (525574 bytes).
226 Transfer complete.
528454 bytes received in 1.296 seconds (398.1 Kbytes/s)
ftp> quit
221 Goodbye.
$

The telnet Utility:

Many times you would be in need to connect to a remote Unix machine and work on that machine remotely. Telnet is a utility that allows a computer user at one site to make a connection, login and then conduct work on a computer at another site.
Once you are login using telnet, you can perform all the activities on your remotely connect machine. Here is example telnet session:
C:>telnet amrood.com
Trying...
Connected to amrood.com.
Escape character is '^]'.

login: amrood
amrood's Password: 
*****************************************************
*                                                   *
*                                                   *
*    WELCOME TO AMROOD.COM                          *
*                                                   *
*                                                   *
*****************************************************

Last unsuccessful login: Fri Mar  3 12:01:09 IST 2009
Last login: Wed Mar  8 18:33:27 IST 2009 on pts/10

   {  do your work }

$ logout
Connection closed.
C:>

The finger Utility:

The finger command displays information about users on a given host. The host can be either local or remote.
Finger may be disabled on other systems for security reasons.
Following are the simple syntax to use finger command:
Check all the logged in users on local machine as follows:
$ finger
Login     Name       Tty      Idle  Login Time   Office
amrood               pts/0          Jun 25 08:03 (62.61.164.115)
Get information about a specific user available on local machine:
$ finger amrood
Login: amrood                           Name: (null)
Directory: /home/amrood                 Shell: /bin/bash
On since Thu Jun 25 08:03 (MST) on pts/0 from 62.61.164.115
No mail.
No Plan.
Check all the logged in users on remote machine as follows:
$ finger @avtar.com
Login     Name       Tty      Idle  Login Time   Office
amrood               pts/0          Jun 25 08:03 (62.61.164.115)
Get information about a specific user available on remote machine:
$ finger amrood@avtar.com
Login: amrood                           Name: (null)
Directory: /home/amrood                 Shell: /bin/bash
On since Thu Jun 25 08:03 (MST) on pts/0 from 62.61.164.115
No mail.
No Plan.

DBT - Models

Models are where your developers spend most of their time within a dbt environment. Models are primarily written as a select statement and ...