Unix

What is Unix ?

The UNIX operating system is a set of programs that act as a link between the computer and the user.

The computer programs that allocate the system resources and coordinate all the details of the computer's internals is called the operating system or kernel.

Users communicate with the kernel through a program known as the shell. The shell is a command line interpreter; it translates commands entered by the user and converts them into a language that is understood by the kernel.

• Unix was originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna.
• There are various Unix variants available in the market. Solaris Unix, AIX, HP Unix and BSD are few examples. Linux is also a flavor of Unix which is freely available.
• Several people can use a UNIX computer at the same time; hence UNIX is called a multiuser system.
• A user can also run multiple programs at the same time; hence UNIX is called multitasking.

Unix Architecture:

Here is a basic block diagram of a UNIX system:

The main concept that unites all versions of UNIX is the following four basics:

• Kernel: The kernel is the heart of the operating system. It interacts with hardware and most of the tasks like memory management, tash scheduling and file management.
• Shell: The shell is the utility that processes your requests. When you type in a command at your terminal, the shell interprets the command and calls the program that you want. The shell uses standard syntax for all commands. C Shell, Bourne Shell and Korn Shell are most famous shells which are available with most of the Unix variants.
• Commands and Utilities: There are various command and utilities which you would use in your day to day activities. cp, mv, cat and grep etc. are few examples of commands and utilities. There are over 250 standard commands plus numerous others provided through 3rd party software. All the commands come along with various optional options.
• Files and Directories: All data in UNIX is organized into files. All files are organized into directories. These directories are organized into a tree-like structure called the filesystem.

System Bootup:

If you have a computer which has UNIX operating system installed on it, then you simply need to turn on its power to make it live.

As soon as you turn on the power, system starts booting up and finally it prompts you to log into the system, which is an activity to log into the system and use it for your day to day activities.

Login Unix:

When you first connect to a UNIX system, you usually see a prompt such as the following:

login:

To log in:

1. Have your userid (user identification) and password ready. Contact your system administrator if you don't have these yet.
2. Type your userid at the login prompt, then press ENTER. Your userid is case-sensitive, so be sure you type it exactly as your system administrator instructed.
3. Type your password at the password prompt, then press ENTER. Your password is also case-sensitive.
4. If you provided correct userid and password then you would be allowed to enter into the system. Read the information and messages that come up on the screen something as below.

login : amrood
amrood's password:
Last login: Sun Jun 14 09:32:32 2009 from 62.61.164.73
$

You would be provided with a command prompt ( sometime called $ prompt ) where you would type your all the commands. For example to check calendar you need to type cal command as follows:

$ cal
     June 2009
Su Mo Tu We Th Fr Sa
    1  2  3  4  5  6
 7  8  9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30

$

Change Password:

All Unix systems require passwords to help ensure that your files and data remain your own and that the system itself is secure from hackers and crackers. Here are the steps to change your password:

1. To start, type passwd at command prompt as shown below.
2. Enter your old password the one you're currently using.
3. Type in your new password. Always keep your password complex enough so that no body can guess it. But make sure, you remember it.
4. You would need to verify the password by typing it again.

$ passwd
Changing password for amrood
(current) Unix password:******
New UNIX password:*******
Retype new UNIX password:*******
passwd: all authentication tokens updated  successfully

$

Note: I have put stars (*) just to show you the location where you would need to enter the current and new passwords otherwise at your system, it would not show you any character when you would type.

Listing Directories and Files:

All data in UNIX is organized into files. All files are organized into directories. These directories are organized into a tree-like structure called the filesystem.

You can use ls command to list out all the files or directories available in a directory. Following is the example of using ls command with -l option.

$ ls -l
total 19621
drwxrwxr-x  2 amrood amrood      4096 Dec 25 09:59 uml
-rw-rw-r--  1 amrood amrood      5341 Dec 25 08:38 uml.jpg
drwxr-xr-x  2 amrood amrood      4096 Feb 15  2006 univ
drwxr-xr-x  2 root   root        4096 Dec  9  2007 urlspedia
-rw-r--r--  1 root   root      276480 Dec  9  2007 urlspedia.tar
drwxr-xr-x  8 root   root        4096 Nov 25  2007 usr
-rwxr-xr-x  1 root   root        3192 Nov 25  2007 webthumb.php
-rw-rw-r--  1 amrood amrood     20480 Nov 25  2007 webthumb.tar
-rw-rw-r--  1 amrood amrood      5654 Aug  9  2007 yourfile.mid
-rw-rw-r--  1 amrood amrood    166255 Aug  9  2007 yourfile.swf

$

Here enteries starting with d..... represent directories. For example uml, univ and urlspedia are directories and rest of the enteries are files.

Who Are You?

While you're logged in to the system, you might be willing to know : Who am I?

The easiest way to find out "who you are" is to enter the whoami command:

$ whoami
 amrood

$

Try it on your system. This command lists the account name associated with the current login. You can try who am i command as well to get information about yourself.

Who is Logged In?

Sometime you might be interested to know who is logged in to the computer at the same time.

There are three commands are available to get you this information, based on how much you'd like to learn about the other users: users, who, and w.

$ users
 amrood bablu qadir

$ who
amrood ttyp0 Oct 8 14:10 (limbo)
bablu  ttyp2 Oct 4 09:08 (calliope)
qadir  ttyp4 Oct 8 12:09 (dent)

$

Try w command on your system to check the output. This would list down few more information associated with the users logged in the system.

Logging Out:

When you finish your session, you need to log out of the system to ensure that nobody else accesses your files while masquerading as you.

To log out:

1. Just type logout command at command prompt, and the system will clean up everything and break the connection

System Shutdown:

The most consistent way to shut down a Unix system properly via the command line is to use one of the following commands:

Command	Description
halt	Brings the system down immediately.
init 0	Powers off the system using predefined scripts to synchronize and clean up the system prior to shutdown
init 6	Reboots the system by shutting it down completely and then bringing it completely back up
poweroff	Shuts down the system by powering off.
reboot	Reboots the system.
shutdown	Shuts down the system.

You typically need to be the superuser or root (the most privileged account on a Unix system) to shut down the system, but on some standalone or personally owned Unix boxes, an administrative user and sometimes regular users can do so.

ERP

What is ERP?

In accounting, ERP is the acronym for enterprise resource planning. ERP could be described as a database software package that supports all of a business's processes and operations including manufacturing, marketing, financial, human resources, and so on. In other words, the goal of ERP is to have one integrated system for the entire company.

The integration of all of a company's information from all departments, processes, operations, etc. requires that an ERP system be very sophisticated. This in turn requires a company to commit considerable resources for planning, training, and implementing an ERP system.

Here are 7 terms you keep hearing many times in a conversation related to ERP -

[1] Human Capital Management (HCM) :: This refers to the support system responsible for managing employee data right from recruitment till retirement, in any organization. It includes tracking, managing and analyzing the skillsets, department, education, roles, achievements, cost to company and other benefits of each and every employee.

[2] Financial Management :: This is the process of managing cash inflow and outflow, advance budgeting, tracking operational expenses, controlling and decision-making

[3] Customer Relationship Management (CRM) :: CRM is one of the most commonly used and important terms you come across. CRM refers to the methodology or processes involved in acquiring new customers and managing existing customers. The aim is to ensure smooth relationship with customers through effective sales and services support, helpdesk, lead management,  field sales tracking and reports.

[4] Supply Chain Management (SCM) :: SCM deals with management of inventory, procurements, packaging, distribution, sales and all other necessary steps involved in supply of any product to the customer.

[5] Production Control :: All activities involved in controlling the production process are collectively termed as production control. Managing orders, material planning, production floor management, quality control etc. are some of the important activities in production control.

[6] Business Intelligence (BI) :: This is the latest buzz word! BI refers to the set of tools that facilitate the senior management to make better decisions by providing a wide range of data and information. The information is provided in graphical or tabular formats. Typically, BI acts like a decision support system.

[7] Product Life Cycle :: In engineering terms, product life cycle refers to the different stages a product goes through right from ideation, design, prototyping, testing, manufacturing to packaging and distribution.  In marketing terms, this refers to the time period during which the product is available in the market for sale. The different phases involved are introduction stage, growth stage, maturity, market saturation and decline stage.

Email Task - Sample create

Creating a sample Email Task:

You can create Email tasks in the Task Developer, Worklet Designer, and Workflow Designer.

1. In the Task Developer, click Tasks > Create.

 

2. The Create Task dialog box appears. Select an Email task and enter a name for the task. Click Create. 

 

3. The Workflow Manager creates an Email task in the workspace, Click Done. 

 

4. Select the email task and Double-click on it to edit --> Go to the Properties tab. 

 

a) Enter the email address of the mail recipient in the Email User Name field.

b) Enter the subject of the email in the Email Subject field Or you can leave this field blank.

c) Click the Open button in the Email Text field to open the Email Editor.

5. Enter the text of the email message in the Email Editor. You can use service, service process, workflow, and worklet variables in the email text. Or, you can leave the Email Text field blank.

  

Note:

1. You can incorporate format tags and email variables in a post-session email. However, you cannot add them to an Email task outside the context of a session.

2. The Integration Service does not limit the type or size of attached files. However, since large attachments can cause problems with the email system, avoid attaching excessively large files, such as session logs generated using verbose tracing. The Integration Service generates an error message in the email if an error occurs attaching the file.

The following table describes the email variables that you can use in a post-session email:

Email Variable	Description
%a<filename>	Attach the named file. The file must be local to the Integration Service. The following file names are valid: %a<c:\data\sales.txt> or %a</users/john/data/sales.txt>. The email does not display the full path for the file. Only the attachment file name appears in the email. Note: The file name cannot include the greater than character (>) or a line break.
%b	Session start time.
%c	Session completion time.
%d	Name of the repository containing the session.
%e	Session status.
%g	Attach the session log to the message.
%i	Session elapsed time.
%l	Total rows loaded.
%m	Name of the mapping used in the session.
%n	Name of the folder containing the session.
%r	Total rows rejected.
%s	Session name.
%t	Source and target table details, including read throughput in bytes per second and write throughput in rows per second.
%u	Repository user name.
%v	Integration Service name.
%w	Workflow name.
%y	Session run mode (normal or recovery).
%z	Workflow run instance name.
Note: The Integration Service ignores %a, %g, and %t when you include them in the email subject. Include these variables in the email message only.

The following table lists the format tags you can use in an Email task:

Formatting	Format Tag
tab	\t
new line	\n

6. Click OK twice to save the changes.

Now we have create re-usable command task and now we need to use this in session inside the workflow. (Open the workflow or worklet where you want to use this command task).

7. Select the session in the workspace, double click to edit and go to the components tab.

 

Note:

1. On Success E-Mail: Send an email alter on session succeeded.
2. On Failure E-Mail: Send an email alter on session failure.

8. Select Task as On Success/Failure E-Mail, Type as Reusable/Non-reusable and Click the Open button in the Email Text field to open the Email Editor

9. Click on 'OK'.

 

Note:

If you want to modify anything in the email click on edit icon and modify it.

 

Email Task

We can send email to designated recipients when the Integration Service runs a workflow.

For Example: If you want to track how long a session takes to complete, you can configure the session to send an email containing the time and date the session starts and completes. Or, if you want the Integration Service to notify you when a workflow suspends, you can configure the workflow to send email when it suspends.

To send email when the Integration Service runs a workflow, perform the following steps:

• Configure the Integration Service to send email. Before creating Email tasks, we need configure the Integration Service to send email (Administrator will config).
• Create Email tasks. Before you can configure a session or workflow to send email, you need to create an Email task.
• Configure sessions to send post-session email. You can configure the session to send an email when the session completes or fails. You create an Email task and use it for post-session email.
• Configure workflows to send suspension email. You can configure the workflow to send an email when the workflow suspends. You create an Email task and use it for suspension email.

The Integration Service sends the email based on the locale set for the Integration Service process running the session.

We can use parameters and variables in the email user name, subject, and text. For Email tasks and suspension email, you can use service, service process, workflow, and worklet variables. For post-session email, you can use any parameter or variable type that you can define in the parameter file. For example, you can use the $PMSuccessEmailUser or $PMFailureEmailUser service variable to specify the email recipient for post-session email.

Working with Email Tasks

We can send email during a workflow using the Email task on the Workflow Manager. We can create reusable Email tasks in the Task Developer for any type of email. Or, we can create non-reusable Email tasks in the Workflow and Worklet Designer.

Use Email tasks in any of the following locations:

• Session properties. You can configure the session to send email when the session completes or fails.
• Workflow properties. You can configure the workflow to send email when the workflow is interrupted.
• Workflows or worklets. You can include an Email task anywhere in the workflow or worklet to send email based on a condition you define.

Using Email Tasks in a Workflow or Worklet

We can use Email tasks anywhere in a workflow or worklet. For example, we might configure a workflow to send an email if a certain number of rows fail for a session.

For example, we may have a Session task in the workflow and you want the Integration Service to send an email if more than 20 rows are dropped. To do this, you create a condition in the link, and create a non-reusable Email task. The workflow sends an email if the session fails more than 20 rows are dropped.

Working with Post-Session Email

We can configure a session to send email when it fails or succeeds. We can create separate email tasks for success and failure email.

The Integration Service sends post-session email at the end of a session, after executing post-session shell commands or stored procedures. When the Integration Service encounters an error sending the email, it writes a message to the Log Service. It does not fail the session.

We can specify a reusable Email that task you create in the Task Developer for either success email or failure email. Or, we can create a non-reusable Email task for each session property. When you create a non-reusable Email task for a session, you cannot use the Email task in a workflow or worklet.

We cannot specify a non-reusable Email task you create in the Workflow or Worklet Designer for post-session email.

We can use parameters and variables in the email user name, subject, and text. Use any parameter or variable type that you can define in the parameter file. For example, we can use the service variable $PMSuccessEmailUser or $PMFailureEmailUser for the email recipient. Ensure that you specify the values of the service variables for the Integration Service that runs the session. You can also enter a parameter or variable within the email subject or text, and define it in the parameter file.

Email Variables and Format Tags

We can use email variables and format tags in an email message for post-session emails. We can use some email variables in the subject of the email. With email variables, we can include important session information in the email, such as the number of rows loaded, the session completion time, or read and write statistics. You can also attach the session log or other relevant files to the email. Use format tags in the body of the message to make the message easier to read.

Note: The Integration Service does not limit the type or size of attached files. However, since large attachments can cause problems with the email system, avoid attaching excessively large files, such as session logs generated using verbose tracing. The Integration Service generates an error message in the email if an error occurs attaching the file.

Tasks Overview

We can create reusable tasks in the Task Developer. Or, create and add tasks in the Workflow or Worklet Designer as you develop the workflow.

The following table summarizes workflow tasks available in Workflow Manager:

Task Name	Tool	Reusable	Description
Assignment	Workflow Designer  Worklet Designer	No	Assigns a value to a workflow variable.
Command	Task Developer Workflow Designer  Worklet Designer	Yes	Specifies shell commands to run during the workflow. You can choose to run the Command task if the previous task in the workflow completes.
Control	Workflow Designer  Worklet Designer	No	Stops or aborts the workflow.
Decision	Workflow Designer  Worklet Designer	No	Specifies a condition to evaluate in the workflow. Use the Decision task to create branches in a workflow.
Email	Task Developer Workflow Designer  Worklet Designer	Yes	Sends email during the workflow.
Event-Raise	Workflow Designer  Worklet Designer	No	Represents the location of a user-defined event. The Event-Raise task triggers the user-defined event when the Integration Service runs the Event-Raise task.
Event-Wait	Workflow Designer  Worklet Designer	No	Waits for a user-defined or a predefined event to occur. Once the event occurs, the Integration Service completes the rest of the workflow.
Session	Task Developer Workflow Designer  Worklet Designer	Yes	Set of instructions to run a mapping.
Timer	Workflow Designer  Worklet Designer	No	Waits for a specified period of time to run the next task.

The Workflow Manager validates tasks attributes and links. If a task is invalid, the workflow becomes invalid. Workflows containing invalid sessions may still be valid.

Informatica Tips

Informatica Performance Improvement Tips

Following standards/guidelines can improve the overall performance:

• Use Source Qualifier if the Source tables reside in the same schema.
• Make use of Source Qualifer  "Filter" Properties if the Source type is Relational.
• If the subsequent sessions are doing lookup on the same table, use persistent cache in the first session. Data remains in the Cache and available for the subsequent session for usage.
• Use flags as integer, as the integer comparison is faster than the string comparison.
• Use tables with lesser number of records as master table for joins.
• While reading from Flat files, define the appropriate data type instead of reading as String and converting.
• Have all Ports that are required connected to Subsequent Transformations else check whether we can remove these ports.
• Suppress ORDER BY using the '--' at the end of the query in Lookup Transformations.
• Minimize the number of Update strategies.
• Group by simple columns in transformations like Aggregate, Source Qualifier.
• Use Router transformation in place of multiple Filter transformations.
• Turn off the Verbose Logging while moving the workflows to Production environment.
• For large volume of data drop index before loading and recreate indexes after load.
• For large of volume of records Use Bulk load Increase the commit interval to a higher value large volume of data.
• Set 'Commit on Target' in the sessions.

UNIX COMMANDS 1

FILE COMMANDS

touch - Create a new file.

• Usage: touch <filename>

cp - Copy files.

• Usage: cp [options] <source-filename> <destination-filename>
• cp [options] <source-filepath > <destination filepath>
• Options: -r recursively copy directory structures.

mv - Move or Rename files or directories.

• Usage: mv [options] <old-filepath> <new-filepath>
• mv [options] <old-filename> <new-filename>
• Options: -i query user for confirmation.

rm - Remove files.

• Usage: rm [options] <filname>
• Options: -r recursively remove directory structures. 
• -i query user for confirmation. 

cat - View complete file content.

• cat <filename>

more - View file contents in sections determined by the size of the terminal.

• Usage: more <filename>

less - View file contents in sections determined by the size of the terminal.

• Has more options and search features than more.
• Usage: less [options] <filename>

compress - Reduces the size of the file. and adds the extension .Z

• Usage: compress <filename>.

uncompress

• Restores a compressed file.
• Usage: uncompress <filename>
• Usage: zcat <filename>