Mox Diamond

IRIX: CHECK FOR NEW CONFIGS

2006-05-11T14:08:00.000+08:00

SGI loves to try and simplify your life with chkconfig switches to toggle various services on
and off. After each upgrade, DOUBLE check the chkconfig switches. If something doesn't
work all of the sudden check here.

For instance, tape drives in IRIX are disabled by a default install. To enable a tape subsystem:

# chkconfig ts on

# /etc/init.d/ts start

KILLING MORE USERS

2006-05-05T12:02:00.001+08:00

To kill all processes of a particular user from root at unix prompt type:

# kill -9 `ps -fu username |awk '{ print $2 }'|grep -v PID`

We can also use the username as an argument and pass it from command line, if this command is put as a script.

CLEANING DIRECTORIES

2006-05-05T12:02:00.000+08:00

The creation of many temporary files in Unix during compilations, occupies a lot of memory space. This can be got rid of by using a simple script.

find $1 $ -name a.out -o -name '*.o' -o -name 'core' -o -name '*.ii' -o -name '*.ti' -o -name '*.class' -o -name '*.pur' $ -exec rm {} \;

Save the above line in a file and run this file after changing permissions. For example, if the file containing the above code has the name 'clean', then:

example% clean

will remove all the files specified in the script in the directory and all other subdirectories within it. You can add or remove any number of files in the script, to suit your needs.

GREP TEXT NOT BINARY

2006-05-05T12:01:00.000+08:00

In some directories such as /etc you have a mix of file types. You may want to grep out a string from one of the files but don't want to worry about the binaries, data, etc. To accomplish
this, searching only text files do this:

grep `file * | egrep 'script|text' | awk -F: '{print $1}'`

REBOOTING BECAUSE OF FORK BOMBS

2006-05-05T11:59:00.000+08:00

There is nothing more frustrating for an Administrator who has to reboot system due to fork bomb

(the number of processes in the system reaches the maximum limit when a user, even a superuser, tries to execute some command, the system will respond with Vfork failed)

In Solaris under SPARC, this can be controlled by specifying a line in /etc/system

set maxuprc=64

And reboot the system. Now a user can have maximum of 64 processes under his ownership. By default the 'maxuprc' value is 16*maxusers - 5 where 'maxusers' is another tunable
parameter in /etc/system

Caution : You should have a backup of /etc/system file before you make the changes. So that you can revert back to old system file using boot -a option in case of
inconsistent system file.

Get the hidden files

2006-04-14T01:53:00.000+08:00

A safe way of grabbing all "hidden" files is to use '.??*' rather than '.*' since this will only match 3 or more characters. Admittedly, this will miss any hidden files that are only a single character after the ., but it will always miss '.' & '..', which is probably more important...

Building a Linux supercomputer using SSH and PVM

2006-04-12T02:20:00.000+08:00

If you have a couple of old Linux boxes sitting around, then you've got the makings of a supercomputer. Dust them off, install Secure Shell (SSH) and Parallel Virtual Machine (PVM), and start your complex algorithms.

All right, it's not quite as simple as that. PVM handles only the messaging between the machines. You must write your own programs to actually do anything.

First, network your PCs and set up NFS on each. I'm not going to go into detail because most Linux distributions take care of everything for you. With Debian, for example, simply connect a cable between your new PC and your network switch, stick in your installation CD, switch the PC on, and follow the prompts. If you need more information, take a look at the Linux.com how-tos on networking and NFS.

Now you can start setting up your PCs as a single supercomputer. In order for them to work as one, you need a single home directory -- hence, the need for NFS. Choose the machine that hosts the home directory and edit /etc/exports. If the file isn't there, then you must set up the PC as an NFS server -- check your distro's documentation. If you're using Debian, simply type sudo apt-get install nfs-kernel-server.

Now add in the details for each of the hosts where you want the common home directory. In this example, I'm exporting my home directory from polydamas (my NFS server) to three hosts: acamas, cassandra, and hector:

/home acamas(rw)
/home cassandra(rw)
/home hector(rw)

You can see the full list of possible options when exporting by typing man exports on the command line. Don't forget to add all hosts into your /etc/hosts file too.

Now either reboot your NFS server or check your distro's documentation for the relevant command that lets your hosts see the exports. On Debian, the command is exportfs –a.

You can now turn to your NFS client hosts and set them up so that they use the home directory that you're exporting from the NFS server. If you feel that exporting the whole /home is overkill, simply export the home directory for the user that you want to be able to run the supercomputer.

If you're confident that everything is going to work, then just move the current /home somewhere safe (don't forget to rename it /home_old). Run mkdir /home, then edit your /etc/fstab file so that it contains the details for the NFS server:

polydamas:/home /home nfs rw,sync 0 0

Make sure that your /etc/hosts file contains the IP address for your server, then either reboot or reload the NFS data:

sudo /etc/init.d/mountnfs.sh

If you're not quite that brave, mount the directories manually before you commit to automating the process fully.

Set up SSH

Now that you have a common /home, you need SSH. Chances are, your Linux distribution came bundled with SSH. Each of my machines uses Debian, which loads OpenSSH automatically.

Set up SSH so that you don't have to enter a password each time you use it. For more information, take a look at Joe Barr's "CLI Magic: OpenSSH" and Joe 'Zonker' Brockmeier's "CLI Magic: More on SSH."

You'll find yourself benefiting from a common /home directory. Instead of having to set up an authorized_keys2 file on each machine, you only have to do it once on the NFS server:

ssh-keygen -t dsa
cat .ssh/id_dsa.pub > .ssh/authorized_keys2

If you just want to be able to run processes in parallel, then you're ready to go.

Looking for more? You might want to create programs that use the resources of all of your machines. Let's say you have three Linux boxes connected to your network, and you have three Linux scripts sitting in your home directory that you need to process. Simply run each one via SSH:

#Run the files on the machines
ssh bainm@acamas ./batch_file1 &
ssh bainm@cassandra ./batch_file2 &
ssh bainm@hector ./batch_file3 &

You can distribute work around your network easily using this technique. Although useful, the scripts don't provide any feedback. You must check each machine manually for the progress of each file before you continue with your computations. However, you can add feedback by making each of the distributed files write its results back to a common file on your home directory.

In this next example, you can calculate pi to any number of decimal places:

#File name: calc_pi
RESULT_FILE=$1
DECIMAL_PLACES=$2
RESULT=$(echo "scale=$DECIMAL_PLACES;4*(4*a(1/5)-a(1/239))"|bc -l)
echo "$(uname -n) Pi: $RESULT" >> $RESULT_FILE

I calculated pi = 4 x ( 4arctan(1/5) - arctan(1/239) because that's what I was taught in college; there are other ways.

Now tell each of your machines to run a process:

ssh bainm@acamas . ./calc_pi pi_results 10 &
ssh bainm@cassandra . ./calc_pi pi_results 20 &
ssh bainm@hector . ./calc_pi pi_results 30 &

After a couple of seconds, a new file (pi_results) contains this code:

acamas Pi: 3.1415926532
cassandra Pi: 3.14159265358979323848
hector Pi: 3.141592653589793238462643383272

Let PVM do the work for you

While this is useful to know, you're probably better off using software that does all the work for you. If you're happy using C, C++, or Fortran, then PVM may be for you. Download it from the PVM Web site, or check if you can load it using your distro's methods. For instance, use this command on Debian:

sudo apt-get install pvm

Install PVM on all of the machines, then log on to the computer you want to use as your central host. If it's not your NFS server, remember to generate a key for it and add it to the .ssh/authorized_keys2 file. Once you start PVM by typing pvm on the command line, you can start adding hosts. Don't worry about starting PVM on the other machines -- that's done automatically when you add a host.

$ pvm
pvm> add acamas
add acamas
1 successful
                   HOST     DTID
                 acamas    80000
pvm>

If that seems a bit long-winded, then list your hosts in a file and get PVM to read it:

$ pvm hostfile

Type conf to check which hosts are loaded:

pvm> conf
conf
4 hosts, 1 data format
                   HOST     DTID     ARCH   SPEED       DSIG
              cassandra    40000    LINUX    1000 0x00408841
                 acamas    80000    LINUX    1000 0x00408841
                 hector    c0000    LINUX    1000 0x00408841
              polydamas   100000    LINUX    1000 0x00408841
pvm>

Type quit to exit PVM and leave it running in the background. Type halt to shut down PVM.

Now you can create a program that uses PVM. You need the PVM source code. As always, check the details for your distro -- usually, you can get the files easily. For example, Debian uses this command:

sudo apt-get install pvm-dev

You need the files on only one of your machines; thanks to the common home directory, you can use any of them. Create a directory called ~/pvm3/examples and look for a file called examples.tar.gz -- you'll probably find it in /usr/share/doc/pvm. Unpack this into the directory you just created. You'll see a set of self-explanatory files that show you exactly how to program with PVM. Start with master1.c and its associated file slave1.c. Examine the source code to see exactly how the process operates. Use this code to see it in action type:

aimk master1 slave1

aimk -- the program for compiling your PVM programs -- creates your executables and places them in ~/pvm3/bin/LINUX. Simply change to this directory and type master1. Assuming you're on the machine where you're running PVM, you should see something like this:

$ master1
Spawning 12 worker tasks ... SUCCESSFUL
I got 1300.000000 from 7; (expecting 1300.000000)
I got 1500.000000 from 8; (expecting 1500.000000)
I got 100.000000 from 1; (expecting 100.000000)
I got 700.000000 from 4; (expecting 700.000000)
I got 1100.000000 from 0; (expecting 1100.000000)
I got 1700.000000 from 9; (expecting 1700.000000)
I got 1900.000000 from 10; (expecting 1900.000000)
I got 2100.000000 from 11; (expecting 2100.000000)
I got 1100.000000 from 6; (expecting 1100.000000)
I got 900.000000 from 5; (expecting 900.000000)
I got 300.000000 from 2; (expecting 300.000000)
I got 500.000000 from 3; (expecting 500.000000)

If you're a Fortran programmer, don't worry -- there are some examples for you as well. Other languages don't offer examples, but look on the PVM Web site for support for numerous languages, including Perl, Python, and Java. You'll also find various applications to help with PVM, such as XPVM for a graphical interface.

Accessing Positional Variables

2006-04-04T01:02:00.000+08:00

Accessing positional variables from the left is simple using Bourne, bash, or KornShell93.
ksh88 will not work and csh's syntax I won't cover it.

You can use the form of ${*:1:1} where the 1st 1 is the number of which positional variable you want to access and the 2nd is the number of variables following it that you want.

So, if you type:

set a b c d, echo ${*:3:1}

will give you c.

You can be sneakier using a looping construct. This will give you the positional parameters in order:

for i in 1 2 3 4
do
echo ${*:$a:1}
done

Suppose you want to reference the 2nd to last positional parameter.
Try ${*:$#-1:1}.

${*:$#:1} gives you the last one. Even sneakier, here's the positional variables in reverse order.

for i in 1 2 3 4
do
echo ${*:$#-$i:1}
done

Beware: different shells act slightly different. Bash, for the above example of set a b c d, gives d for echo ${*:$#-5:1} while ksh gives ksh.

Linux Renaming Sets of Files

2006-04-03T10:14:00.000+08:00

Easy way to rename a set of files in Linux:

*** Need not use any SHELL script
*** or any other prog.
*** simply use the rename command

**SYNTAX:**
% rename . .

eg: To rename all .jpg to .gif use:

% rename .jpg .gif *

To rename only .jpg with starting
letter as t to .gif use:

% rename .jpg .gif t*

Separate Shell Command history

2006-04-03T10:12:00.000+08:00

One other suggestion on how to separate shell command history files by terminal (It works on
HP-UX):

Change the permissions of your
.sh_history to 000.

$ chmod 000 .sh_history

After this is done, the various shells that may exist will not save every command to the file
thus keeping the history in memory. Since every memory is different, as every shell is run
in a different process, every shell will have its own history (in memory) of commands.

NFS between Solaris & Linux

2006-03-30T11:50:00.000+08:00

If you receive error messages such as "unknown version" when attempting to mount a Linux based NFS server from Solaris, you probably have an incompatibility between the NFS versions
running on both of them. Linux uses version 2, while Solaris uses version 3. In order to get the machines to communicate, you have to use the vers option on the Solaris machine as follows:

mount -o vers=2 nfsserver:/remotedir /localdir

How To Set Up Database Replication In MySQL

2006-03-30T11:40:00.000+08:00

This tutorial describes how to set up database replication in MySQL. MySQL replication allows you to have an exact copy of a database from a master server on another server (slave), and all updates to the database on the master server are immediately replicated to the database on the slave server so that both databases are in sync. This is not a backup policy because an accidentally issued DELETE command will also be carried out on the slave; but replication can help protect against hardware failures though.

In this tutorial I will show how to replicate the database exampledb from the master with the IP address 192.168.0.100 to a slave. Both systems (master and slave) are running Debian Sarge; however, the configuration should apply to almost all distributions with little or no modification.

Both systems have MySQL installed, and the database exampledb with tables and data is already existing on the master, but not on the slave.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Configure The Master

First we have to edit /etc/mysql/my.cnf. We have to enable networking for MySQL, and MySQL should listen on all IP addresses, therefore we comment out these lines (if existant):

#skip-networking
#bind-address            = 127.0.0.1

Furthermore
we have to tell MySQL for which database it should write logs (these logs are used by the slave to see what has changed on the master),
which log file it should use, and we have to specify that this MySQL

log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db=exampledb
server-id=1

Then we restart MySQL:

/etc/init.d/mysql restart

Then we log into the MySQL database as root and create a user with replication privileges:

mysql -u root -p
Enter password:

Now we are on the MySQL shell.

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY ''; (Replace with a real password!)
FLUSH PRIVILEGES;

Next (still on the MySQL shell) do this:

USE exampledb;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

The last command will show something like this:

+---------------+----------+--------------+------------------+
| File          | Position | Binlog_do_db | Binlog_ignore_db |
+---------------+----------+--------------+------------------+
| mysql-bin.006 | 183      | exampledb    |                  |
+---------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Write down this information, we will need it later on the slave!

Then leave the MySQL shell:

quit;

There are two possibilities to get the existing tables and data from exampledb from the master to the slave. The first one is to make a database dump, the second one is to use the LOAD DATA FROM MASTER; command on the slave. The latter has the disadvantage the the database on the master will be locked during this operation, so if you have a large database on a high-traffic production system, this is not what you want, and I recommend to follow the first method in this case. However, the latter method is very fast, so I will describe both here.

If you want to follow the first method, then do this:

mysqldump -u root -p --opt exampledb > exampledb.sql (Replace with the real password for the MySQL user root! Important: There is no space between -p and !)

This will create an SQL dump of exampledb in the file exampledb.sql. Transfer this file to your slave server!

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.

Finally we have to unlock the tables in exampledb:

mysql -u root -p
Enter password:
UNLOCK TABLES;
quit;

Now the configuration on the master is finished. On to the slave...

2 Configure The Slave

On the slave we first have to create the database exampledb:

mysql -u root -p
Enter password:
CREATE DATABASE exampledb;
quit;

If you have made an SQL dump of exampledb on the master and have transferred it to the slave, then it is time now to import the SQL dump into our newly created exampledb on the slave:

mysql -u root -p exampledb < /path/to/exampledb.sql (Replace with the real password for the MySQL user root! Important: There is no space between -p and !)

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.

Now we have to tell MySQL on the slave that it is the slave, that the master is 192.168.0.100, and that the master database to watch is exampledb. Therefore we add the following lines to /etc/mysql/my.cnf:

server-id=2
master-host=192.168.0.100
master-user=slave_user
master-password=secret
master-connect-retry=60
replicate-do-db=exampledb

Then we restart MySQL:

/etc/init.d/mysql restart

If you have not imported the master exampledb with the help of an SQL dump, but want to go the LOAD DATA FROM MASTER; way, then it is time for you now to get the data from the master exampledb:

mysql -u root -p
Enter password:
LOAD DATA FROM MASTER;
quit;

If you have phpMyAdmin installed on the slave you can now check if all tables/data from the master exampledb is also available on the slave exampledb.

Finally, we must do this:

mysql -u root -p
Enter password:
SLAVE STOP;

In the next command (still on the MySQL shell) you have to replace the values appropriately:

CHANGE MASTER TO MASTER_HOST='192.168.0.100', MASTER_USER='slave_user', MASTER_PASSWORD='', MASTER_LOG_FILE='mysql-bin.006', MASTER_LOG_POS=183;

MASTER_HOST is the IP address or hostname of the master (in this example it is 192.168.0.100).
MASTER_USER is the user we granted replication privileges on the master.
MASTER_PASSWORD is the password of MASTER_USER on the master.
MASTER_LOG_FILE is the file MySQL gave back when you ran SHOW MASTER STATUS; on the master.
MASTER_LOG_POS is the position MySQL gave back when you ran SHOW MASTER STATUS; on the master.

Now all that is left to do is start the slave. Still on the MySQL shell we run

START SLAVE;
quit;

That's it! Now whenever exampledb is updated on the master, all changes will be replicated to exampledb on the slave. Test it!

Links

MySQL: http://www.mysql.com

My sysadmin toolbox

2006-03-26T14:50:00.000+08:00

Torsmo

Torsmo is a desktop system monitoring tool, and one of the best I have ever used.

Torsmo differs from other system monitors, such as GKrellM, in that it does not spawn a new window, but instead renders text directly to your desktop. It can display almost anything about your system, including uptime, current CPU usage, network activity, hard drive usage, memory usage, and swap usage. The program's developers wrote it to use as little of your system's resources as possible, and it does a good job of this.

You can configure what torsmo displays through its configuration file, normally found in your home directory as .torsmorc. You can look at my configuration file at http://realfolkblues.org/torsmorc.

ImageMagick

ImageMagick makes it easy to perform many operations on images directly from the command line. Among its many useful tools, identify is used to display information about an image, import can save any window on an X server to an image file, and convert can convert an image to almost any format with a single command.

You can use identify to show detailed information about a photo or image by running identify imagename . For example:

jon@gimli:~$ identify /media/pics/Group-Photo.jpg
/media/pics/Group-Photo.jpg JPEG 819x614 DirectClass 371kb 0.050u 0:01

I often use import to take a screenshot of my desktop. For example, to save a screenshot of your desktop to your home directory as a PNG image named screenshot.png, run import -window root $HOME/screenshot.png. ImageMagick will save the screenshot in the image format specified by the file extension.

Using convert to convert an image from one format to another could not be easier -- just run convert imagename.png imagename.jpg . Again, ImageMagick takes the format from the extension, so you don't need to give it an additional option to specify the new format.

Aterm

While KDE and GNOME come with their own terminal applications, these applications do much more than I need. Aterm, on the other hand, is a simple terminal program with fewer features, so it appears almost instantly when you start it. Though it's not as bulky as other terminal emulators, Aterm does have many useful options, which you can read about by typing aterm --help at the command line.

I like to start Aterm with a black background, white text, a 1,000-line history buffer, and display all text using the font "drift" from the "artwiz" font family. To get this, use aterm -bg black -fg white -sl 1000 -fn "-artwiz-drift-*".

Root-tail

Sometimes I use tail -f to monitor logfiles for changes. While useful, it's awkward to have a terminal window open all the time to monitor a logfile. Root-tail provides an excellent alternative by displaying logfiles as text rendered on your desktop in whatever font and color you specify. It also updates the text on your desktop at the interval you specify.

To use root-tail, just run root-tail filename to monitor a file. Root-tail has many useful options, which you can see by typing root-tail --help, or just read its man page.

Quod Libet

Out of the hordes of music players available for Linux, Quod Libet is my favorite. One of the things I like about Quod Libet is its ability to make playlists based on regular expressions. You can operate the player from the command line by running the program with an argument, making it simple to set up hotkeys with KDE to control the player. For instance, if I go to the KDE Control Center hotkey section and add a hotkey such as Control-Alt-X to run the command quodlibet --play, I can then simply press Control-Alt-X to cause Quod Libet to play music. See all of the command line arguments that are available by running quodlibet --help.

Quod Libet has excellent ID3 tag editing, with the ability to edit tags based on the filenames of songs, rename files based on their tags, and change many files at once. Quod Libet also supports album cover art.

In addition to its native features, Quod Libet also has an extensive collection of plugins that can greatly extend its functionality. One particularly interesting plugin, Animated On-Screen Display, can display information about the music Quod Libet is playing. See the full list of plugins on the Quod Libet site.

Transmission

While I normally use Azureus as my BitTorrent client, it's fairly resource-intensive, and that makes it less appealing for me. In situations where I need a simple and fast client, I use Transmission. It handles torrents using a fraction of the memory and CPU time that Azureus uses. Unlike Azureus, it has the ability to run all of the torrents on a single port, removing the need to allow entire port ranges through a firewall in order to use the program.

Transmission is perfect for users who occasionally need to download a torrent. While Azureus uses Java to draw its interface, Transmission uses GTK+, helping it fit in perfectly with a GNOME desktop. Transmission also sports a command-line interface that is especially useful when you must run it in a remote environment.

Network Monitoring with Zabbix

2006-03-26T14:03:00.000+08:00

The ZABBIX server requires the following system resources:

* 10 MB of disk space (100 MB recommended)
* 64 MB of RAM (128 MB recommended)
* MySQL or PostgreSQL as backend database

First we define 2 locations:

The Server, here comes all the info together and is processed in a database, note that the server can be monitored to so it runs an agent too.

The Agent, Information is gathered and polled by the server.

Setup of the Server:

http://prdownloads.sourceforge.net/zabbix/zabbix-1.1beta7.tar.gz?download

1 - Make the zabbix user and group

groupadd zabbix
useradd -c 'Zabbix' -d /home/zabbix -g zabbix -s /bin/bash zabbix

mkdir /home/zabbix
chown -R zabbix.zabbix /home/zabbix

2 - Untar the sources

cd /home/zabbix

tar zxvpf zabbix-1.1beta7.tar.gz

mv zabbix-1.1beta7 zabbix

cd zabbix

chown -R zabbix.zabbix .

su - zabbix

3 - Create a zabbix database and populate it

mysql -p -u root

create database zabbix;
quit;

cd create/mysql

mysql -u root -p zabbix < schema.sql

cd ../data

mysql -u root -p zabbix < data.sql
mysql -u root -p zabbix < images.sql

cd ../../

4 - Configure compile and install the server
We run an agent on the server to so we compile that too ;)

./configure --prefix=/usr --with-mysql --with-net-snmp --enable-server --enable-agent &&
make
su
make install
exit

5 - Prepare the rest of the system

As root edit

/etc/services

Add:

zabbix_agent 10050/tcp # Zabbix ports
zabbix_trap 10051/tcp

mkdir /etc/zabbix

cp misc/conf/zabbix_agentd.conf /etc/zabbix/
cp misc/conf/zabbix_server.conf /etc/zabbix/

Edit /etc/zabbix/zabbix_agentd.conf

make sure that the Server parameter points to the server addres,
for the agent that runs on the server its like this:

Server=127.0.0.1

Edit /etc/zabbix/zabbix_server.conf

For small sites this default file will do, however if you are into
tweaking your config for your 10+ hosts site, this is the place

Start the server

su - zabbix
zabbix_server
exit

Start the client:

su - zabbix
zabbix_agentd
exit

6 - Configure web interface

Edit frontends/php/include/db.inc.php:

$DB_TYPE ="MYSQL";
$DB_SERVER ="localhost";
$DB_DATABASE ="zabbix";
$DB_USER ="root";
$DB_PWD ="secret";

mkdir /home/zabbix/public_html
cp -R frontends/php/* /home/zabbix/html/

Edit /etc/apache/httpd.conf

Make this work:

AllowOverride FileInfo AuthConfig Limit Indexes
Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec

Order allow,deny
Allow from all

Order deny,allow
Deny from all

/etc/init.d/apache restart

chown -R zabbix.zabbix public_html

Setup of an Agent

http://prdownloads.sourceforge.net/zabbix/zabbix-1.1beta7.tar.gz?download

1 - Make the zabbix user and group

groupadd zabbix
useradd -c 'Zabbix' -d /home/zabbix -g zabbix -s /bin/bash zabbix

mkdir /home/zabbix
chown -R zabbix.zabbix /home/zabbix

2 - Untar the sources

cd /home/zabbix

tar zxvpf zabbix-1.1beta7.tar.gz

mv zabbix-1.1beta7 zabbix

cd zabbix

chown -R zabbix.zabbix .

su - zabbix

3 - Configure compile and install the agent

./configure --prefix=/usr --with-mysql --with-net-snmp --enable-agent
make
su
make install
exit

mkdir /etc/zabbix
cp misc/conf/zabbix_agentd.conf /etc/zabbix/

Edit /etc/zabbix/zabbix_agentd.conf

make sure that the Server parameter points to the server addres

Server=xxx.xxx.xxx.xxx

4 - Prepare the rest of the system

Edit /etc/services

Add:

zabbix_agent 10050/tcp # Zabbix ports
zabbix_trap 10051/tcp

5 - Start the agent

su - zabbix
zabbix_agentd
exit

What Next ?

Now point your browser to:

http:www.example.com/~zabbix

Login with username: Admin
No Password

Here you can really knock your self out.
This howto intended to show you how to install this mother.
Configuring the monitoring functions is and whole other ballgame.

For now i leave you here with some pointers to documentation

http://www.zabbix.com/documentation.php
http://sourceforge.net/projects/zabbix
http://www.google.com/search?q=zabbix

Preventing SSH Dictionary Attacks With DenyHosts

2006-03-25T18:48:00.000+08:00

In this HowTo I will show how to install and configure DenyHosts. DenyHosts is a tool that observes login attempts to SSH, and if it finds failed login attempts again and again from the same IP address, DenyHosts blocks further login attempts from that IP address by putting it into /etc/hosts.deny. DenyHosts can be run by cron or as a daemon. In this tutorial I will run DenyHosts as a daemon.

From the DenyHosts web site:

"DenyHosts is a script intended to be run by Linux system administrators to help thwart ssh server attacks.

If you've ever looked at your ssh log (/var/log/secure on Redhat, /var/log/auth.log on Mandrake, etc...) you may be alarmed to see how many hackers attempted to gain access to your server. Hopefully, none of them were successful (but then again, how would you know?). Wouldn't it be better to automatically prevent that attacker from continuing to gain entry into your system?

DenyHosts attempts to address the above... "

This tutorial is based on a Debian Sarge system, however, it should apply to other distributions with almost no modifications.

1 Installation

DenyHosts is written in Python, therefore we must install Python and also the Python development files first:

apt-get install python2.3-dev python2.3

Then we download and install DenyHosts like this:

cd /tmp
wget http://mesh.dl.sourceforge.net/sourceforge/denyhosts/DenyHosts-2.0.tar.gz
tar xvfz DenyHosts-2.0.tar.gz
cd DenyHosts-2.0
python setup.py install

This installs DenyHosts to /usr/share/denyhosts.

2 Configuration

Now we have to create the DenyHosts configuration file /usr/share/denyhosts/denyhosts.cfg. We can use the sample configuration file /usr/share/denyhosts/denyhosts.cfg-dist for this:

cd /usr/share/denyhosts
cp denyhosts.cfg-dist denyhosts.cfg

Then we must edit denyhosts.cfg with our favourite editor such as vi, for example. Mine looks like this:

 ############ THESE SETTINGS ARE REQUIRED ############

########################################################################
#
# SECURE_LOG: the log file that contains sshd logging info
# if you are not sure, grep "sshd:" /var/log/*
#
# The file to process can be overridden with the --file command line
# argument
#
# Redhat or Fedora Core:
#SECURE_LOG = /var/log/secure
#
# Mandrake, FreeBSD or OpenBSD:
SECURE_LOG = /var/log/auth.log
#
# SuSE:
#SECURE_LOG = /var/log/messages
#
########################################################################

########################################################################
# HOSTS_DENY: the file which contains restricted host access information
#
# Most operating systems:
HOSTS_DENY = /etc/hosts.deny
#
# Some BSD (FreeBSD) Unixes:
#HOSTS_DENY = /etc/hosts.allow
#
# Another possibility (also see the next option):
#HOSTS_DENY = /etc/hosts.evil
#######################################################################


########################################################################
# PURGE_DENY: removed HOSTS_DENY entries that are older than this time
#             when DenyHosts is invoked with the --purge flag
#
#      format is: i[dhwmy]
#      Where 'i' is an integer (eg. 7)
#            'm' = minutes
#            'h' = hours
#            'd' = days
#            'w' = weeks
#            'y' = years
#
# never purge:
PURGE_DENY =
#
# purge entries older than 1 week
#PURGE_DENY = 1w
#
# purge entries older than 5 days
#PURGE_DENY = 5d
#######################################################################


#######################################################################
# BLOCK_SERVICE: the service name that should be blocked in HOSTS_DENY
#
# man 5 hosts_access for details
#
# eg.   sshd: 127.0.0.1  # will block sshd logins from 127.0.0.1
#
# To block all services for the offending host:
#BLOCK_SERVICE = ALL
# To block only sshd:
BLOCK_SERVICE  = sshd
# To only record the offending host and nothing else (if using
# an auxilary file to list the hosts).  Refer to:
# http://denyhosts.sourceforge.net/faq.html#aux
#BLOCK_SERVICE =
#
#######################################################################


#######################################################################
#
# DENY_THRESHOLD_INVALID: block each host after the number of failed login
# attempts has exceeded this value.  This value applies to invalid
# user login attempts (eg. non-existent user accounts)
#
DENY_THRESHOLD_INVALID = 5
#
#######################################################################

#######################################################################
#
# DENY_THRESHOLD_VALID: block each host after the number of failed
# login attempts has exceeded this value.  This value applies to valid
# user login attempts (eg. user accounts that exist in /etc/passwd) except
# for the "root" user
#
DENY_THRESHOLD_VALID = 10
#
#######################################################################

#######################################################################
#
# DENY_THRESHOLD_ROOT: block each host after the number of failed
# login attempts has exceeded this value.  This value applies to
# "root" user login attempts only.
#
DENY_THRESHOLD_ROOT = 5
#
#######################################################################


#######################################################################
#
# WORK_DIR: the path that DenyHosts will use for writing data to
# (it will be created if it does not already exist).
#
# Note: it is recommended that you use an absolute pathname
# for this value (eg. /home/foo/denyhosts/data)
#
WORK_DIR = /usr/share/denyhosts/data
#
#######################################################################

#######################################################################
#
# SUSPICIOUS_LOGIN_REPORT_ALLOWED_HOSTS
#
# SUSPICIOUS_LOGIN_REPORT_ALLOWED_HOSTS=YES|NO
# If set to YES, if a suspicious login attempt results from an allowed-host
# then it is considered suspicious.  If this is NO, then suspicious logins
# from allowed-hosts will not be reported.  All suspicious logins from
# ip addresses that are not in allowed-hosts will always be reported.
#
SUSPICIOUS_LOGIN_REPORT_ALLOWED_HOSTS=YES
######################################################################

######################################################################
#
# HOSTNAME_LOOKUP
#
# HOSTNAME_LOOKUP=YES|NO
# If set to YES, for each IP address that is reported by Denyhosts,
# the corresponding hostname will be looked up and reported as well
# (if available).
#
HOSTNAME_LOOKUP=YES
#
######################################################################


######################################################################
#
# LOCK_FILE
#
# LOCK_FILE=/path/denyhosts
# If this file exists when DenyHosts is run, then DenyHosts will exit
# immediately.  Otherwise, this file will be created upon invocation
# and deleted upon exit.  This ensures that only one instance is
# running at a time.
#
# Redhat/Fedora:
#LOCK_FILE = /var/lock/subsys/denyhosts
#
# Debian
LOCK_FILE = /var/run/denyhosts.pid
#
# Misc
#LOCK_FILE = /tmp/denyhosts.lock
#
######################################################################


      ############ THESE SETTINGS ARE OPTIONAL ############


#######################################################################
#
# ADMIN_EMAIL: if you would like to receive emails regarding newly
# restricted hosts and suspicious logins, set this address to
# match your email address.  If you do not want to receive these reports
# leave this field blank (or run with the --noemail option)
#
ADMIN_EMAIL =
#
#######################################################################

#######################################################################
#
SMTP_HOST = localhost
SMTP_PORT = 25
SMTP_FROM = DenyHosts 
SMTP_SUBJECT = DenyHosts Report
#SMTP_USERNAME=foo
#SMTP_PASSWORD=bar
#
#######################################################################

######################################################################
#
# ALLOWED_HOSTS_HOSTNAME_LOOKUP
#
# ALLOWED_HOSTS_HOSTNAME_LOOKUP=YES|NO
# If set to YES, for each entry in the WORK_DIR/allowed-hosts file,
# the hostname will be looked up.  If your versions of tcp_wrappers
# and sshd sometimes log hostnames in addition to ip addresses
# then you may wish to specify this option.
#
#ALLOWED_HOSTS_HOSTNAME_LOOKUP=NO
#
######################################################################

######################################################################
#
# AGE_RESET_VALID: Specifies the period of time between failed login
# attempts that, when exceeded will result in the failed count for
# this host to be reset to 0.  This value applies to login attempts
# to all valid users (those within /etc/passwd) with the
# exception of root.  If not defined, this count will never
# be reset.
#
# See the comments in the PURGE_DENY section (above)
# for details on specifying this value or for complete details
# refer to:  http://denyhosts.sourceforge.net/faq.html#timespec
#
AGE_RESET_VALID=5d
#
######################################################################

######################################################################
#
# AGE_RESET_ROOT: Specifies the period of time between failed login
# attempts that, when exceeded will result in the failed count for
# this host to be reset to 0.  This value applies to all login
# attempts to the "root" user account.  If not defined,
# this count will never be reset.
#
# See the comments in the PURGE_DENY section (above)
# for details on specifying this value or for complete details
# refer to:  http://denyhosts.sourceforge.net/faq.html#timespec
#
AGE_RESET_ROOT=25d
#
######################################################################

######################################################################
#
# AGE_RESET_INVALID: Specifies the period of time between failed login
# attempts that, when exceeded will result in the failed count for
# this host to be reset to 0.  This value applies to login attempts
# made to any invalid username (those that do not appear
# in /etc/passwd).  If not defined, count will never be reset.
#
# See the comments in the PURGE_DENY section (above)
# for details on specifying this value or for complete details
# refer to:  http://denyhosts.sourceforge.net/faq.html#timespec
#
AGE_RESET_INVALID=10d
#
######################################################################

######################################################################
#
# PLUGIN_DENY: If set, this value should point to an executable
# program that will be invoked when a host is added to the
# HOSTS_DENY file.  This executable will be passed the host
# that will be added as it's only argument.
#
#PLUGIN_DENY=/usr/bin/true
#
######################################################################


######################################################################
#
# PLUGIN_PURGE: If set, this value should point to an executable
# program that will be invoked when a host is removed from the
# HOSTS_DENY file.  This executable will be passed the host
# that is to be purged as it's only argument.
#
#PLUGIN_PURGE=/usr/bin/true
#
######################################################################

######################################################################
#
# USERDEF_FAILED_ENTRY_REGEX: if set, this value should contain
# a regular expression that can be used to identify additional
# hackers for your particular ssh configuration.  This functionality
# extends the built-in regular expressions that DenyHosts uses.
# This parameter can be specified multiple times.
# See this faq entry for more details:
#    http://denyhosts.sf.net/faq.html#userdef_regex
#
#USERDEF_FAILED_ENTRY_REGEX=
#
#
######################################################################




  ######### THESE SETTINGS ARE SPECIFIC TO DAEMON MODE  ##########



#######################################################################
#
# DAEMON_LOG: when DenyHosts is run in daemon mode (--daemon flag)
# this is the logfile that DenyHosts uses to report it's status.
# To disable logging, leave blank.  (default is: /var/log/denyhosts)
#
DAEMON_LOG = /var/log/denyhosts
#
# disable logging:
#DAEMON_LOG =
#
######################################################################

#######################################################################
#
# DAEMON_LOG_TIME_FORMAT: when DenyHosts is run in daemon mode
# (--daemon flag) this specifies the timestamp format of
# the DAEMON_LOG messages (default is the ISO8061 format:
# ie. 2005-07-22 10:38:01,745)
#
# for possible values for this parameter refer to: man strftime
#
# Jan 1 13:05:59
#DAEMON_LOG_TIME_FORMAT = %b %d %H:%M:%S
#
# Jan 1 01:05:59
#DAEMON_LOG_TIME_FORMAT = %b %d %I:%M:%S
#
######################################################################

#######################################################################
#
# DAEMON_LOG_MESSAGE_FORMAT: when DenyHosts is run in daemon mode
# (--daemon flag) this specifies the message format of each logged
# entry.  By default the following format is used:
#
# %(asctime)s - %(name)-12s: %(levelname)-8s %(message)s
#
# Where the "%(asctime)s" portion is expanded to the format
# defined by DAEMON_LOG_TIME_FORMAT
#
# This string is passed to python's logging.Formatter contstuctor.
# For details on the possible format types please refer to:
# http://docs.python.org/lib/node357.html
#
# This is the default:
#DAEMON_LOG_MESSAGE_FORMAT = %(asctime)s - %(name)-12s: %(levelname)-8s %(message)s
#
#
######################################################################


#######################################################################
#
# DAEMON_SLEEP: when DenyHosts is run in daemon mode (--daemon flag)
# this is the amount of time DenyHosts will sleep between polling
# the SECURE_LOG.  See the comments in the PURGE_DENY section (above)
# for details on specifying this value or for complete details
# refer to:    http://denyhosts.sourceforge.net/faq.html#timespec
#
#
DAEMON_SLEEP = 30s
#
#######################################################################

#######################################################################
#
# DAEMON_PURGE: How often should DenyHosts, when run in daemon mode,
# run the purge mechanism to expire old entries in HOSTS_DENY
# This has no effect if PURGE_DENY is blank.
#
DAEMON_PURGE = 1h
#
#######################################################################


  #########   THESE SETTINGS ARE SPECIFIC TO     ##########
  #########       DAEMON SYNCHRONIZATION         ##########


#######################################################################
#
# Synchronization mode allows the DenyHosts daemon the ability
# to periodically send and receive denied host data such that
# DenyHosts daemons worldwide can automatically inform one
# another regarding banned hosts.   This mode is disabled by
# default, you must uncomment SYNC_SERVER to enable this mode.
#
# for more information, please refer to:
#        http:/denyhosts.sourceforge.net/faq.html#sync
#
#######################################################################


#######################################################################
#
# SYNC_SERVER: The central server that communicates with DenyHost
# daemons.  Currently, denyhosts.net is the only available server
# however, in the future, it may be possible for organizations to
# install their own server for internal network synchronization
#
# To disable synchronization (the default), do nothing.
#
# To enable synchronization, you must uncomment the following line:
#SYNC_SERVER = http://xmlrpc.denyhosts.net:9911
#
#######################################################################

#######################################################################
#
# SYNC_INTERVAL: the interval of time to perform synchronizations if
# SYNC_SERVER has been uncommented.  The default is 1 hour.
#
#SYNC_INTERVAL = 1h
#
#######################################################################


#######################################################################
#
# SYNC_UPLOAD: allow your DenyHosts daemon to transmit hosts that have
# been denied?  This option only applies if SYNC_SERVER has
# been uncommented.
#
#SYNC_UPLOAD = no
#
# the default:
#SYNC_UPLOAD = yes
#
#######################################################################


#######################################################################
#
# SYNC_DOWNLOAD: allow your DenyHosts daemon to receive hosts that have
# been denied by others?  This option only applies if SYNC_SERVER has
# been uncommented.
#
#SYNC_DOWNLOAD = no
#
# the default:
#SYNC_DOWNLOAD = yes
#
#######################################################################

#######################################################################
#
# SYNC_DOWNLOAD_THRESHOLD: If SYNC_DOWNLOAD is enabled this paramter
# filters the returned hosts to those that have been blocked this many
# times by others.  That is, if set to 1, then if a single DenyHosts
# server has denied an ip address then you will receive the denied host.
#
#SYNC_DOWNLOAD_THRESHOLD = 10
#
# the default:
#SYNC_DOWNLOAD_THRESHOLD = 3
#
#######################################################################

Make sure you set SECURE_LOG and LOCK_FILE to the correct values for your distribution! For Debian, these are:

SECURE_LOG = /var/log/auth.log
LOCK_FILE = /var/run/denyhosts.pid

As we want to run DenyHosts as a daemon, we need the daemon control script /usr/share/denyhosts/daemon-control. Again, we can use the sample script /usr/share/denyhosts/daemon-control-dist to create the needed file:

cp daemon-control-dist daemon-control

Edit /usr/share/denyhosts/daemon-control and make sure you set the correct values for DENYHOSTS_BIN, DENYHOSTS_LOCK, and DENYHOSTS_CFG. For Debian, these are:

DENYHOSTS_BIN = "/usr/bin/denyhosts.py"
DENYHOSTS_LOCK = "/var/run/denyhosts.pid"
DENYHOSTS_CFG = "/usr/share/denyhosts/denyhosts.cfg"

So my /usr/share/denyhosts/daemon-control file looks like this:

#!/usr/bin/env python
# denyhosts     Bring up/down the DenyHosts daemon
#
# chkconfig: 2345 98 02
# description: Activates/Deactivates the
#    DenyHosts daemon to block ssh attempts
#
###############################################

###############################################
#### Edit these to suit your configuration ####
###############################################

DENYHOSTS_BIN   = "/usr/bin/denyhosts.py"
DENYHOSTS_LOCK  = "/var/run/denyhosts.pid"
DENYHOSTS_CFG   = "/usr/share/denyhosts/denyhosts.cfg"


###############################################
####         Do not edit below             ####
###############################################

import os, sys, signal, time

STATE_NOT_RUNNING = -1
STATE_LOCK_EXISTS = -2

def usage():
   print "Usage: %s {start [args...] | stop | restart [args...] | status | debug | condrestart [args...] }" % sys.argv[0]
   print
   print "For a list of valid 'args' refer to:"
   print "$ denyhosts.py --help"
   print
   sys.exit(0)


def getpid():
   try:
       fp = open(DENYHOSTS_LOCK, "r")
       pid = int(fp.readline().rstrip())
       fp.close()
   except Exception, e:
       return STATE_NOT_RUNNING

   if os.access(os.path.join("/proc", str(pid)), os.F_OK):
       return pid
   else:
       return STATE_LOCK_EXISTS


def start(*args):
   cmd = "%s --daemon " % DENYHOSTS_BIN
   if args: cmd += ' '.join(args)

   print "starting DenyHosts:   ", cmd

   os.system(cmd)


def stop():
   pid = getpid()
   if pid >= 0:
       os.kill(pid, signal.SIGTERM)
       print "sent DenyHosts SIGTERM"
   else:
       print "DenyHosts is not running"

def debug():
   pid = getpid()
   if pid >= 0:
       os.kill(pid, signal.SIGUSR1)
       print "sent DenyHosts SIGUSR1"
   else:
       print "DenyHosts is not running"

def status():
   pid = getpid()
   if pid == STATE_LOCK_EXISTS:
       print "%s exists but DenyHosts is not running" % DENYHOSTS_LOCK
   elif pid == STATE_NOT_RUNNING:
       print "Denyhosts is not running"
   else:
       print "DenyHosts is running with pid = %d" % pid


def condrestart(*args):
   pid = getpid()
   if pid >= 0:
       restart(*args)


def restart(*args):
   stop()
   time.sleep(1)
   start(*args)


if __name__ == '__main__':
   cases = {'start':       start,
            'stop':        stop,
            'debug':       debug,
            'status':      status,
            'condrestart': condrestart,
            'restart':     restart}

   try:
       args = sys.argv[2:]
   except:
       args = []

   try:
       option = sys.argv[1]

       if option in ('start', 'restart', 'condrestart'):
           if '--config' not in args and '-c' not in args:
               args.append("--config=%s" % DENYHOSTS_CFG)

       cmd = cases[option]
       apply(cmd, args)
   except:
       usage()

Next we have to make that file executable:

chown root daemon-control
chmod 700 daemon-control

Afterwards, we create the system bootup links for DenyHosts do that it is started automatically when the system is booted:

cd /etc/init.d
ln -s /usr/share/denyhosts/daemon-control denyhosts
update-rc.d denyhosts defaults

Finally, we start DenyHosts:

/etc/init.d/denyhosts start

DenyHosts logs to /var/log/denyhosts, if you are interested in the logs. The SSH daemon logs to /var/log/auth.log on Debian. You can watch both logs and try to log in with an invalid user or with a valid user and incorrect password, etc. via SSH and see what happens. After you have crossed the threshold of incorrect login attempts, the IP address from which you tried to connect should get listed in /etc/hosts.deny, like this:

# /etc/hosts.deny: list of hosts that are _not_ allowed to access the system.
#                  See the manual pages hosts_access(5), hosts_options(5)
#                  and /usr/doc/netbase/portmapper.txt.gz
#
# Example:    ALL: some.host.name, .some.domain
#             ALL EXCEPT in.fingerd: other.host.name, .other.domain
#
# If you're going to protect the portmapper use the name "portmap" for the
# daemon name. Remember that you can only use the keyword "ALL" and IP
# addresses (NOT host or domain names) for the portmapper. See portmap(8)
# and /usr/doc/portmap/portmapper.txt.gz for further information.
#
# The PARANOID wildcard matches any host whose name does not match its
# address.

# You may wish to enable this to ensure any programs that don't
# validate looked up hostnames still leave understandable logs. In past
# versions of Debian this has been the default.
# ALL: PARANOID
sshd: 192.168.0.203

This means that the system with the IP address 192.168.0.203 cannot connect anymore using SSH.

You can specify if/when IP addresses are removed again from /etc/hosts.deny - have a look at the PURGE_DENY variable in /usr/share/denyhosts/denyhosts.cfg. You must start DenyHosts with the --purge option to make the PURGE_DENY variable effective, like this:

/etc/init.d/denyhosts start --purge

However, you can also remove IP addresses manually from there, and as soon as they have got removed, these IP addresses can try to log in again via SSH.

How do I scan my Linux system for rootkits, worms, trojans, etc.?

2006-03-24T20:37:00.000+08:00

Either with ckrootkit or with rkhunter.

chkrootkit:

Either install the package that comes with your distribution (on Debian you would run

apt-get install chkrootkit

), or download the sources from www.chkrootkit.org and install manually:

wget --passive-ftp ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz

tar xvfz chkrootkit.tar.gz

cd chkrootkit-/

make sense

Afterwards, you can move the chkrootkit directory somewhere else, e.g. /usr/local/chkrootkit:

cd ..

mv chkrootkit-/ /usr/local/chkrootkit

Now you can run chkrootkit manually:

cd /usr/local/chkrootkit

./chkrootkit

(if you installed a chkrootkit package coming with your distribution, your chkrootkit might be somewhere else).

You can even run chkrootkit by a cron job and get the results emailed to you:

Run

crontab -e

to create a cron job like this:

0 3 * * * (cd /usr/local/chkrootkit-; ./chkrootkit 2>&1 | mail -s "chkrootkit output my server" you@yourdomain.com)

That would run chkrootkit every night a 3.00h.

rkhunter:

Download the latest rkhunter sources from www.rootkit.nl:

wget http://downloads.rootkit.nl/rkhunter-1.2.7.tar.gz

tar xvfz rkhunter-1.2.7.tar.gz

cd rkhunter/

./installer.sh

This will install rkhunter to the directory /usr/local/rkhunter. Now run

rkhunter --update

to download the latest chkrootkit/trojan/worm signatures (you should do this regularly).

Now you can scan your system for malware by running

rkhunter -c

Convert man pages to text

2006-03-24T20:35:00.000+08:00

There can often be times when you want man pages in print, or to simply save them to text for
something like HTML use. The common thing that people do is:

man lp > lp.txt

This will take the output of lp and put it to a file named lp.txt, but it will keep all the formatting
in the document. In order to get rid of those characters, do the following:

man lp | col -b > lp.txt

Create the text file both ways naming one of them "lp2.txt" and do a "diff" on them. You'll see the
difference then.

Poor Mans Raid & Clone Script Tutorial

2006-03-23T01:00:00.000+08:00

POORMANS RAID & CLONE SCRIPT

1. Get yourself two hardives setup your first drive (HDA) with the your perfek system.
2. Power up your second drive (HDC)
3. Slice HDC up by using cfdisk (command is cfdisk /dev/hdc)

Hda1 - primary boot begining Label [ / ]
Hda5 - logical beginning [ /usr ]
Hda6 - logical beginning [ /var ]
Hda7 - logical beginning [ swap ]
Hda8 - logical beginning [ /tmp ]
Hda9 - logical beginning [ /home ]

To Setup your filesystem and your labels

Mkfs -t ext3 -L / dev/hdc1
Mkfs -t ext3 -L /usr dev/hdc5
Mkfs -t ext3 -L /var dev/hdc6
Mkfs -t ext3 -L /tmp dev/hdc8
Mkfs -t ext3 -L /home dev/hdc9

okay so you have sliced HDC and setup your filesystems with Labels.

Vi and copy the below script save a file called clone, make it chmod 777 and run it.

once copied run Grub
#grub --no-floppy
#grub>
# device (hd0) /dev/hdc
# root (hd0,0)
# setup (hd0)
# quit

Clone Script"make sure nothing is mounted in /mnt!"
echo "going to unmount /mnt"
/bin/umount /mnt

mount /dev/hdc1 /mnt
cd /
cp -ax / /mnt

mount /dev/hdc5 /mnt/usr
cd /usr
tar lcpBf - /usr | (cd /mnt; tar xBf -)

mount /dev/hdc9 /mnt/home
cd /home
tar lcpBf - /home | (cd /mnt; tar xBf -)

mount /dev/hdc8 /mnt/tmp
cd /tmp
tar lcpBf - /tmp | (cd /mnt; tar xBf -)

#Stop crtical services
/etc/init.d/mysql stop

mount /dev/hdc6 /mnt/var
cd /var
tar lcpBf - /var | (cd /mnt; tar xBf -)

#start all the services again
/etc/init.d/mysql start

echo "all copied"
exit 0

Out-going admin cron jobs

2006-03-20T15:02:00.000+08:00

As the In-Coming Sys Admin, one the checks you should never miss is to check all the Cron and at jobs during the handover period from the out-going Admin!

Try:
# crontab -u [user] -l

Manually change the user for of the users in /etc/passwd or script this substitution if the number of users on your system is worth the effort.

Who hits my website

2006-03-20T14:59:00.000+08:00

This will generate a listing of all IPs that have accessed a specific directory of my website
(the /foo/ directory), and print a count (from greatest to least) next to the IP.

if you want to get a total listing for all directories of a domain, just omit the "grep -e /foo/" statement.

cat /home/server.com/logs/access_log |
grep -e /flesh/ |
sort |
uniq -w15 -c |
cut -f 1 -d- |
sort -r -g

Using this format for the apache log files:
63.26.57.218 - - [07/Mar/2001:21:12:15 -0500] "GET
/flesh/fleshpix/thumbs/facial2/poofylips.jpg_t.jpg HTTP/1.1" 200 4353

Sample output is:
123 62.155.255.18
123 216.17.9.140
121 62.98.129.125
120 66.32.16.55
120 216.196.144.202
119 194.100.2.65
116 194.170.1.68
114 62.155.255.21

Scripts for all occasions

2006-03-20T14:55:00.000+08:00

Sun Microsystems has a great archive of useful scripts at:

http://www.sun.com/bigadmin/scripts/index.html

Check it out. You'll be glad you did!

Bash shell option

2006-03-20T14:53:00.000+08:00

If you are using bash shell. There is a way to cd a particular directory even if you spelled incorrectly on the command line. Set the shell option to:

shopt -s cdspell

eg:-
Suppose you want to cd to "cd /tmp" and you have miss typed to "cd /pmp" still it will cd to "cd /tmp".

This setting will be very usefull if you have a long named directory.

Get the hidden files

2006-03-20T14:51:00.000+08:00

Chrooted SSH HowTo

2006-03-20T01:58:00.000+08:00

This tutorial describes how to install and configure OpenSSH so that it will allow chrooted sessions for users. With this setup, you can give your users shell access without having to fear that they can see your whole system. Your users will be jailed in a specific directory which they will not be able to break out of.

This setup is based on a Debian Sarge (Debian 3.1) system, and the chrooted SSH will be installed in such a way that it will still use the configuration files of the standard OpenSSH Debian package which are in /etc/ssh/, and you will be able to use the standard OpenSSH Debian init script /etc/init.d/ssh. Therefore you do not have to create your own init script and configuration file.

1 Install The Newest Zlib Version

Because there was a security hole in zlib-1.2.2 about which the chrooted SSH will complain when we try to compile it, we install the newest zlib version right now:

cd /tmp
wget http://www.zlib.net/zlib-1.2.3.tar.gz
tar xvfz zlib-1.2.3.tar.gz
cd zlib-1.2.3
make clean
./configure -s
make
make install

2 Install The Chrooted SSH

This is quite easy. We download the patched OpenSSH sources, and we configure them with /usr as directory for the SSH executable files, with /etc/ssh as directory where the chrooted SSH will look for configuration files, and we also allow PAM authentication:

cd /tmp
apt-get install libpam0g-dev openssl libcrypto++-dev libssl0.9.7 libssl-dev ssh
wget http://chrootssh.sourceforge.net/download/openssh-4.2p1-chroot.tar.gz
tar xvfz openssh-4.2p1-chroot.tar.gz
cd openssh-4.2p1-chroot
./configure --exec-prefix=/usr --sysconfdir=/etc/ssh --with-pam
make
make install

3 Create The Chroot Environment

Next I create a chroot environment under /home/chroot. This is the directory that all chrooted SSH users will get jailed in, i.e. they will not be able to see any files/directories outside /home/chroot.

I have to create some directories in /home/chroot, and I have to copy a few binaries like /bin/bash, /bin/ls, etc. as well as the libraries on which these binaries depend into the chroot environment so that they are available to any chrooted user.

mkdir /home/chroot/
mkdir /home/chroot/home/
cd /home/chroot
mkdir etc
mkdir bin
mkdir lib
mkdir usr
mkdir usr/bin
mkdir dev
mknod dev/null c 1 3
mknod dev/zero c 1 5

Now that we have created the necessary directories, we are goning to copy some binaries and all the libraries on which they depend into the chroot environment. This is an excerpt of a script that I found on http://mail.incredimail.com/howto/openssh/create_chroot_env that does this. Just copy and paste the following lines into your shell, and hit Return. If you want to make more programs available to your chrooted users, just add these programs to the APPS line:

APPS="/bin/bash /bin/ls /bin/mkdir /bin/mv /bin/pwd /bin/rm /usr/bin/id /usr/bin/ssh /bin/ping /usr/bin/dircolors"
for prog in $APPS;  do
       cp $prog ./$prog

       # obtain a list of related libraries
       ldd $prog > /dev/null
       if [ "$?" = 0 ] ; then
               LIBS=`ldd $prog | awk '{ print $3 }'`
               for l in $LIBS; do
                       mkdir ./`dirname $l` > /dev/null 2>&1
                       cp $l ./$l
               done
       fi
done

Then we do this:

cp /lib/libnss_compat.so.2 /lib/libnsl.so.1 /lib/libnss_files.so.2 ./lib/
echo '#!/bin/bash' > usr/bin/groups
echo "id -Gn" >> usr/bin/groups
touch etc/passwd
grep /etc/passwd -e "^root" > etc/passwd

You should also copy the line of the group in which you will create new users from /etc/group to /home/chroot/etc/group. In this tutorial we will create users in the group users, so we do this:

grep /etc/group -e "^root" -e "^users" > etc/group

and restart SSH:

/etc/init.d/ssh restart

4 Create A Chrooted User

Even with the chrooted SSH that we have just installed you can log in without being chrooted (which makes sense if you log in as root, for example). Now, how does the chrooted SSH decide whom to chroot and whom not? That's easy: the chrooted SSH looks up the user who is trying to log in in /etc/passwd. If the user's home directory in /etc/passwd has a . in it, then the user is going to be chrooted.

Example (from /etc/passwd):

user_a:x:2002:100:User A:/home/user_a:/bin/bash

This user will not be chrooted.

user_b:x:2003:100:User B:/home/chroot/./home/user_b:/bin/bash

This user will be chrooted.

Now we create the user testuser with the home directory /home/chroot/./home/testuser and the group users (which is the default group for users on Debian so you do not have to specify it explicitly):

useradd -s /bin/bash -m -d /home/chroot/./home/testuser -c "testuser" -g users testuser

Then we give testuser a password:

passwd testuser

Finally, we have to copy the line for testuser in /etc/passwd to /home/chroot/etc/passwd:

grep /etc/passwd -e "^testuser" >> /home/chroot/etc/passwd

We have already copied the users group line from /etc/group to /home/chroot/etc/group so we do not have to do this here again. If you create a chrooted user in another group than users, add this group to /home/chroot/etc/group:

grep /etc/group -e "^othergroup" >> /home/chroot/etc/group

Now try to log in to SSH as testuser. You should be chrooted and not be able to browse files/directories outside /home/chroot.

Have fun!

How to create IP-IP tunnel between FreeBSD and Linux

2006-03-17T22:25:00.000+08:00

Sometimes, I need to connect remote Unix servers with tunnels to provide some specific services or to get access to some internal networks. I was very surprised, when my friend, young system administrator, asked me about how to bring up IP-IP tunnel between different Unix operating systems (FreeBSD and Linux in his case) and said, that he can’t find information about this configuration. As the result of my discovering, this HOWTO has been created.

Lets see to what we have and what we need to do.

We have 2 servers:

Server1:
- OS: Linux
- Network Interface: eth0
- IP: 100.100.100.100
Server2:
- OS: FreeBSD
- Network Interface: fxp0
- IP: 200.200.200.200

We need to get IPv4 over IPv4 tunnel with the following parameters between described servers:

Server1: 10.0.0.1 / 255.255.255.252
Server2: 10.0.0.2 / 255.255.255.252

To setup described configuration on Linux server we need to do following steps:

Create ipip tunnel interface:

# ip tunnel add tun0 mode ipip > remote 200.200.200.200 local 100.100.100.100 dev eth0

Set interface IP addresses:

# ifconfig tun0 10.0.0.1 netmask 255.255.255.252 > pointopoint 10.0.0.1

Set interface MTU and bring interface up:
```
# ifconfig tun0 mtu 1500 up
```

Now we have following interface on the Linux server:

linux:~# ifconfig tun0
tun77     Link encap:IPIP Tunnel  HWaddr
         inet addr:10.0.0.1  P-t-P:10.0.0.2  Mask:255.255.255.252
         UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
         RX packets:6 errors:0 dropped:0 overruns:0 frame:0
         TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:504 (504.0 b)  TX bytes:624 (624.0 b)

Now, we need to create tunnel point at the FeeeBSD server:

Create gif tunnel interface:
```
# # ifconfig gif0 create
```

Set interface transport IP addresses:

# gifconfig gif0 inet 200.200.200.200 100.100.100.100

Set interface IP addresses:

# ifconfig gif0 10.0.0.2 netmask 255.255.255.252 10.0.0.1

Set interface MTU and bring interface up:
```
# ifconfig gif0 mtu 1500 up
```

The result at the FreeBSD side is following:

# ifconfig gif0
gif0: flags=8051 mtu 1500
       tunnel inet 200.200.200.200 –> 100.100.100.100
       inet 10.0.0.2 –> 10.0.0.1 netmask 0xfffffffc

To check the result we can use ping utility at linux side:

linux:~# ping -c 4 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.139 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.138 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.138 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.136 ms

— 172.17.0.1 ping statistics —
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.136/0.137/0.139/0.014 ms

That is all! Now we have “direct” connection between our two servers and we are able to do some routing via this link.

How can I have two default routes?

2006-03-17T22:22:00.000+08:00

This is where advanced routing comes into play (finally!). With advanced routing, you can have as many routing tables as you need; in this case you need to add just one for the new DSL line.

First, add a name for the new routing table to the file /etc/iproute2/rt_tables. You can append it to the file with command "echo 2 dsl2 >> /etc/iproute2/rt_tables".

# echo 2 dsl2 >> /etc/iproute2/rt_tables
# cat /etc/iproute2/rt_tables             list the file contents
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
2 dsl2                          the line you just added

Earlier I mentioned that typing "ip route" is a shortcut for the longer command "ip route show table main". Well, to list the new routing table you have to use the long form: "ip route show table dsl2" If you enter this command right now, you will see the new table is empty.

You really only need to add the new default route to the new table; the old "main" table will continue to handle everything else. You will see why in a minute. Once again, here is the existing "main" table.

# ip route show table main
63.63.63.0/29 dev eth0  proto kernel  scope link  src 63.63.63.1
30.31.32.0/29 dev eth1  proto kernel  scope link  src 30.31.32.1
default via 63.63.63.6 dev eth0

Add the new default route to table dsl2 and then look at the (short) table.

# ip route add default via 30.31.32.6 dev eth1 table dsl2
# ip route show table dsl2

default via 30.31.32.6 dev eth1    the whole table is just one line

But the new table is not used yet!

You need to learn one more command, "ip rule"

A routing table tells where packets should go (its destination). You need to be able to tell the kernel to use a different table, based on where a packet is from (its source address).

The existing ip ruleset is very simple, look at it now.

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

You need to add just one new rule:

# ip rule add from 30.31.32.1 lookup dsl2 prio 1000

This command says "add a rule" to handle the case when a packet has a "from" pattern of "30.31.32.1"; use the routing table called "dsl2", and assign the rule a priority level of "1000". Now relist the rules. In this example, the "pattern" only needs to match one address but if you build a Linux router, you could set patterns that would match different sets of addresses.

# ip rule
0:      from all lookup local
1000:   from 30.31.32.1 lookup dsl2
32766:  from all lookup main
32767:  from all lookup default

The kernel searches the ip rules in order, starting with the lowest priority and continuing through each rule and routing table until the packet has been routed successfully.

Your default ruleset will always have a 'local' table with 'all' as the match pattern. The local table handles traffic that is supposed to stay on the local machine, and broadcast traffic.

Our new rule comes next, with a priority of 1000. I picked this number to make adding other rules before and after ours easy later on.

After our rule comes the 'main' table, which is the one that is manipulated by the old 'route' command. Finally comes the 'default' table. I don't know the official purpose of default, it's empty on all the systems I have set up. There is a 'default' route in the table 'main', so no traffic ever gets to the table 'detault'.

Caveats

When you are playing with multiple routing tables, you have to remember to add the 'table' portion to the command. I have only forgotten about 1000 times now. It can be mystifying when rules change in the wrong table (main). And of course, you are sure to confuse things when learning and lock yourself out if you are remotely logged in. The changes happen FAST. Use a console connection.

Another tip. Routes are cached. This means that if you update a routing table and nothing seems to happen, don't get frustrated, just flush the cache. You can make several changes at once and then flush the cache at the end so that the changes effectively all happen simultaneously. This is handy when working on a running router.

The flush command I use is "ip route flush table cache". Be very careful with the flush command!! Enter it wrong and you will remove all routing rules, instantly cutting off your networks.

Revealing your routing tables

2006-03-17T22:18:00.000+08:00

From a command line on any Linux system, you can see the existing routing table by simply typing 'route' at the prompt. (Type '/sbin/route' if /sbin is not in your path.) Your routing table will be similar to this:

# route
destination     gateway  interface
63.63.63.0      *  eth0
default  63.63.63.6 eth

Advanced routing commands are issued as arguments to the 'ip' command. To see the routing table with iproute2, you can use the long version "ip route show table main" or the shortcut version "ip route" like this:

# ip route
63.63.63.0/29 dev eth0 proto kernel scope link src 63.63.63.1
default via 63.63.63.6 dev eth0

Recall that in our simple example, all ip addresses are considered to be either 'local' or 'not local'. For this network, a local address will be in the range from 63.63.63.1 to 63.63.63.6. A non-local address is just anything else.* It's traditional to assign the router the highest address in the range (ending in 6 for this network).

The first line in the routing table says that if the address is local, your computer puts the packet right out onto the ethernet wire using interface /dev/eth0. The only other device on your local network right now is the DSL router at 63.63.63.6. You can probably use a browser to bring up the router's built-in web-based configuration interface by connecting to http://63.63.63.6/

When you want to surf over to linuxforums.org, the packets need to go to the Internet. You computer will use DNS to translate the name linuxforums.org into its IP address (67.15.52.42). Your computer sees that address is not local, so it uses teh 'default' entry in the routing table and sends the packet to your DSL router.

In turn, the DSL router looks at the destination address, and sees that the data is destined for somewhere on the Internet. The DSL router then pushes the packet out its DSL connection to the next router upstream which is at your ISP. Just as with the postcard analogy, it's now out of your hands; more upstream routers send it on its way to the linuxforums.org server.

When the response comes back from linuxforums.org, there is a destination address of 63.63.63.1 in the packet, so when it hits your DSL router it will pass the packet onto your LAN. Your desktop computer sees the packet on the LAN and picks it up. The round trip connection is now completed. It takes many such exchanges of packets to pass all the data required for just a single page.

If you have a web server running on your local computer, everything works pretty much the same way, but the traffic flows the other direction. Requests for pages come in from the Internet and your web server responds by sending pages back out via the DSL router. Usually you arrange to get your ip address into a DNS server somewhere so that you can publish a URL for your server with a friendly name like "http://myveryownserver.org" but you could just as easily not bother with that and just tell all your friends that your URL is http://63.63.63.1/.

When a friend surfs to http://63.63.63.1/, your web server will receive the request with a source address pointing back to your friend's computer. Thus the reply will be routed back out via your one and only default router.

* Yes, I know there is another internal network on the address 127.0.0.1. It's called the loopback interface /dev/lo. But let's not worry about it here, okay?

Where do the IP addresses come from?

2006-03-17T22:15:00.000+08:00

You used to have to set up ip addresses, netmasks, gateways, and name resolvers all manually. Refer to any basic networking text to learn more about this aspect.

These days DHCP (Dynamic Host Control Protocol) is very popular. Let's assume that a DHCP server (typically inside your Cable/DSL modem) and issues all the network information required. The network engineer at your upstream provider has configured the modem for you so that it will work with their network.

With a typical home setup, you are assigned just one IP address and if you have more than one computer at home, you have to play games (NAT or MASQUERADE) to share the one IP address. A typical small business connection might have 5 IP addresses assigned statically so that you can publish them; a sixth address in the range lets you communicate with the router in your DSL modem.

For a 6 address block, you'd be assigned a netmask with 29 bits (255.255.255.248 in "dotted decimal" format). You don't need to know what a netmask is right now, but it divides the address into two parts, network and local. In the postcard analogy, a p.o. box number would be the local part of your address.

Let's say your first DSL provider gives you the address range from 63.63.63.1 to 63.63.63.6; your computer is at 63.63.63.1; your DSL router is at 63.63.63.6; and the remaining 4 addresses are currently unused.

Just what is routing?

2006-03-17T22:04:00.000+08:00

I like to compare network packet routing with the post office. Let's say that you and a friend play chess by sending postcards to each other. Each postcard has a source and a destination address (From: and To: addresses) and a small amount of information, the next move in your game. You send a postcard with your move on it, then you wait for a reply to come back with your friend's next move. The game progresses as a series of transmissions and replies; the whole thing routed through any number of post offices without your knowledge. Under ordinary circumstances, to tap into this vast network, all you have to need to know are the addresses and how to find your mailbox.

Each postcard is like a network packet. With the postal system, you could send a letter if your information doesn't fit on a card. With a network, you just send more packets instead because you can't (normally) change the packet size; your operating system handles the task of breaking down large messages into small packets for transmission; likewise it reassembles incoming packets for you. This is what happens when you request a web page; you enter one address (the URL) and the network coordinates the exchange of hundreds of packets unbenownst to you. To learn about routing though, you can ignore this fact and concentrate only on the routing of a single packet.

After you drop your postcard into a mailbox, a system for routing it to its destination takes over; the reply comes back the same way. You only care about this routing mechanism when something changes, for example, if you move to a new address and want things forwarded. Otherwise it all remains invisible to you.

With the Internet, you normally only deal with one connection to this invisible network; the router provided to you by your Internet Service Provider (ISP) is your connection.

If you have one computer and one network connection, this is just about all you need to know about routing. Any packet leaving your computer heads out to the Internet, and any reply coming back from the Internet is destined for your one computer.

Continuing with the postal analogy, if you work in the mailroom for a large company, you still generally only have one connection to the outside world, but there is an internal mailbox for each employee. As the clerk in the mailroom, you decide based on the To: address to either route a letter internally (delivering it to its local mailbox) or as before, you throw it into an outgoing pile for your connection to the world.

When you add a second computer in your office, each of your computers will have addresses indicating they are connected to the same LAN (local area network). The routing table in each of your computers has to know the correct LAN address range so that they can talk to each other.

With additional equipment, network complexity grows and things become progressively more complicated and much more interesting (to me anyway); there can be many routes in your computer's routing table. We are not getting into that here.

In the simple set up we are doing here, you only have to recognize two types of address, internal (LAN) or external (Internet).

Mysql Database Backup Script

2006-03-15T18:34:00.000+08:00

About AutoMySQLBackup

A script to take daily, weekly and monthly backups of your MySQL databases using mysqldump. Features - Backup mutiple databases - Single backup file or to a seperate file for each DB - Compress backup files - Backup remote servers - E-mail logs – More

AutoMySQLBackup Script Features

• Backup mutiple MySQL databases with one script. (Now able to backup ALL databases on a server easily. no longer need to specify each database seperately)
• Backup all databases to a single backup file or to a seperate directory and file for each database.
• Automatically compress the backup files to save disk space using either gzip or bzip2 compression.
• Can backup remote MySQL servers to a central server.
• Runs automatically using cron or can be run manually.
• Can e-mail the backup log to any specified e-mail address instead of "root". (Great for hosted websites and databases).
• Can email the compressed database backup files to the specified email address.
• Can specify maximun size backup to email.
• Can be set to run PRE and POST backup commands.
• Choose which day of the week to run weekly backups.

AutoMySQLBackup Download

The AutoMySQLBackup project has now been moved to Sourceforge.net for better maintainence.

http://sourceforge.net/projects/automysqlbackup/

AutoMySQLBackup Requirements

The AutoMySQLBackup script only requires mysqldump (A standard utility provided with the mysql client) and gzip or bzip2 for compression of the backup files.

If you would like to have the log emailed to you then you will need to have permission to execute the "mail" program. If you want the compressed backup files mailed to you then Mutt must be available on the server.
Finally you will need a bash shell and the standard system tools and utilities (all these requirements should be the default on most linux system.)

What AutoMySQLBackup does

Every day AutoMySQLBackup will run (if setup on /etc/cron.daily) and using mysqldump and gzip will dump your specified databases to the /backups/daily directory, it will rotate daily backups weekly so you should never have more than 7 backups in there..

Every Saturday AutoMySQLBackup will again backup the databases you have chosen but they will be placed into /backups/weekly, these will be rotated every 5 weeks so there should never be more than 5 backups in there..

Every 1st of the month AutoMySQLBackup will create a backup of all databases and place them into /backups/monthly. These will never be rotated so it will be up to you to do your own house keeping. I would suggest taking a copy of this offline every month or two so that if you have a hard drive failure you will be able to restore your database..

AutoMySQLBackup Installation

The install is as simple as editing a few variables in the AutoMySQLBackup file. The full setup is documented in the AutoMySQLBackup script file below the variables section..

Here is a quick minimum setup step by step..

1. Download automysqlbackup.sh and place it into your /etc/cron.daily directory or your home directory.

2. Edit (at least) the following lines :-
USERNAME=dbuser (The user must have at least select privileges to the databases) PASSWORD=password
DBNAMES="DB1 DB2 DB3" (make sure to keep the quotes " " otherwise it won't work)

3. Make the file executable :- chmod u+rwx

4. Create the following directory./backups

5. That's it.. Now you can run it using the command line "./automysqlbackup.sh" or if it is in /etc/cron.daily it will run each day when cron runs.

Listing files by size

2006-03-15T03:55:00.000+08:00

Where am i always

2006-03-14T01:59:00.000+08:00

In csh, if you always want to know where you are, put in your .cshrc-file the following 2 lines:

alias sp 'set prompt="`hostname`:`pwd`> "' alias cd 'cd \!*;sp'

Possible to get the administrator password ?

2006-03-13T19:54:00.000+08:00

something "funny" :

There is a file that contains all the installation logs :
/var/log/installer/cdebconf/questions.dat
In this file, there is all the questions asked to the user abd all the user's answers.

So, near the end of the file, we can find the user created during the installation... and its password (not hidden).

Then, tell me if I'm wrong :
_ in the normal installation mode, the user created can get the root privileges with sudo
_ in the expert mode, there is a root account created

In both case, it's possible to get an administrator username/password.

Moreover, this file can be read by all users (contrary to the syslog).

Personally, the user I have created during the installation is the computer administrator and I had no reason (until now) to change its password after the installation. I've just created a non-administrator user after the installation.

I think it's risky to store an user's password in a file readable by everybody. (for example if we can login via ssh on an Ubuntu server)

I don't know what you think of this...

11 Unix Tricks

2006-03-13T02:18:00.000+08:00

Whether you’re a newbie to Linux or you are a seasoned guru, you’re bound to find some scripts or programs which just make your life on the computer exponentially easier. The following is a list of them and explains what they do.

1. tac

tac is a command similar to cat, except that it reverses the output of the specified file.

tac

2. Multiple X-Sessions

Using multiple X-Sessions allows a user to have open more than just one instance of XFree, so multiple desktop environments can be used at a time.

startx — :

terminal name is simply which terminal X should start on. By default it starts on

0:0. If you want two open, you may want it open on 0:1, which can be run by:

startx — :1

The resulting desktops will exist on F7 - F12.

3. SSH

SSH is a telnet replacement which includes encryption for higher security.

ssh -l

is the name, specified in /etc/passwd, that you want to login as. Remove the -l for anonymous.

is simply the hostname or IP address of the computer to connect to.

4. links

links is a text based browser which has full support for tables

links

is simply the hostname or IP address of the computer to connect to.

5. Gkrellm

Gkrellm is a graphical application that displays system information in real-time. It displays everything from processor consumption to if new email exists in that specific

user’s inbox.

6. Keybindings

Keybindings allow a user to to run a certain program or command by hitting only a small keycombo.

In Sawfish, click on the Keybindings capplet and select your keybindings by adding a new one, selecting the command, and then select the desired keybinding.

7. more

more causes any info sent to the terminal after more than one page has been output. This allows the user to see a page at a time, taking their time on each page, without them having to worry about the text rolling off the screen.

more can be used as a straight program to view a file or can be piped to by any other applications

Note: an alternative to more is less

8. grep

grep is a command that allows the user to specify text to show on the screen and display only the line containing the quered string.

| grep string

9. wc

Utility to print basic formatting information about a file.

wc or wc can be piped to

10. find

A simple utility made to search your hard disk.

find

11. locate

A quick alternative to find

locate

Who sent me mail

2006-03-13T02:10:00.000+08:00

To know who all have sent you mail.

---------------CUT HERE ----------
#!/bin/sh

: ${USER:=`expr "\`LANG=C id\`" : 'uid=[0-9]*($[^)]*$.*'`}

for MBox in /var/mail/$USER /usr/spool/mail/$USER ""
do
[ -r "$MBox" ] && break
done

[ -z "$MBox" ] && exec echo "No mailbox"

exec grep "^From " "$@" "$MBox"

---------------CUT HERE ----------

Strings - old but a goodie

2006-03-13T02:06:00.000+08:00

To check content of an object file(binary file ) we can't use vi or cat command, for that use strings
command.

% strings [name of binary file]

It will print all the printable strings present in object file. Basically strings command looks for ASCII strings in executable file and print it.

Great for core files and other binary error files

Run daily with at

2006-03-13T02:02:00.000+08:00

UNIX provides a command called "at" which can be used to run jobs according to the specfied time.

To run a particular job in every hour, every day use the following set of commands in a file called "at.sh" which will be executed recursively everyday.

########### CUT HERE ##################

#! /usr/bin/sh

# dt is a variable used to store
# current date

dt=`date | cut -c5-10`

# tm is a variable used to store
# current time

tm=`date | cut -c12-13`

while [ $tm -le 23 ]
do

# "at" is the command ad -f is the
# option used to execute a specified
# file. "file Name" should be an
# executable file.

at -f ./"file Name" $tm $dt
tm=`expr $tm + 1`
done

# With out manual intervention, automatic
# change over to the next day's job
# scheduling

at -f ./"File Name" 2358 $dt
dt=`expr $dt + 1`
at -f ./at.sh 0002 $dt

########### CUT HERE ##################

Find those hidden files

2006-03-10T09:06:00.000+08:00

Finding only hidden files (starting with .) in a directory.

ls -a | grep "^\." OR
ls -a | awk '$0~/^\./ {print $0}'

Run on last Sunday

2006-03-09T19:11:00.000+08:00

If you want a job to run on the last sunday of every month, you can use the following syntax from within cron:

18 * * * 0 [`date "+%d"` -gt 24] && /path/to/script

i.e. on sundays at 18:00 check if the day of the month is greater than 24 - if so run the job (if 23 is
specified the job will run on the last 2 sundays of the month)

NOTE: There back-ticks around the date command, not single quotes.

Simple Mask Overlooked

2006-03-08T18:48:00.000+08:00

Another way to CD to the long directory.

Works in both csh and ksh

cd too*

Will change to the directory "toobigtotype" As long as there is only one ocurrance of the word too....

This is great when there is only one sub directory within a directory, just use:

cd *

Auto word completion

2006-03-07T19:18:00.000+08:00

In tcsh, the TAB key will automatically complete any name in the current directory for any application (for pico, cd, rm, chown)... It will even complete paths from root.

Soo... If you want to open your .cshrc from root you do..

"pico /ho"[TAB] -> "pico /home/"
Then continue with "yuo"[tab] -> "yourusername"
Then you get "pico /home/yourusername/"
And then type ".cs"[TAB] -> ".cshrc"
So in the end you get "pico /home/yourusername/.cshrc"
In a lot less stokes...

But say there are 2 users called yourusername and yourothername.

Well, then the shell will beep, and if you hit TAB again, you will get a listing off all the files that start with "you". You can then continue to the point where the names differ, and then hit TAB.

This also works for commands.

Change the suffix

2006-03-07T19:13:00.000+08:00

If you want to change the suffix of multiple files, you can't do:

% mv *.abc *.def

However the following shell script can be used to do the required opperation:

***

Change all *.abc file to *.def the following shell script would work:

#!/bin/sh
for f in *.abc; do
mv $f `basename $f .abc`.def
done

How it works:

for f in *.abc; do

Set up a look for all files ending in .abc, and each time around setup $f as the filename

mv $f `basename $f.abc`.def

`basename $f .abc` takes the filename in $f and removes any trailing occurences of .abc, we then append .def to the result and the resulting command becomes "mv file.abc file.def"

done

Ends the "for" loop above.

Under "csh" or "tcsh" a similar thing could be done with:

foreach f in ( *.abc )
mv $f `basename $f .abc`.def
end

Deleting blank lines using grep

2006-03-07T19:09:00.000+08:00

For thos who are not familiar with awk, but still want a quick and easy way of removing blank lines from a flat ascii file, remember that the use of 'cat' in conjuction with 'grep' is just as effective.

cat file1 | grep -v '^$' >file2
mv -f file2 file1

Limit the size of data area

2006-03-07T19:00:00.000+08:00

You can limit the size of "data area" using ulimit and limit.

Under sh or ksh:
$ ulimit -d SIZE_IN_KB

Displays all current resource limits
$ ulimit -a

under csh or tcsh:
$ limit datasize SIZE_IN_KB

Displays all current resource limits
$ limit

For example:
$ ulimit -d 141073
$ ulimit -a

VI copy file to file

2006-03-06T19:20:00.000+08:00

Here is how to copy the required number of lines from one file to another in VI editor. First use the following key combinations in the source file.

Press ESCAPE
Press Shift "(Shift double quotes)
Press a
Press the number of lines you want to copy
press y followed by another y

Now press " : " (COLON) to get the vi prompt.
Hit e "Destination file name"
Once you enter the Destination file go to the line where you want the lines copied to be inserted.

Press ESCAPE.
Press SHIFT "(Double quotes).
Press a.
Press p.

The lines get copied.

Search several files

2006-03-06T19:13:00.000+08:00

To search several files for a string which you know sparsely, the following searching technique
would be useful:

#grep .*.*<... part>
or,
#ls|xargs grep .*.*<... part>

This will print all the filenames and lines which consists of "<0><0> ......"

For example, you want to search a data structure among several header files where it has been declared not where it is being used.

If the data structure be XYZ, then search will be:

#grep }.*XYZ

because you know it might be how
XYZ has been declared,
struct {
.....
.....
} XYZ

You don't know exactly how many blank characters are there unless you count them. But when XYZ will be used it will be done as follows(for example):

XYZ *xyz;

and you don't want them to find out as they will be coming in several header or C files.

Arithmetic Comparison

2006-03-05T18:51:00.000+08:00

In UNIX shell arithmetic comparison is limited to integer values. Here is a tip to compare floating values using basic shell commands.

--------- CUT HERE-----------------

#! /bin/sh
# test shell script
n1="01.401"
n2="01.350"

function compareFloatSmall
{
sort -n <<: | head -1
$n1
$n2
:
}

function compareFloatGreat
{
sort -r -n <<: | head -1
$n1
$n2
:
}

small=$(compareFloatSmall $n1 $n2)
echo "Comparing $n1 to $n2: smaller $less"
great=$(compareFloatGreat $n1 $n2)
echo "Comparing $n1 to $n2: greater $great"

--------- CUT HERE-----------------

Alternatively you can use a small 'awk' program.

--------- CUT AGAIN HERE-----------
#! /bin/sh
# A couple of examples in awk.
n1="03.550"
n2="02.550"

echo "$n1 $n2" | awk '{
if ( $1 >= $2 ) print $1
if ( $1 <= $2 ) print $2
if ( $1 > $2 ) print $1
if ( $1 < $2 ) print $2
if ( $1 == $2 ) print $1, $2
}'
--------- CUT AGAIN HERE-----------

Copy a tree with cpio

2006-03-05T18:50:00.000+08:00

Using standard UNIX tar to copy a tree doesn't preserve the original owner and group information for the directories and files during the copy. To do this, use find(1) piped to cpio(1) this way:

% cd
% find . -depth -print | cpio -pudm

This will create a mirror image of the
tree in .

Automatically set display

2006-03-05T18:49:00.000+08:00

To set automatically the DISPLAY variable after rlogin (csh):

if (`who am i | wc -w` > 5) then
setenv DISPLAY `who am i | cut -d"(" -f2 | cut -d")" -f1`:0.0
endif

May vary from flavor to flavor

File consistancy check

2006-03-05T18:47:00.000+08:00

To check the number of feilds and the consistancy of the file here is simple command:

nawk -F "delimiter" '{print NF}' file_name|sort -u|more

Lock down telnet or ftp

2006-03-04T18:38:00.000+08:00

When inbound access isn't required into a system deny users Telnet or FTP access do the following:

vi /etc/inetd.conf

Comment the line starts with Telnet or FTP. Save the file and exit.

Stop and start the inetd daemon now by following commands:

/etc/rc.d/init.d/inet stop

/etc/rc.d/init.d/inet start

(Your flavor may be /etc/init.d)

Now on nobody can telnet or FTP to your server from outside network.

Tracking of logins and logouts

2006-03-04T18:31:00.000+08:00

In the .login file add the commands:
------------------------------------

echo login time `date` >> .daylogs/masterlog

grep -i "sun" .daylogs/masterlog > .daylogs/sunday.log
grep -i "mon" .daylogs/masterlog > .daylogs/monday.log
grep -i "tue" .daylogs/masterlog > .daylogs/tuesday.log
grep -i "wen" .daylogs/masterlog > .daylogs/wensday.log
grep -i "thu" .daylogs/masterlog > .daylogs/thursday.log
grep -i "fri" .daylogs/masterlog > .daylogs/friday.log
grep -i "sat" .daylogs/masterlog > .daylogs/saturday.log

In the .logout file add this line
-----------------------------------

echo logout time `date`>> .daylogs/masterlog

This script assumes you have a hidden directory called .daylogs this helps keep it out of sight and away from prying eyes and if you keep root ownership of the directory change the mode to:

chmod 744 .daylogs

This will not allow anyone to get in to the directory to look around.

IRIX: check for new configs

2006-03-04T14:21:00.000+08:00

SGI loves to try and simplify your life with chkconfig switches to toggle various services on and off. After each upgrade, DOUBLE check the chkconfig switches. If something doesn't work all of the sudden check here.

For instance, tape drives in IRIX are disabled by a default install. To enable a tape subsystem:

# chkconfig ts on

# /etc/init.d/ts start

Operating on multiple files

2006-03-04T10:55:00.000+08:00

Have you ever felt the need to perform a set of operations on multiple files simultaneously???

Here is a solution for that.

For instance, if it is required to perform multiple operations like searching a string (using grep), and executing an awk or perl script etc, etc. on not just one file but a set of files, use the following commands at the unix prompt:

$<: foreach i ()
? echo $i
? grep $i > tmp
? awk -f awk_script tmp >> report
? ....
? ....
? end
$<:

The files list in the brackets can be either

* Specifically mentioned

* A unix variable which contains a list of file names. For instance, the variable "p" can be assigned all the files starting with string "data" as follows: set p = (data*)

Other examples are:
set g = `grep -l "Startpoint" * `
or
set all = *
(This assigns all file names in the current directory to the variable "all" )

And its usage with the "foreach" command will be as follows:

$<: foreach i ($all)
.....
.....
end

I sed blank

2006-03-03T23:24:00.000+08:00

Using sed, you can remove blank lines, and lines that contain only whitespace, from a file using the following:

sed -e '/^[ ]*$/d' InputFile >OutputFile

Within the single quotes ('), the forward slashes (/) delimit the regular expression that will be interpreted by sed. The "d" before the closing single quote, tells sed to delete any lines that match the regular expression.

Within the regular expression, the caret (^) matches the beginning of a line. The []* matches zero to many occurrences of the character list between the open bracket ([) and the close bracket (]) (in the above regular expression, you must insert a space and a tab between the brackets). The dollar sign ($) matches the end of a line.

These three constructs together match any blank line or any line that contains only spaces and tabs (in any combination).

Since the standard operation of sed is to echo lines to stdout, all lines except blank lines (or lines that only contain whitespace) will be sent to OutputFile.

Files opened by a process

2006-03-03T23:23:00.000+08:00

This is supported on Sun Solaris, not too sure about other Unix flavours.

I badly wanted to find out all the files opened by a process, and that is when I found pfiles.

Usage :
/usr/proc/bin/pfiles

Where pid is the process-id of the process

It lists the inode numbers of all the files, opened
by that process.

Backup the remote essentials

2006-03-03T23:19:00.000+08:00

There are many programs, data files, and especially license files that sit out on remote workstations and servers that you may should be backing up remotely.

If backing up the entire system puts a strain on your network, make sure that all the essential
files at least get backed up.

Some files may be
/var/flexlm
/usr/local/flexlm
/var/netls
/var/license
/etc/passwd
/etc/hosts
/etc/groups
/etc/netgroups
/etc/shadow
/vmunix
/kernel
etc....

String Removal

2006-03-03T23:15:00.000+08:00

What the following does:

rm `ls -al | grep str | awk '{if ($9 !~ /^str/) {print $9}'`

Removes all files that contains the string "str" excepts
those that begin with it. Changing the !~ to =~ does the
opposite.

Killing more users

2006-03-03T23:14:00.000+08:00

To kill all processes of a particular user from root
at unix prompt type:

# kill -9 `ps -fu username |awk '{ print $2 }'|grep -v PID`

We can also use the username as an argument and pass it from
command line, if this command is put as a script.

Bash Hotkeys

2006-03-03T14:59:00.000+08:00

Bash provides many hot keys to ease use. Like
ctrl-l -- clear screen
ctrl-r -- does a search in the previously given commands so that you don't have to repeat long command.
ctrl-u -- clears the typing before the hotkey.
ctrl-a -- takes you to the begining of the command you are currently typing.
ctrl-e -- takes you to the end of the command you are currently typing in.
esc-b -- takes you back by one word while typing a command.
ctrl-c -- kills the current command or process.
ctrl-d -- kills the shell.
ctrl-h -- deletes one letter at a time from the command you are typing in.
ctrl-z -- puts the currently running process in background, the process can be brought back to run state by using fg command.
esc-p -- like ctrl-r lets you search through the previously given commands.
esc-. -- gives the last command you typed.

More on VI lazy exists

2006-03-03T10:37:00.000+08:00

Addendum to VI LAZY EXITS

There is a difference between ZZ and :wq! ZZ will NOT force a write on a read only file :wq! will.

Also be careful with ZZ if you inadvertently press Control ZZ you will have stopped jobs.

Pipes and quoting

2006-03-03T10:34:00.000+08:00

When you do rsh (remsh in HP-UX) followed by a series ofpipes, consider carefully whether you want the pipe processesto run on the local or the remote host. If you quote thepipes, then they become part of the arguments passed tothe remote host and evaluated there. E.g.

rsh myRemoteHost last "" grep userName "" head

(In this example, we pipe the output of last to grep,because in Solaris "last userName" only shows consolelogins, not FTP logins.)

Spaces to underscore 1 liner

2006-03-03T10:30:00.000+08:00

Do you hate users who put spaces in file names, preventingyou from doing any sort of scripting?

Well then this is the little one-liner for you to pop in a shell script then...

for i in $1 ; do mv "$i" `echo $i sed 's/ /_/g'` ; done

the echo $i sed 's/ /_/g' is surrounded by back-tics not regular quotes.

Metavalues from a shell script

2006-03-02T22:58:00.000+08:00

Getting a file's size, link count or other metavalue in a shell script:

When you want to extract a specific piece of meta information about a specific file (such as it's size, link count, or one of its time stamps) you can use the find -printf option and any of the various %-directives listed in the find(1) man page. Example:

size=$(find some_file -printf "%s" )

(under ksh, bash, and similar shells). Here the $() form is used as a more readable equivalent to the older `` (back tick) operator. (Another advantage to the $() is that it is nestable). To get the user name of the owner of a file "foo" you could use:

set owner = `find foo -maxdepth 0 -printf "%u"`

(here we're using the csh syntax). This also uses the -maxdepth option in case "foo" is actually a directory name; since we don't want find to spend time traversing directory and printing the owners to ALL of the entries thereunder. A maxdepth of zero ensures that this will only print the detail we want on the specific link that we named on the command line.

This can be much more flexible than the options provided by the test command (try to see which of two files is larger, or if to files are owned by the same user or assigned to the same group, using just test).

Bash Tip: Convert Unix Newline to Dos Format

2006-03-02T12:57:00.000+08:00

If you’re on a Winblowz box and trying to read some source code from *nix open source projects, all you see are non-human readable format where there’s funny looking squares all over the place. It’s very annoying. You will see this if you use a text editor like Notepad, but Wordpad can convert those into human readable DOS format. Tonight, I finally wrote a bash script to convert all files in the current directory and all files under any subdirectories to DOS linefeed format. (I needed an excuse to practice my script-fu :) Here’s the script (there’s other ways of doing this), and save it as removeunixnewlines.sh. Make sure to chmod +x removeunixnewlines.sh):

#!/bin/bash
# This script will remove Unix newlines
# so it can be human readable in Windows
# COUNTER=0
for filename in $(find . -print)
do
if [ -f $filename ]
then
if [ $filename == “./removeunixnewlines.sh” ]
then
continue;
else
unix2dos –u2d $filename
let COUNTER+=1
fi
fi
done
# Print the number of files that have
# newlines removed
echo -n “$COUNTER ”
if [ $COUNTER -eq 1 ]
then
echo -n “file has ”
else
echo -n “files have ”
fi
echo “have Unix newlines removed.”
# Exit script
exit 0
You’ll need to run this under Cygwin in Winbloz. Navigate to the directory (e.g, cd /cgydrive/c/ ) where it contains files you want converted and execute by typing:$ ./removeunixnewlines.sh
As an exercise for you, young ninjas, expand this script to accept arguments that can either convert *nix to DOS linefeed format or vice versa. Get going, lil grasshoppaz!