Deutsch English
Jump to
Your feedback:
Did you like this page? vote3 Yes
vote2 Partly
vote1 No
Your comment?

Add your email address, if you want to get a response

Your name, if you like

Do not change this:
Feedback
Search

Tiny Linux / Unix Commandos

06-08-2005 13.09

No More Syslog Messages on Your Console

dmesg -n level
13-07-2008 14.33

Start a New Xserver

startx fvwm2 -- :2 -depth 16
13-07-2008 14.33

xlock as Screensaver

xautolock -exit 2>/dev/null; xautolock -corners "+---" -time 10 -notify 30 -locker "xlock -mode random"
13-07-2008 14.33

Make Interactive Services Scripable with expect

#!/usr/bin/expect
set myserver "10.0.0.1"
set myport "21"
send "Try to connect to server $myserver port $myport\n"
spawn telnet $myserver $myport
expect "220 " { send "USER ftp\n" }
expect "331 " { send "PASS foo\n" }
expect "230 " { send "PASV\n" }
expect "227 " { send "HELP\n" }
expect "214 " { interact }
13-07-2008 14.33

Download a File and Split It on the Fly

wget ftp://... -nv -O - | split ....
13-07-2008 14.33

Sync Two Files (Directories)

/usr/bin/rsync --rsync-path=/usr/local/bin/rsync -rlpe ssh QUELLE ZIEL
rsync --compress --progress --no-whole-file -rlpe ssh remote:/path/file file
13-07-2008 14.33

List Files Sorted by Filesize

ls -lS

Get file attributes

Get file attributes which can be easily parsed
stat myfile.txt
04-08-2010 00.07

Count from ALPHA to OMEGA

seq ALPHA OMEGA
13-07-2008 14.33

Text filter


Today's Messages from /var/log/syslog

cat /var/log/syslog | grep ^"`date +%b\ %d`"

Format of date output

# date +'%Y%m%d_%H%M%S'
20081103_1759026

Filter Empty Lines and Comments

grep -ve ^\ *\#'\|'^$ /etc/*.conf

Highlight what machted

echo "ABC123DEF456GHI" | grep --color [0-9H]
ABC123DEF456GHI

List each match

echo "ABC123DEF456GHI" | grep -E --only-matching [0-9H]+
123
456
H

List only the names of the files where hits where found

grep --files-with-match foo *

References in sed to machted parts

echo "ABC123DEF456GHI" | sed 's/[0-9H]\+/_&_/g'

ABC_123_DEF_456_G_H_I

echo "ABC223DEF456GHI" | sed s/"\([0-9]\+\)"/"_\1_"/

ABC_223_DEF456GHI

use awk instead of grep

If you use grep to find lines and the awk to print columns
grep error /var/log/syslog | awk '{ print $3 }'

you can save the grep call as awk can also grep for you
awk '/error/ { print $3 }' /var/log/syslog

Regular Expression Lookahead

If you have the following String
EXAMMLE
and want to replace the second M with a P.

Positiv Lookahead

Search for an M which is followed by an L
M(?=L)
The difference to this one
ML
is that the the L in the first case would not be replaced, but in the second case it would be replaced

Negativ Lookahead

Search for an M, which is not followed by an M
M(?!M)
Search for a String which does not start with CA
^(?!CA)

See also Regular Expressions Lookaround
18-03-2013 01.28

awk

Use different field separator for awk

awk 'BEGIN {FS=" - "} {print $1 $2 $3}'
awk 'BEGIN {FS="\t"} {print $3}

Split a String with awk at a given separator

echo foo,bar,test | awk 'BEGIN {FS=","} {split($0,a,","); for (i=1; i<=NF; i++) print a[i]; }'

Calculate with awk

echo 42 11 | awk '{print $1*1000, $2}'

Add all values in a column
cat foo | awk '{mycol2 += $2; mycol3 += $3}; END {print mycol2":"mycol3}'


Only deal with lines which match a given condition

Only deal with lines where the first column is larger than 45
cat foo | awk '$1>45 {print $0}'

Only deal with lines beginning with line number 12
cat foo | awk 'NR>11 { print $0 }'

Only deal with lines where in the second column is the word "FooBar":
cat foo | awk '$2~/FooBar/ {print $0}'

Define output format for awk

echo 1 2 | awk '{ printf("%.3d %s\n", $1, $2)  }'
22-09-2012 11.37

Console Calculation

echo "1 + 2" | bc
echo "1 / 3" | bc -l
14-07-2008 19.56

Differences between two files

diff -ubB FOO BAR

Test a string vs a regular expression

expr FooBar-5.2 : ".*-5.2"
18-05-2008 15.06

Rename files with mmv

Rename *a* to *b*:
mmv "*a*" "#1b#2"
13-07-2008 14.33

Mount SMB Share

mount -t smbfs -o username=uname,password=passwd //smbserv/share /destination

Label fat32 vfat Filesystem

mlabel -i /dev/sdc1 ::FOOBAR
26-01-2010 00.03

Kernel


Build Kernel (Traditionally)

nice --5 make dep clean bzImage modules bzlilo modules_install

Kernel Bootparameter

/usr/src/linux/Documentation/kernel-parameters.txt
22-09-2012 11.37

Create Xface

48x48 ppm -> pgm -> pbm -> xbm
cat file.xbm | xbm2ikon > file.48
compface file.48 file.face
13-07-2008 14.33

pdf2ps (with Password)

gs -sPDFPassword=password -sDEVICE=pswrite -dNOPAUSE -sOutputFile=out.ps in.pdf

Convert some files in a pretty looking file suitable for printing

a2ps --medium A4dj -E -o /tmp/pretty.ps A B C D ...
Unicode / utf-8: u2ps ...(gnome-u2ps)

Create a Postscript Book

psbook normal.ps | psnup -2 -m25 > book.ps
22-09-2012 11.37

Multimedia


vob2avi

Convert VOB files with Transcode into another video format.
transcode -i myVobFolder -o movie.avi -y xvid -x vob
Scale
-B 4
Select a different audio channel, don't select a non existing audio channel or your VOB files will only be read once and no content will be generated
-a1
Encoder bitrate
-w 1800
Clipping, this cuts the first and last 16 rows of
-j 16,0
Linear interpolation for interlaced videos
-I1
Cut the results
avisplit -c -i movie.avi -t 00:00:00.00-00:05:10.00,00:11:59.00-00:22:41.00
Use tcaud
-y xvid,tcaud
Create DivX files
-y ffmpeg -F mpeg4
Use mencoder instead of transcode
cat *.vob | mencoder -oac mp3lame -ovc xvid -xvidencopts pass=1             - -o movie.avi;
cat *.vob | mencoder -oac mp3lame -ovc xvid -xvidencopts pass=2:bitrate=700 - -o movie.avi;

mkv2avi

mencoder -oac pcm -ovc copy  -ofps 30000/1001 -o Movi.avi Movi.mkv

Extract audio from video

nice transcode -i vts_01_6.vob -o soundtrack.wav -y null,wav -x vob -e 44000
mplayer -ao pcm:fast:file=audio.wav -vo null -vc null video.avi

jpg2avi

Mplayer

mencoder 'mf://*.jpg' -mf type=jpg:fps=10 -xvidencopts bitrate=1600 -ovc xvid -oac copy -o output.avi

ImageMagick

animate -delay 10 *.jpg
animate -help

convert -delay 10 *.jpg out.gif

Automated picture manipulation

ImageMagick
convert large.jpg -resize 120x120 small.jpg
convert large.jpg -resize 50% small.jpg
Got pictures from different cameras from one event? Use the exif header in the JPEG pictures to extract the date and time and put it into the filename so you can sort the files chronologically.
Make a backup and check you don't overwrite files!
for i in *.jpg; do d=`exif -t 0x9003 "$i" | grep Value | sed s/^"  Value: "// | sed s/\ /_/g | sed s/:/-/g`; mv "$i" "$d""_""$i"; done
Change the date from 2005 to 2006
for i in *;
do
n=`exif -t 0x9004 "$i" | grep ^" Value: " | sed s/^" Value: "// | sed s/2005/2006/`;
x="exif --ifd=EXIF -t 0x9004 --set-value=\"$n\" $i";
eval "$x";
done
Increase time by one hour
for i in *.jpg;
do
value=`exif -t 0x9003 "$i" | grep Value | sed s/^" Value: "//;`
value_date=`echo "$value" | awk '{ print $1 }'`
value_time=`echo "$value" | awk '{ print $2 }'`
value_date_yyyy=`echo "$value_date" | awk 'BEGIN {FS=":"} {print $1}'`
value_date_mm=`echo "$value_date" | awk 'BEGIN {FS=":"} {print $2}'`
value_date_dd=`echo "$value_date" | awk 'BEGIN {FS=":"} {print $3}'`
value_time_hh=`echo "$value_time" | awk 'BEGIN {FS=":"} {print $1}'`
value_time_mm=`echo "$value_time" | awk 'BEGIN {FS=":"} {print $2}'`
value_time_ss=`echo "$value_time" | awk 'BEGIN {FS=":"} {print $3}'`

value_time_hh=`expr $value_time_hh + 1`
if [ $value_time_hh -lt 10 ]; then
value_time_hh="0$value_time_hh"
fi

newvalue="$value_date_yyyy:$value_date_mm:$value_date_dd $value_time_hh:$value_time_mm:$value_time_ss"

echo "$i" "$value" " -> " "$newvalue"

exif --ifd=EXIF -t 0x9003 --set-value "$newvalue" "$i"
done
20-10-2013 15.35

unix2dos, dos2unix

alias unix2dos='recode lat1..ibmpc'
alias dos2unix='recode ibmpc..lat1'

Convert file encoding from ISO to Unicode

iconv --from-code=ISO-8859-1 --to-code=UTF-8 ./oldfile.htm > ./newfile.html
04-08-2010 00.07

ACL

setfacl -m u:foo:r-- DEST
setfacl -x u:foo DEST
setfacl -m g:bar:r-- DEST
setfacl -x g:bar DEST
setfacl -m d:u:foo:r-- DIR
setfacl -m d:g:bar:r-- DIR
setfacl -m m:r-- DEST
setfacl -m d:m:r-- DIR
13-07-2008 14.33

Be Able to Start Several Instances of Mozilla

alias my_mozilla='mozilla -remote "ping()" && mozilla -remote "openURL (,new-window)" || mozilla -splash'
17-01-2010 01.49

Spamassassin

Bayes: How Many Mails Have Been Learned

sa-learn --dump | grep non-token\ data: | grep nspam$'\|'nham$

Bayes: List with All Learned Words and if They Were Found in a Spam Message

sa-learn --dump | grep -v non-token\ data: | sort -n

Send mails from the command line

(echo "Hello world"; uuencode file1.txt ReadThis.txt; uuencode file2.txt AndThis.txt) | mailx -s subject foo@example.com

echo "Hello world" | mutt -a file1.txt foo@example.com
18-05-2008 15.06

Upload files with sftp

If you can not use scp for any reason
sftp -o "batchmode no"  -b meinebefehle.sftp user@computer

meinebefehle.sftp:
lcd /home/foo/4upload/
cd /tmp/uploaded/
put Backup1.img
put Backpu2.img
quit
17-01-2010 02.37

Archiving / Backup


Size of some storage media
Name(Bytes)
DVD4590200
CDFIXME

Backup Remotehost with tar

ssh root@otherhost -c blowfish 'cd /YOURPATH; tar -f - -p -P -c --exclude=/YOURPATH/tmp --exclude=/YOURPATH/var/cache /' | dd of=/daten/backup.tar

tar multi volume

tar --multi-volume --tape-length=1024 -cf archive.tar foo // n archive.001.tar n ...
tar --multi-volume --tape-length=1024 -c -f archive.tar -f archive001.tar -f archive002.tar -f ... foo
tar --multi-volume --tape-length=1024 --new-volume-script=/usr/local/bin/tar_changer.sh -cf archive.tar foo
tar --multi-volume -t -f archive.tar -f archive001.tar -f archive002.tar -f ...

tar_changer.sh:
#!/bin/sh

new_vol=`echo $TAR_VOLUME | awk '{printf "%04d\n", $1;}'`

new_name="$TAR_ARCHIVE"
new_name="`echo "$new_name" | sed s/"\."tar$//`"
new_name="`echo "$new_name" | sed s/"\."[0-9][0-9]*$//`"
new_name="$new_name"".""$new_vol"".tar"

echo "For volume $TAR_VOLUME of $TAR_ARCHIVE the next file name will be: $new_name"

if [ -e "$new_name" ]; then
echo "Error: file already exists!"
exit 1
else
echo "$new_name" >&$TAR_FD
fi

Add redundancy to files

par2 create myparfiles foo*

Create a DVD image larger than 2GB

mkisofs -udf -allow-limited-size -o foo.img foo

Complete Compressed Partionbackup Into an Image

mount /dev/hda1 / -o remount,rw;
dd if=/dev/zero of=/delme1 bs=4048; rm /delme1;
mount /dev/hda1 / -o remount,ro;

Local

dd if=/dev/hda1 bs=4048 | bzip2 > /mountpoint/image_hda1.bz;

Remote


ssh host -c blowfish 'dd if=/dev/hda1 bs=4048 | bzip2' | dd of=/image_hda1.bz
17-01-2010 01.49

Software Raid

Software RAID
13-07-2008 14.33

Turn Beep Off (e.g. while autocomplete)

xterm: xset b off
setterm -blength
echo 'set bell-style visible' >> ~/.inputrc
13-07-2008 14.33

$TERM

vt100
xterm
linux
13-07-2008 14.33

chmod

chmod ABCD FILE
A:
4=user ID, 2=group ID, 1=sticky
B(owner), C(group), D(other):
4=r, 2=w, 1=x
06-08-2005 13.09

Restore Bootmanager

Backup your Bootsector

if=/dev/hda of=/tmp/mbrbackupwithouttable.img bs=446 count=1
if=/dev/hda of=/tmp/mbrbackupwithtable.img bs=512 count=1

Lilo

Boot from Floppy/CD, mount /dev/linuxrootpartition /mnt; chroot /mnt; lilo; exit; umount /mnt; reboot

Grub

  • Start the Grub shell
  • Find out where your Linux partition is (e.g. use find to search one of your files) and tell grub with the root command where it is.
    grub> find /boot/grub/stage1
    grub> root (hd0,3)
  • Write Grub into the masterbootsector or in one of your partitions
    grub> setup (hd0)
    grub> setup (hd0,3)

yaboot

ybin
13-07-2008 14.33

less

export LESS="-i"

Upper- and Lower Case

less

If you do a lowercase search in less the search will ignore upper and lower case
export LESS="-i"

sed

echo Cool | sed s/c/T/i

Change Upper and Lower Case

echo fOo | tr "[:lower:]" "[:upper:]" # FOO
echo fOo | tr "[:upper:]" "[:lower:]" # foo
09-12-2007 02.27

Sort after special columns

First after the 3. one, then the 2. and finally the 6. columns (the last one numeric).
sort -k 3,3 -k 2,2 -k 6,6n

Do not only choose which column is used for sorting, but also define the column separator (here ":"):
# echo "1:1:1
1:1:0" | sort -t : -k 3

1:1:0
1:1:1
18-05-2008 15.06

Bash

Bash Array

Define and initialise an array in the Bash shell. Afterwards various outputs of the array.
# fruits=(apple banana strawberry)

# echo ${fruits}
apple

# echo ${fruits[1]}
banana

# echo ${fruits[*]}
apple banana strawberry

# echo ${#fruits[*]}
3

# echo ${fruits[*]}
apple banana strawberry

# for ((idx=(${#fruits[*]}-1); idx>=0; idx--)); do echo ${fruits[$idx]}; done
strawberry
banana
apple

Bash history

Bash Event Designators
!! Run the last command you entered again
!-3 Run the command you entered three steps before this one
!mount Re-run the last command which started with mount
!!:s/mount/umount/ Re-run the last command and replace the string mount with the string remount
^mount^umount^ Alternative for the last command
export HISTSIZE=1000 Remember the last 1000 commands
export HISTCONTROL=ignoreboth Don't store commands which start with a space or successive identical entered commands
shopt -s histappend The last Bash session you close won't overwrite the history
shopt -s histverify Commands you repeat with ! are not executed immediately

Shortcuts

ShortcutDescription
Ctrl+Amove to the start of the line
Ctrl+Emove to the end of the line
Ctrl+UCut from the cursor to the beginning of the line into the clipboard
Ctrl+KCut from the cursor to the end of the line into the clipboard
Ctrl+WCut from the cursor to the start of the word into the clipboard
Ctrl+Ypastes text from the clipboard
22-09-2012 11.37

find

find / -printf "%CY%Cm%Cd%CI%CM | %d | %i | %s | %h | %f\n"
find can spawn a new command for each file or start the command once and give it all found files
find / -exec echo  '{}' +
find / -exec echo '{}' ';'

Find old files

List files which are at least 165 days old, exactly 165 days old, not older than 165 days
find /foo -type f -ctime +165
find /foo -type f -ctime 165
find /foo -type f -ctime -165

Find large files

List files which are at least 3MB large
find /foo -size +3000000c
find /foo -size +3M

Us find to find files with given file permissions
CommandDescriptionWill findWill not find
# find . -perm 400
# find . -perm u=r
Finds files with exactly these permissions -r-------- -r--r-----
# find . ! -perm 400
# find . ! -perm u=r
Finds files whose permissions are not equal to these permissions -rwxrwxrwx -r--------
# find . -perm -440
# find . -perm -u=r,g=r
At least these permissions have to be there -rwxrwxrwx
-rw-r--r--
lrwxrwxrwx
-r--------
----r-----
# find . -perm /440
# find . -perm /u=r,g=r
At least one of these permissions have to be there -rwxrwxrwx
-r--------
----r-----
--w-------

Find dangling symlinks
find /tmp/tg1/ -type l | while read link
do
if [ -h "$link" -a ! -e "$link" ];
then echo "$link";
fi;
done

find /tmp/tg1/ -printf '%Y %p \n' | grep ^N\  | sed s/^N\ //

List all files which are linked to the given file
find /mnt/mnt/ -L -samefile /mnt/mnt/bin/foo
18-03-2013 01.28

OSX

Display and Modify Routes in OSX

netstat -rn
route delete default
route add default 1.2.3.4

Battery status

ioreg -l | grep IOBa
17-01-2010 02.37

Detect interactive shells

if shopt -q login_shell ; then
...
else
...
fi
16-03-2008 09.41

Jump to recently visited directories


Push a new directory to the stack and change to that directory

# pwd
/tmp

# pushd /etc/network/

# pushd /var/log/
/var/log /etc/network /tmp

# pushd /usr/sbin/
/usr/sbin /var/log /etc/network /tmp

# dirs
/usr/sbin /var/log /etc/network /tmp

A simple pushd swaps the topmost directory on the stack with the next one and changes the current directory to the new topmost one. Very handy to jump between two directories.

# pushd
/var/log /usr/sbin /etc/network /tmp

# pwd
/var/log

# pushd
/usr/sbin /var/log /etc/network /tmp

# pwd
/usr/sbin

With +x or -x you can select the x. directory (count starts with zero) counted from the left or the right side to be the new topmost directory.

# pushd +2
/etc/network /tmp /usr/sbin /var/log

/# pwd
/etc/network

With popd the topmost directory is removed from the stack and the current directory will be set to the new topmost directory. With -n the current directory is not changed.
# dirs
/etc/network /usr/sbin /var/log

# popd
/usr/sbin /var/log

# pwd
/usr/sbin

# popd
/var/log

# pwd
/var/log

With +x and -x you can again remove the x. directory counted from left or from right. If you remove the topmost directory (+0) the current directory is change to the new topmost directory.

You can also get the last directory you were before the current directory with the following variable
echo "$OLDPWD"
16-03-2008 09.41

Command Substitution

Repeat last token of the last command

# echo /etc/group /etc/passwd >/dev/null
# echo !$
echo /dev/null
With
ESC .
you get the last token at the current cursor position.

Repeat the last argument of the last command

# cp backup2008a.tar /mnt/streamer/backup2008a.tar 2>/dev/null
# echo $_
/mnt/streamer/backup2008a.tar

Repeat the last command with a substitution

# cp backup2008.tar /mnt/streamer/
# ^2008^2009^
cp backup2009.tar /mnt/streamer/
However, this substitutes only the first occurrence, which also applies to the long version
# cp backup2008a.tar backup2008b.tar /mnt/streamer/
# !!:s/2008/2009/
cp backup2009a.tar backup2008b.tar /mnt/streamer/

The long version can also substitute all occurrences
# cp -v backup2008a.tar backup2008b.tar /mnt/streamer/
# !!:gs/2008/2009/
01-01-2009 20.44

Change output of a shellscript and restore output again later

exec 3>&1
exec 1>>/var/log/mylogfile.txt
...
exec 1>&3
exec 3>&-
16-03-2008 09.41

Change file content without temporary files with sponge

Usually you have to use temporary files if you want to change the content of a file, as you can not read and write to a file simultaneously. This can be very inconvenient, especially as secure creation of temp files is not too easy.
One solution for this problem is sponge:
cat foo.txt | sed s/t/T/ | sponge foo.txt

However, this does only work if sponge writes to a file. If you redirect its output this will not work. In Debian sponge can be found in the moreutils package.
16-03-2008 09.41

cygwin complains about too open permissions

If you use ssh in cygwin it might happen that you get a complain that the permissions of your private ssh key are too open.
$ ssh foo@bar.example.com
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/cygdrive/p/.ssh/id_dsa' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /cygdrive/p/.ssh/id_dsa
Permission denied (publickey).

You can overwrite this warning like this
$ CYGWIN=nontsec ssh foo@bar.example.com
16-03-2008 09.41

System Infos

# free -m
# df -ah
# top -n1
htop
# vmstat 5 3
smartctl --all /dev/sda
memstat -w

Who blocks a mounted drive

If you can not unmount a mounted drive find out who blocks it
lsof +D /mnt/foo
fuser /mnt/foo

Get the number of CPU cores

# nproc
6

Use this information to start make with multiple threads
# make -j $(nproc)

Benchmarks

Use zcav to measure the speed of a mass storage device depending on the position on the drive you are currently reading from
zcav /dev/sdb > zcav.txt
gnuplot
set yrange [0:*]
set format y '%g Blocks/s'
set format x '%g MB'
plot "zcav.txt" using 1:2 title "Foo"
Benchmark a drive with bonnie++
bonnie++ -u root -d /mnt/mnt/
20-01-2013 22.36

Solaris

More verbose ps in Solaris

In Solaris there is the normal ps with
ps -ef
prstat -a
truss -f /bin/date
truss -fp PID
but you also have another ps which can be more verbose
/usr/ucb/ps -auxww

Solaris 10 md5sum

digest -v -a md5 foo
20-01-2013 22.36

Perl

Install Perl module

# perl -MCPAN -e 'install HTML::CalendarMonthSimple'

# cpan
cpan> install HTML::CalendarMonthSimple

Test if a module is already installed
require HTML::CalendarMonthSimple;

Perl Debugger

perl -d myPerlScript.pl

s execute the next command (and step also in method calls)

n next command (but skip method calls)

Return repeat last selection

x $FOO prints the value of the variable

r run until the end of the method (or the next breakpoint)

c run until the end (or the next breakpoint)

l (small L) print the next 10 lines of code

- print the previous 10 lines of code

S print all method names

S main print all method names

b METHODENNAME Brakepoint for this method

b NUMMER Brakepoint for this line number (use l (small L) to get the line numbers)

t print the executed lines

q quit debugger

See also Perl Debugger
18-03-2013 01.28

Use xargs to start processes in parallel

One gzip process gets a long List of files to work with
ls *.txt | xargs gzip

One gzip for each file
ls *.txt | xargs -n1 gzip

One gzip for each file, all start in parallel
ls *.txt | xargs -n1 gzip &

One gzip for each file, only 5 run in parallel
ls *.txt | xargs -n1 -P5 gzip
03-05-2010 01.13
Powered by PHP Created with Xemacs Valid XHTML 1.0! Valid CSS!