systemctl daemon-reload
systemctl start foo
systemctl enable foo
systemctl stop foo

Get current dns settings

systemd-resolve --status

Constant Names for Network Interfaces

# cat /etc/udev/rules.d/local-interfaces.rules
KERNEL="eth*", SYSFS{address}="00:48:54:55:00:00", NAME="heimnetz"

Now replace eth0 with heimnetz everywhere:

find /etc/ -type f -exec grep -He 'eth0' '{}' ';'

Now your network interface has a constant name.

Traffic Prioritisation

Real-Time applications like e.g. VOIP will benefit a lot from this: .


VPN to connect two computers (or two networks (or one computer and one network)) over an unsecure connection. Your situation may look like this:

( NET_1 )-[IP__1]~~~~~~~~~[IP__2]-( NET_2 )
  • Which passwords to use:

Change this to your own one or use

pwgen -s 24 2

to create a new one.

  • IP addresses of the gateways and the networks behind them. In this examp,e 1 is a gateway to the world and 2 only a single computer without a network behind it.
  • Now assign the passwords to the IP addresses, start the tunnel and deny unecrypted packages (this one is for the gateway 1):
echo "

add $IP__1 $IP__2 esp 12340 -m tunnel -E 3des-cbc "$PASS1";
add $IP__2 $IP__1 esp 12341 -m tunnel -E 3des-cbc "$PASS2";

spdadd $IP__2 $NET_1 any -P in ipsec esp/tunnel/$IP__2-$IP__1/require;
spdadd $NET_1 $IP__2 any -P out ipsec esp/tunnel/$IP__1-$IP__2/require;

" | "$SK" -c

For gateway 2 just swap "in" and "out":

echo "

add $IP__1 $IP__2 esp 12340 -m tunnel -E 3des-cbc "$PASS1";
add $IP__2 $IP__1 esp 12341 -m tunnel -E 3des-cbc "$PASS2";

spdadd $IP__2 $NET_1 any -P out ipsec esp/tunnel/$IP__2-$IP__1/require;
spdadd $NET_1 $IP__2 any -P in ipsec esp/tunnel/$IP__1-$IP__2/require;

" | "$SK" -c



/etc/fstab: /dev/foo /mnt/bar ext3 defaults,usrquota 0 2
mount /dev/foo /mnt/bar -o remount

quotacheck -acguvm
quotacheck -cguvm /mnt/bar

edquota -u USERNAME
edquota -u USERNAME -f /dev/foo
edquota -p USERNAME -u USERNAME2 -f /dev/foo

edquota -t

quotaon -avug
quotaoff -avug

repquota -a

su USERNAME -c 'quota -v'

Software Raid

Create partitions with the type fd, e.g. /dev/sda1 und /dev/hda1

mdadm --create /dev/md0 --verbose --level=raid1 --raid-devices=2 /dev/hda1 /dev/sda1
mdadm --create /dev/md1 --verbose --level=raid5 --raid-devices=3 /dev/hda5 /dev/sda5 /dev/sdb5
mdadm --create /dev/md1 --verbose --level=raid5 --chunk=128 --raid-devices=3 /dev/hda5 /dev/sda5 /dev/sdb5

If there are not all hard disks available yet which you want to add to the raid, you might use the word missing instead of the partitions on that disk.

mdadm --create /dev/md0 -vv --level=raid1 --raid-devices=2 /dev/sda1 missing

If you get this error

mdadm: error opening /dev/md0: No such device or address

and the device is really there, check if you loaded the module md.

Do not forget to add the output of

mdadm --detail --scan

to the file /etc/mdadm/mdadm.conf.

If you loose one of your hard disks, this is how you can see which partitions of your raid are still there

cat /proc/mdstat
mdadm --detail /dev/md0

You can check if a partitions was previously part of a raid

mdadm -Q /dev/hda1 /dev/hda1: device 0 in 2 device mismatch raid1 /dev/md0. Use mdadm --examine for more detail.

If you have a new hard disk you can add partitions to a raid like this

mdadm --manage --add /dev/md0 /dev/hda1

One of these errors

mdadm: add new device failed for /dev/hda1 as 2: No space left on device
mdadm: add new device failed for /dev/hda1 as 2: Invalid argument

probably indicates that your partition is too small for the raid. Check yourself

cat /proc/partitions

The size of the raid can also be changed (number of used partitions / utilisation of used partitions)

mdadm /dev/md1 --grow --raid-devices=4
mdadm /dev/md1 --grow --size=max

One day I noticed that one of my devices (in my case sdb) of a raid started to have problems, read errors, ... After getting a replacement device connect it additionally (in my case came up as sdd). Then tell the raid system it should replace the failing one:

mdadm /dev/md0 --add /dev/sdd1
mdadm /dev/md0 --replace /dev/sdb1 --with /dev/sdd1

This did not work at first

mdadm: Marked /dev/sdb1 (device 1 in /dev/md0) for replacement
mdadm: Failed to set /dev/sdd1 as preferred replacement.

For my the reason was that there was already a sync running in the background due to the failing device (check with cat /proc/mdstat). This is how to stop the sync and try again

echo "idle" > /sys/block/md0/md/sync_action

Another time I temporally lost a disk of a raid5, I had troubles restarting the raid. This is what I did to recover from this situation (written down from memory)

  • Boot from CD (e.g. Knoppix), if necessary.

/etc/init.d/mdadm start

mdadm --detail --scan >> /etc/mdadm/mdadm.conf
/etc/init.d/mdadm-raid restart
  • Check where the problem is
mdadm --detail /dev/mdX
  • Although I only lost one drive, the raid refused to start again
raid5: cannot start dirty degraded array for md1
  • I got it back with assemble and the force option
mdadm --assemble --scan --run --force

Watch the progress of the recovery like this

watch -n5 cat /proc/mdstat

You can speed up the minimum speed progress, e.g. increase it to 50MB/s

echo 50000 >/proc/sys/dev/raid/speed_limit_min

You can remove partitions from a raid like this

mdadm --fail /dev/md0 /dev/hda1
mdadm --remove /dev/md0 /dev/hda1

Stop a running raid

mdadm --stop /dev/md1

Remove raid markers from a partition

mdadm --zero /dev/hda5

If you like to boot from a raid1, it is important to write the bootmanager to both harddrives. Just act as if the both partitions of your raid are regular partitions and no raid1 devices. First add both hard disks to the file


Call one hd0 and one hd1. Then start grub, specify the device where /boot is installed (,0 is the first partition) and then write the bootsector of both hard disks

root (hd0,0) setup (hd0)
root (hd1,0) setup (hd1)

Now try to boot with only one of the hard disks attached. Do not forget to repair the raid in between.


LVM Howto

Overview of the elements of an LVM system

PV PV PV (e.g., /dev/hda, /dev/hdb5, /dev/sdb3)
| /
/ |
LV LV LV (e.g., /mnt/mydata, /mnt/moredate)

Prepare PV (discs or partitions)

# pvcreate /dev/sdb1
# pvdisplay -m /dev/sdb1

--- Physical volume ---
PV Name /dev/sdb1
Allocated PE 54206

Create new VG with the prepared PV

# vgcreate MYLVMVOLUME1 /dev/hda /dev/hdb5
# vgchange -a y MYLVMVOLUME1

Remove VG

# vgchange -a n MYLVMVOLUME1
# vgremove MYLVMVOLUME1

Add new PV to VG

# vgextend my_volume_group /dev/sdb3

Remove PV from VG

(e.g., when want to remove an old disc) Is there enough space even without the PV?

# pvscan
# pvdisplay /dev/hda

If not, first add a new disc like described above:

  • Prepare discs / partitions
  • Add new PV to VG

Then do a backup! Note that the following will require that you enabled the kernel option


Move everything from the old discs to the other PV

# pvmove /dev/hda

Or specify to which PV the data should be moved

# pvmove /dev/hda /dev/hde3

Now remove the PV

# vgreduce MYLVMVOLUME1 /dev/sdb3
# pvremove /dev/sdb3


The command pvmove obviously requires, that you have enough space left on your VG which has not yet been occupied by a LV. But even when you have enough free space in total, pvmove will need your help if the amount of space is not available on a single PV. Even you try to move for example this sdb1

# pvcreate /dev/sdb1
# pvdisplay -m /dev/sdb1

--- Physical volume ---
PV Name /dev/sdb1
Allocated PE 54206

but there is no other PV which can carry it, this will fail

# pvmove /dev/sdb1
Insufficient suitable contiguous allocatable extents for logical volume pvmove0: 43955 more required

Just help it with some manual intervention. Use pvdisplay to find the PV with the largest value for Free PE for your LV. In this example it is sda6

# pvdisplay

--- Physical volume ---
PV Name /dev/sda6
Free PE 33267
Allocated PE 0

Now you can ask pvmove to move 33267 PE on this PV

# pvmove /dev/sdb1:0-33267 /dev/sda6

Check if the destination is now full, if not, just repeat the last command with an increased upper limit. When it is finally full just move the next area

# pvmove /dev/sdb1:33268-40000 /dev/hda6

to the next PV with some free space. If you specify an area where parts have already been moved, this parts will be automatically ignored. So this one

# pvmove /dev/sdb1:0-40000 /dev/hda6

would be equivalent in our case, as we moved the part at the beginning already. However, this method might leads to LV which are fragmented. A better way is, to initiate a backup of the LVM structure


change into this directory


and read the information about your LVM. Let's assume you have the following output

segment2 {
start_extent = 1
extent_count = 250 # 1000 Megabytes

type = "striped"
stripe_count = 1 # linear

stripes = [
"pv2", 25346

The source device and the start point can be found in the stripes section


This shows how long it is

extent_count = 250

Now we only have to find the mapping for pv2 to a real device in the file. In this example we assume it is sda6. Now the segment can be moved completely to another device

pvmove -v /dev/sda6:25346-`echo 25346 - 1 + 250 | bc` /dev/sdb6

If you move the segments of a LVM consecutively to the same device they will merge automatically.

Create LV on a VG

How much space is left?

# vgdisplay MYLVMVOLUME1

Create LV with 1.5GB size, linear

# lvcreate -L1500 -n MYDATA MYLVMVOLUME1

Create LV with 1.5GB size, stripped

# lvcreate -i2 -I4 -L500 -n MOREDATA MYLVMVOLUME1

Create LV with 1.5GB size, linear and use only /dev/hdb5

# lvcreate -L1500 -n FASTDATA MYLVMVOLUME1 /dev/hdb5

Now create a filesystem on it

# mkfs.ext3 -L NAME -O dir_index -T news -m0 -N 250000 /dev/MYLVMVOLUME1/FILESYSTEM1
# mkfs.ext4 -L NAME -O dir_index,extent -T largefile -m0 /dev/MYLVMVOLUME1/FILESYSTEM1

# tune2fs -c 31

Remove LV


Extend LV

How much space is left?

# vgdisplay MYLVMVOLUME1

Extend +1 GB (negativ numbers are discussed below, dangerous!)

# lvextend -L+1G /dev/MYLVMVOLUME1/MYDATA

Now extend the filesystem (ext2 / ext3)

# resize2fs -p /dev/MYLVMVOLUME1/MYDATA

Extend the filesystem (reiserfs, unmount may be optional)

# resize_reiserfs /dev/MYLVMVOLUME1/MYDATA
# mount -treiserfs /dev/MYLVMVOLUME1/MYDATA

Shrink LV

Use lvm to shrink lvm and filesystem in one step

lvresize --resizefs --size 40G /dev/mapper/foo

Do it by yourself:

First reduce the filesystems size, then the LV! This will reduce the filesystem to a size of 20GB

resize2fs -p /dev/MYLVMVOLUME1/MYDATA 20G

Then shrink the LV to a size of 21GB (for paranoia a bit larger)

lvreduce -L21G /dev/MYLVMVOLUME1/MYDATA

If you get this error

Volume group mapper doesn't exist

you probably tried to use the /dev/mapper/... device instead of the normal device.

Now you can extend the filesystem up to the maximum size which is possible with your LV, removing the gap we introduced for paranoia.

resize2fs -p /dev/MYLVMVOLUME1/MYDATA

Forgot to issue vgreduce

When you moved all the data from a PV with pvmove away, but forgot to use vgreduce to remove it from the VG, the next LVM start will fail with this error

Couldn't find device with uuid ...
Couldn't find all physical volumes for volume group ...

If you are sure that there is no more data on the PV, you can skip and remove it

vgchange -ay --partial
vgchange --removemissing

Move VG to another system

unmount everything Disable VG

# vgchange -an MYLVMVOLUME1


# vgexport MYLVMVOLUME1

On the other system

# vgimport MYLVMVOLUME1
# vgchange -ay MYLVMVOLUME1

mount again

Partition table has been deleted

LVM fails to start with:

Couldn't find device with uuid ...
Couldn't find device with uuid ...
PV unknown device VG foo lvm2 [186,26 GiB / 158,33 GiB free]
PV unknown device VG foo lvm2 [372,13 GiB / 0 free]
PV /dev/sda6 VG foo lvm2 [212,16 GiB / 212,16 GiB free]
Total: 6 [1,57 TiB] / in use: 6 [1,57 TiB] / in no VG: 0 [0 ]

In this case the partition table of /dev/sdb had be deleted.

Try to guess the old partition table (while take some time ...)

# gpart /dev/sdb
Begin scan...
Possible partition(Linux swap), size(486mb), offset(0mb)

If it succeeds:

# gpart -W /dev/sdb /dev/sdb

Hard-disk Encryption


First Time Usage

Install the software, on Debian

# apt-get install cryptsetup

Create a test file to play with and mount it via loopback

# dd if=/dev/urandom of=testfile bs=1M count=100
# losetup /dev/loop/0 testfile

Now encrypt it with LUKS

# cryptsetup luksFormat /dev/loop/0
This will overwrite data on /dev/loop/0 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: GEHEIM
Verify passphrase: GEHEIM
Command successful.

# cryptsetup luksOpen /dev/loop/0 mytestfs
Enter LUKS passphrase: GEHEIM
key slot 0 unlocked.
Command successful.
# mkfs.ext3 /dev/mapper/mytestfs
# mount /dev/mapper/mytestfs /mnt/mnt/
# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

You may even use a file with a secret to avoid the need to enter the password on each mount.

# pwgen -s 500 > /mnt/usbstick/mykeyfile
# cryptsetup luksFormat /dev/loop/0 /mnt/usbstick/mykeyfile

This will overwrite data on /dev/loop/0 irrevocably.

Are you sure? (Type uppercase yes): YES
Command successful.

# cryptsetup -d /mnt/usbstick/mykeyfile luksOpen /dev/loop/0 mytestfs
# mkfs.ext3 /dev/mapper/mytestfs
# mount /dev/mapper/mytestfs /mnt/mnt/
# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

Even when you started with a keyfile you can always add a manual password

# cryptsetup -d /mnt/usbstick/mykeyfile luksAddKey /dev/loop0
key slot 0 unlocked.
Enter new passphrase for key slot: FOOBAR
Verify passphrase: FOOBAR
Command successful.

You can always add additional passwords (e.g. for different users)

# cryptsetup luksAddKey /dev/loop0
Enter any LUKS passphrase: FOOBAR
key slot 1 unlocked.
Enter new passphrase for key slot: SECRET
Verify passphrase: SECRET
Command successful.

Once you configured everything you have to decrypt the data first. Either with the keyfile

# cryptsetup -d /mnt/usbstick/mykeyfile luksOpen /dev/loop/0 mytestfs
key slot 0 unlocked.
Command successful.

or with the password

# cryptsetup luksOpen /dev/loop/0 mytestfs
Enter LUKS passphrase: SECRET
key slot 1 unlocked.
Command successful.

Now everyone can mount and access it

# mount /dev/mapper/mytestfs /mnt/mnt/

until you unmount and close it again

# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

Normal Usage with Keyfile

You probably want to use real devices, e.g one for swap and one for the crypt partition swap: /dev/hdz6 crypt: /dev/hdz7 Encrypt the crypt partition (fill it with random data first is probably better)

cryptsetup luksFormat /dev/hdz7 /mnt/usbstick/mykeyfile

Optionally add an encrypted swap partition, will be formated on each reboot automatically /etc/crypttab

# <target device> <source device> <key file> <options>
myswap /dev/hdz6 /dev/random swap
mycrypt /dev/hdz7 /mnt/usbstick/mykeyfile luks

Opened device should appear after reboot and has to be formated (once).

mkfs.ext3 /dev/mapper/mycrypt

Now it can be mounted via normal fstab entry /etc/fstab

# <file system> <mount point> <type> <options> <dump> <pass>
/dev/mapper/mycrypt /mnt/mycrypt1 ext3 defaults 0 2
/dev/mapper/myswap none swap sw 0 0

Don't loose the keyfile ;-)


Recently I moved most of the services that were installed on my server into Linux VServers. Now all of these services run in their own Linux installation. This seems to have only advantages for me:

  • Each VServer Linux installation has only the minimum amount of packages installed to offer their service. I use

debfoster and deborphan to preserve the current status.

  • The access to the ports of each VServer is restricted by a packet filter on the host.
  • If a a security hole in one of the services allows to break into the system the intruder only has access to the affected VServer and not to all the other VServers with their services. Additionally the rights of each VServer can be restricted CAPABILITIES).
  • It's no longer necessary to install many (any) public reachable services on the host-system so it's much harder to attack it.
  • The host-system can observe the VServers, for example with Integrit
  • The VServers run without having their own kernel. If the Verserver idls they hardly consume any resources. Additionally thanks to vhashify files that exist on several Verserver only use space on the hard disk once.
  • It's very easy to create new installations (e.g. for testing) and you can even give root access to friends without the risk that they accidently interfere important things.

VServer Installation

Get the VServer kernel-patch, apply it and boot the patched kernel. Install the VServer utils (I used the Debian VServer utils)

Now this is all you need to do to get a new VServer:

# find free context id
find /etc/vservers/ -name context -exec cat '{}' ';' | sort -n
# assume 1234 is still unused
vserver foo build -m debootstrap --hostname --netdev eth0 --interface --context 1234 -- -d jessie

This will create a VServer called "foo" with a Debian jessie Installation, the VServer believes be named and will have the IP-address

Create a Gentoo VServer:

vserver-new mygentoo --hostname mygentoo --interface eth0: --context 1253 stage3 ./stage3-i686-20060317.tar.bz2 x86

The stage file can be found on each Gentoo mirror, the vserver-new script can be found in the Gentooo vserver package. See also the Gentoo Linux-VServer howto.

In order to make it possible to use vhashify (yes, the folder is called vunify not vhashify):

mkdir -p /etc/vservers/foo/apps/vunify

Exclude the folder from the vhashify (for all vservers):


VServer File Permissions

If you can't delete a file:

# whoami
# ls -ld foo/bar.txt
# ls -ld foo/
# lsattr foo/bar.txt
# lsattr -d foo/
# showattr foo/bar.txt
# showattr foo/

(in the showattr output only upper case letters are really active)

# su
# chmod u+w foo/
# chmod u+w foo/bar.txt
# chattr -i foo/bar.txt
# setattr --~iunlink-but-not-immutable foo/bar.txt
# setattr --~iunlink-but-not-immutable foo

VServer Configuration

  • In order to allow the VServer to raise the priority of a process (cron-jobs which start su with a nice level will need this):
  • Automatically start a VServer while booting:

If you have lots of VServers or if some of your vservers have to be started in a given order, you can use other names instead of "default". Don't forger to copy and modify the "default" init.d vserver script to also start and stop (at the desired time) the other VServers.

VServer Masquerading SNAT

My VServers all have private IP-addresses. The following iptables instruction (issued on the host-system) will rewrite all packages from to vserver to look like they came the public IP of the host. (fill/replace the variables with your values):

iptables -t nat -I POSTROUTING -s $VSERVER_NET ! -d $VSERVER_NET -j SNAT --to $EXT_IP


If there are services (like e.g. ssh) which should run on the host and on the VServers you need to prevent the host system to acquire all request for your service. Therefore bind the service on the host only to some (one) ip address and not to all. Most services that are started as deamon can be configured this way, services started via inetd must be switched to xinetd. In xinetd it's quite easy:

bind =

Linux Bonding

One Linux computer with at least 2 network cards. Connect your network cards with one or more normal switches.

Advantage: Higher network performance and if one of the network cables fails the connection is not interrupted.

apt-get install net-tools ethtool ifenslave-2.6 bmon

allow eth0
iface eth0 inet dhcp

allow eth1
iface eth1 inet dhcp

auto bond0
iface bond0 inet dhcp
bond-mode 6
bond-miimon 1000
bond-slaves eth0 eth1

Now you have one virtual network device called bond0 which distributes the data on both real cards eth0 and eth1. If one looses connection this will be detected and the data will be transferred by the remaining cards

bonding: bond0: link status definitely down for interface eth0, disabling it
bonding: bond0: making interface eth1 the new active one.


apt-proxy is a caching proxy for use with Debian systems to ensure that installation-packages for all Debian VServers only have to be downloaded once.


It's easier as you might have thought to set up your own Bittorrent tracker. After all it's very handy to be able to distribute huge files to lots of recipients in a little while.

  • At first you need a running tracker (if you don't have a working one to use)
bttrack --port 6689 --dfile my_dstate
  • Now create a torrent
btmakemetafile FILE

Where FILE is the file you want to distribute and ist the host/IP-address where your tracker can be reached. If your public IP is a dynamic one you might consider to use a dynamic DNS service. Now you habe a file called e.g. FILE.torrent. Everyone who wants to participate in your file distribution needs this file. With this file they could start to download, however nobody has the file yet. Therefore you should start a client yourself and point it to the original file.

It seemed that it was necessary to enable the promiscuous-mode for my network interface to make a Bittorrent-tracker on one of the VServers possible. I couldn't find out yet why.


I use the Bind name-server to resolve the IP-address of all VServers and real local machines. Additionally it forwards all request to external name-servers. In order to make bind work inside a VServer you can (beside others) set this opiton:



Very handy is a service like e.g. which helps you to make your machine (with only a dynamic IP) reachable via a constant hostname.

#!/usr/sbin/ez-ipupdate -c

### The following lines were generated by debconf


### Changes below this line will be preserved on upgrades.


FTP Server

My current FTP server is ProFTPD. A wonderful addendum to apt-proxy is to . Then the apt-proxy only has to deal with security updates.

If you would like to offer a public FTP-server check out the following configuration. It enables public readable files and file upload. Uploaded files can't be downloaded again to prevent unauthorised file-sharing.

DefaultTransferMode binary

ServerName "My FTP server"
#ServerType inetd
ServerType standalone

Port 21

Umask 022 022

# This ONLY works in standalone mode
MaxInstances 10

# Set the user and group that the server normally runs at.
User nobody
Group nogroup

# Forbit normal user logins
<Limit LOGIN>

# A basic anonymous configuration, (extra upload directories)
<Anonymous ~ftp>
User ftp
Group nogroup
UserAlias anonymous ftp
RequireValidShell off
# MaxClients 10

# Allow anonymous logins
<Limit LOGIN>

# Limit WRITE everywhere in the anonymous chroot
<Directory *>
<Limit WRITE>

# Allow write to incoming dir
<Directory ~ftp/incoming>
<Limit STOR>

# Allow /pub_world from everywhere
<Directory ~ftp/pub_world>
<Limit WRITE>
Order deny,allow



To be able to read and process all your mails from different places worldwide a IMAP server will be very usefull. In order to enable IMAP-SSL either create a self-signed certificate or check my howto.


Leafnode is my local NNTP / Usenet server.


CUPS printer daemon.



I configured Exim as my mailserver like I described in my howto about .


Apache webserver.

Transparent Proxy

With Squid and iptables I've configured a transparent proxy so each file that is downloaded from the local network only has to be downloaded once and then is automatically delivered from the proxy.