My Linux Server

Server

Constant Names for Network Interfaces

# cat /etc/udev/rules.d/local-interfaces.rules
KERNEL="eth*", SYSFS{address}="00:48:54:55:00:00", NAME="heimnetz"

Now replace eth0 with heimnetz everywhere:

find /etc/ -type f -exec grep -He 'eth0' '{}' ';'

Now your network interface has a constant name.

Traffic Prioritisation

Real-Time applications like e.g. VOIP will benefit a lot from this:
Script to set priorities on traffic.

Quota

/etc/fstab: /dev/foo /mnt/bar ext3  defaults,usrquota  0 2
mount /dev/foo /mnt/bar -o remount

quotacheck -acguvm
quotacheck -cguvm /mnt/bar

edquota -u USERNAME
edquota -u USERNAME -f /dev/foo
edquota -p USERNAME -u USERNAME2 -f /dev/foo

edquota -t

quotaon  -avug
quotaoff -avug

repquota -a

su USERNAME -c 'quota  -v'

Software Raid

Create partitions with the type fd, e.g. /dev/sda1 und /dev/hda1

mdadm --create /dev/md0 --verbose --level=raid1             --raid-devices=2 /dev/hda1 /dev/sda1
mdadm --create /dev/md1 --verbose --level=raid5             --raid-devices=3 /dev/hda5 /dev/sda5 /dev/sdb5
mdadm --create /dev/md1 --verbose --level=raid5 --chunk=128 --raid-devices=3 /dev/hda5 /dev/sda5 /dev/sdb5

If there are not all hard disks available yet which you want to add to the raid, you might use the word missing instead of the partitions on that disk.

mdadm --create /dev/md0 -vv --level=raid1 --raid-devices=2 /dev/sda1 missing

If you get this error

mdadm: error opening /dev/md0: No such device or address

and the device is really there, check if you loaded the module md.

Do not forget to add the output of

mdadm --detail --scan

to the file /etc/mdadm/mdadm.conf.

If you loose one of your hard disks, this is how you can see which partitions of your raid are still there

cat /proc/mdstat
mdadm --detail /dev/md0

You can check if a partitions was previously part of a raid

mdadm -Q /dev/hda1 /dev/hda1: device 0 in 2 device mismatch raid1 /dev/md0. Use mdadm --examine for more detail.

If you have a new hard disk you can add partitions to a raid like this

mdadm --manage --add /dev/md0 /dev/hda1

One of these errors

mdadm: add new device failed for /dev/hda1 as 2: No space left on device
mdadm: add new device failed for /dev/hda1 as 2: Invalid argument

probably indicates that your partition is too small for the raid. Check yourself

cat /proc/partitions

The size of the raid can also be changed (number of used partitions / utilisation of used partitions)

mdadm /dev/md1 --grow --raid-devices=4
mdadm /dev/md1 --grow --size=max

After I temporally lost a disk of a raid5, I had troubles restarting the raid. This is what I did to recover from this situation (written down from memory)

Boot from CD (e.g. Knoppix), if necessary.
/etc/init.d/mdadm start

mdadm --detail --scan >> /etc/mdadm/mdadm.conf
/etc/init.d/mdadm-raid restart

Check where the problem is
```
dmesg
mdadm --detail /dev/mdX
```
Although I only lost one drive, the raid refused to start again
```
raid5: cannot start dirty degraded array for md1
```
I got it back with assemble and the force option
```
mdadm --assemble --scan --run --force
```

Watch the progress of the recovery like this

watch -n5 cat /proc/mdstat

You can speed up the minimum speed progress, e.g. increase it to 50MB/s

echo 50000 >/proc/sys/dev/raid/speed_limit_min

You can remove partitions from a raid like this

mdadm --fail   /dev/md0 /dev/hda1
mdadm --remove /dev/md0 /dev/hda1

Stop a running raid

mdadm --stop /dev/md1

Remove raid markers from a partition

mdadm --zero /dev/hda5

If you like to boot from a raid1, it is important to write the bootmanager to both harddrives. Just act as if the both partitions of your raid are regular partitions and no raid1 devices.
First add both hard disks to the file

/boot/grub/device.map

Call one hd0 and one hd1. Then start grub, specify the device where /boot is installed (,0 is the first partition) and then write the bootsector of both hard disks

root (hd0,0) setup (hd0)
root (hd1,0) setup (hd1)

Now try to boot with only one of the hard disks attached. Do not forget to repair the raid in between.

LVM

LVM Howto

Overview of the elements of an LVM system

PV PV PV   (e.g., /dev/hda, /dev/hdb5, /dev/sdb3)
 \ | /
  \|/
  VG       (e.g., MYLVMVOLUME1)
  /|\
 / | \
LV LV LV   (e.g., /mnt/mydata, /mnt/moredate)

Prepare PV (discs or partitions)

# pvcreate /dev/sdb1
# pvdisplay -m /dev/sdb1

  --- Physical volume ---
  PV Name               /dev/sdb1
  ...
  Allocated PE          54206

Create new VG with the prepared PV

# vgcreate      MYLVMVOLUME1 /dev/hda /dev/hdb5
# vgchange -a y MYLVMVOLUME1

Remove VG

# vgchange -a n MYLVMVOLUME1
# vgremove      MYLVMVOLUME1

Add new PV to VG

# vgextend my_volume_group /dev/sdb3

Remove PV from VG

(e.g., when want to remove an old disc)
Is there enough space even without the PV?

# pvscan
# pvdisplay /dev/hda

If not, first add a new disc like described above:

Prepare discs / partitions
Add new PV to VG

Then do a backup!
Note that the following will require that you enabled the kernel option

CONFIG_DM_MIRROR

Move everything from the old discs to the other PV

# pvmove /dev/hda

Or specify to which PV the data should be moved

# pvmove /dev/hda /dev/hde3

Now remove the PV

# vgreduce MYLVMVOLUME1 /dev/sdb3
# pvremove /dev/sdb3

pvmove

The command pvmove obviously requires, that you have enough space left on your VG which has not yet been occupied by a LV. But even when you have enough free space in total, pvmove will need your help if the amount of space is not available on a single PV.
Even you try to move for example this sdb1

# pvcreate /dev/sdb1
# pvdisplay -m /dev/sdb1

  --- Physical volume ---
  PV Name               /dev/sdb1
  ...
  Allocated PE          54206

but there is no other PV which can carry it, this will fail

# pvmove /dev/sdb1
Insufficient suitable contiguous allocatable extents for logical volume pvmove0: 43955 more required

Just help it with some manual intervention. Use pvdisplay to find the PV with the largest value for Free PE for your LV. In this example it is sda6

# pvdisplay

  --- Physical volume ---
  PV Name               /dev/sda6
  ...
  Free PE               33267
  Allocated PE          0

Now you can ask pvmove to move 33267 PE on this PV

# pvmove /dev/sdb1:0-33267 /dev/sda6

Check if the destination is now full, if not, just repeat the last command with an increased upper limit. When it is finally full just move the next area

# pvmove /dev/sdb1:33268-40000 /dev/hda6

to the next PV with some free space. If you specify an area where parts have already been moved, this parts will be automatically ignored. So this one

# pvmove /dev/sdb1:0-40000 /dev/hda6

would be equivalent in our case, as we moved the part at the beginning already.
However, this method might leads to LV which are fragmented. A better way is, to initiate a backup of the LVM structure

vgcfgbackup

change into this directory

/etc/lvm/backup

and read the information about your LVM. Let's assume you have the following output

segment2 {
 start_extent = 1
 extent_count = 250      # 1000 Megabytes

 type = "striped"
 stripe_count = 1        # linear

 stripes = [
            "pv2", 25346
           ]
 }

The source device and the start point can be found in the stripes section

pv2
25346

This shows how long it is

extent_count = 250

Now we only have to find the mapping for pv2 to a real device in the file. In this example we assume it is sda6. Now the segment can be moved completely to another device

pvmove -v /dev/sda6:25346-`echo 25346 - 1 + 250 | bc` /dev/sdb6

If you move the segments of a LVM consecutively to the same device they will merge automatically.

Create LV on a VG

How much space is left?

# vgdisplay MYLVMVOLUME1

Create LV with 1.5GB size, linear

# lvcreate         -L1500 -n MYDATA   MYLVMVOLUME1

Create LV with 1.5GB size, stripped

# lvcreate -i2 -I4 -L500  -n MOREDATA MYLVMVOLUME1

Create LV with 1.5GB size, linear and use only /dev/hdb5

# lvcreate         -L1500 -n FASTDATA MYLVMVOLUME1 /dev/hdb5

Now create a filesystem on it

# mkfs.ext3 /dev/MYLVMVOLUME1/FILESYSTEM1
-L NAME
-O dir_index
-T news
-T largefile
-m 0
-N 250000
tune2fs -c 31

Remove LV

# lvremove /dev/MYLVMVOLUME1/FASTDATA

Extend LV

How much space is left?

# vgdisplay MYLVMVOLUME1

Extend +1 GB (negativ numbers are discussed below, dangerous!)

# lvextend -L+1G /dev/MYLVMVOLUME1/MYDATA

Now extend the filesystem (ext2 / ext3)

# umount    /dev/MYLVMVOLUME1/MYDATA
# resize2fs -p /dev/MYLVMVOLUME1/MYDATA
# mount     /dev/MYLVMVOLUME1/MYDATA

Extend the filesystem (reiserfs, unmount may be optional)

# umount           /dev/MYLVMVOLUME1/MYDATA
# resize_reiserfs  /dev/MYLVMVOLUME1/MYDATA
# mount -treiserfs /dev/MYLVMVOLUME1/MYDATA

Shrink LV

First reduce the filesystems size, then the LV!
This will reduce the filesystem to a size of 20GB

resize2fs -p /dev/MYLVMVOLUME1/MYDATA 20G

Then shrink the LV to a size of 21GB (for paranoia a bit larger)

lvreduce -L21G /dev/MYLVMVOLUME1/MYDATA

If you get this error

Volume group mapper doesn't exist

you probably tried to use the /dev/mapper/... device instead of the normal device.

Now you can extend the filesystem up to the maximum size which is possible with your LV, removing the gap we introduced for paranoia.

resize2fs -p /dev/MYLVMVOLUME1/MYDATA

Forgot to issue vgreduce

When you moved all the data from a PV with pvmove away, but forgot to use vgreduce to remove it from the VG, the next LVM start will fail with this error

Couldn't find device with uuid ...
Couldn't find all physical volumes for volume group ...

If you are sure that there is no more data on the PV, you can skip and remove it

vgchange -ay --partial
vgchange --removemissing

Move VG to another system

unmount everything
Disable VG

# vgchange -an MYLVMVOLUME1

Export

# vgexport     MYLVMVOLUME1

On the other system

# vgimport     MYLVMVOLUME1
# vgchange -ay MYLVMVOLUME1

mount again

Hard-disk Encryption

Links

LUKS

First Time Usage

Install the software, on Debian

# apt-get install cryptsetup

Create a test file to play with and mount it via loopback

# dd if=/dev/urandom of=testfile bs=1M count=100
# losetup /dev/loop/0 testfile

Now encrypt it with LUKS

# cryptsetup luksFormat /dev/loop/0
WARNING!
========
This will overwrite data on /dev/loop/0 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase: GEHEIM
Verify passphrase: GEHEIM
Command successful.

# cryptsetup luksOpen /dev/loop/0 mytestfs
Enter LUKS passphrase: GEHEIM
key slot 0 unlocked.
Command successful.
# mkfs.ext3 /dev/mapper/mytestfs
# mount /dev/mapper/mytestfs /mnt/mnt/
# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

You may even use a file with a secret to avoid the need to enter the password on each mount.

# pwgen -s 500 > /mnt/usbstick/mykeyfile
# cryptsetup luksFormat /dev/loop/0 /mnt/usbstick/mykeyfile

WARNING!
========
This will overwrite data on /dev/loop/0 irrevocably.

Are you sure? (Type uppercase yes): YES
Command successful.

# cryptsetup -d /mnt/usbstick/mykeyfile luksOpen /dev/loop/0 mytestfs
# mkfs.ext3 /dev/mapper/mytestfs
# mount /dev/mapper/mytestfs /mnt/mnt/
# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

Even when you started with a keyfile you can always add a manual password

# cryptsetup -d /mnt/usbstick/mykeyfile luksAddKey /dev/loop0
key slot 0 unlocked.
Enter new passphrase for key slot: FOOBAR
Verify passphrase: FOOBAR
Command successful.

You can always add additional passwords (e.g. for different users)

# cryptsetup  luksAddKey /dev/loop0
Enter any LUKS passphrase: FOOBAR
key slot 1 unlocked.
Enter new passphrase for key slot: SECRET
Verify passphrase: SECRET
Command successful.

Once you configured everything you have to decrypt the data first. Either with the keyfile

# cryptsetup -d /mnt/usbstick/mykeyfile luksOpen /dev/loop/0 mytestfs
key slot 0 unlocked.
Command successful.

or with the password

# cryptsetup luksOpen /dev/loop/0 mytestfs
Enter LUKS passphrase: SECRET
key slot 1 unlocked.
Command successful.

Now everyone can mount and access it

# mount /dev/mapper/mytestfs /mnt/mnt/

until you unmount and close it again

# umount /dev/mapper/mytestfs
# cryptsetup luksClose /dev/mapper/mytestfs

Normal Usage with Keyfile

You probably want to use real devices, e.g one for swap and one for the crypt partition
swap: /dev/hdz6
crypt: /dev/hdz7
Encrypt the crypt partition (fill it with random data first is probably better)

cryptsetup luksFormat /dev/hdz7 /mnt/usbstick/mykeyfile

Optionally add an encrypted swap partition, will be formated on each reboot automatically
/etc/crypttab

# <target device>  <source device>  <key file>               <options>
myswap             /dev/hdz6        /dev/random              swap
mycrypt            /dev/hdz7        /mnt/usbstick/mykeyfile  luks

Opened device should appear after reboot and has to be formated (once).

mkfs.ext3 /dev/mapper/mycrypt

Now it can be mounted via normal fstab entry
/etc/fstab

# <file system>      <mount point>  <type>  <options>   <dump> <pass>
/dev/mapper/mycrypt  /mnt/mycrypt1  ext3    defaults    0      2
/dev/mapper/myswap   none           swap    sw          0      0

Don't loose the keyfile ;-)

25-01-2010 23.56

VServer

Recently I moved most of the services that were installed on my server into Linux VServers. Now all of these services run in their own Linux installation. This seems to have only advantages for me:

Each VServer Linux installation has only the minimum amount of packages installed to offer their service. I use
debfoster and deborphan to preserve the current status.
The access to the ports of each VServer is restricted by a packet filter on the host.
If a a security hole in one of the services allows to break into the system the intruder only has access to the affected VServer and not to all the other VServers with their services. Additionally the rights of each VServer can be restricted CAPABILITIES).
It's no longer necessary to install many (any) public reachable services on the host-system so it's much harder to attack it.
The host-system can observe the VServers, for example with Integrit
The VServers run without having their own kernel. If the Verserver idls they hardly consume any resources. Additionally thanks to vhashify files that exist on several Verserver only use space on the hard disk once.
It's very easy to create new installations (e.g. for testing) and you can even give root access to friends without the risk that they accidently interfere important things.

VServer Installation

Get the VServer kernel-patch, apply it and boot the patched kernel. Install the VServer utils (I used the Debian VServer utils)

Now this is all you need to do to get a new VServer:

# find free context id
find /etc/vservers/ -name context -exec cat '{}' ';' | sort -n
# assume 1234 is still unused
vserver foo build -m debootstrap --hostname foo.example.com --netdev eth0 --interface 10.0.1.5/32 --context 1234 -- -d etch

This will create a VServer called "foo" with a Debian Sarge Installation, the VServer believes be named foo.example.com and will have the IP-address 10.0.1.5.

Create a Gentoo VServer:

vserver-new mygentoo --hostname mygentoo --interface eth0:10.0.1.6 --context 1253  stage3 ./stage3-i686-20060317.tar.bz2 x86

The stage file can be found on each Gentoo mirror, the vserver-new script can be found in the Gentooo vserver package. See also the Gentoo Linux-VServer howto.

In order to make it possible to use vhashify (yes, the folder is called vunify not vhashify):

mkdir -p /etc/vservers/foo/apps/vunify

Exclude the folder from the vhashify (for all vservers):

/usr/lib/util-vserver/defaults/vunify-exclude

VServer File Permissions

If you can't delete a file:

# whoami
# ls    -ld foo/bar.txt
# ls    -ld foo/
# lsattr    foo/bar.txt
# lsattr -d foo/
# showattr  foo/bar.txt
# showattr  foo/

(in the showattr output only upper case letters are really active)

# su
# chmod u+w foo/
# chmod u+w foo/bar.txt
# chattr -i foo/bar.txt
# setattr --~iunlink-but-not-immutable foo/bar.txt
# setattr --~iunlink-but-not-immutable foo

VServer Configuration

In order to allow the VServer to raise the priority of a process (cron-jobs which start su with a nice level will need this):
```
/etc/vservers/foo/flags
igneg_nice
```
Automatically start a VServer while booting:
```
/etc/vservers/foo/apps/init/mark
default
```
If you have lots of VServers or if some of your vservers have to be started in a given order, you can use other names instead of "default". Don't forger to copy and modify the "default" init.d vserver script to also start and stop (at the desired time) the other VServers.

VServer Masquerading SNAT

My VServers all have private IP-addresses. The following iptables instruction (issued on the host-system) will rewrite all packages from to vserver to look like they came the public IP of the host. (fill/replace the variables with your values):

iptables -t nat -I POSTROUTING -s $VSERVER_NET  ! -d $VSERVER_NET -j SNAT --to $EXT_IP

xinetd

If there are services (like e.g. ssh) which should run on the host and on the VServers you need to prevent the host system to acquire all request for your service. Therefore bind the service on the host only to some (one) ip address and not to all. Most services that are started as deamon can be configured this way, services started via inetd must be switched to xinetd. In xinetd it's quite easy:

bind = 10.0.0.1

14-07-2008 19.56

apt-proxy

apt-proxy is a caching proxy for use with Debian systems to ensure that installation-packages for all Debian VServers only have to be downloaded once.

15-01-2006 02.53

Bittorrent

It's easier as you might have thought to set up your own Bittorrent tracker. After all it's very handy to be able to distribute huge files to lots of recipients in a little while.

At first you need a running tracker (if you don't have a working one to use)
```
bttrack --port 6689 --dfile my_dstate
```
Now create a torrent
```
btmakemetafile http://example.com:6969/announce FILE
```
Where FILE is the file you want to distribute and example.com ist the host/IP-address where your tracker can be reached.
If your public IP is a dynamic one you might consider to use a dynamic DNS service.
Now you habe a file called e.g. FILE.torrent. Everyone who wants to participate in your file distribution needs this file. With this file they could start to download, however nobody has the file yet. Therefore you should start a client yourself and point it to the original file.

It seemed that it was necessary to enable the promiscuous-mode for my network interface to make a Bittorrent-tracker on one of the VServers possible. I couldn't find out yet why.

15-01-2006 03.14

Nameserver

I use the Bind name-server to resolve the IP-address of all VServers and real local machines. Additionally it forwards all request to external name-servers.
In order to make bind work inside a VServer you can (beside others) set this opiton:

/etc/vservers/myvserver/bcapabilities
CAP_SYS_RESOURCE

DynDNS

Very handy is a service like e.g. yi.org which helps you to make your machine (with only a dynamic IP) reachable via a constant hostname.


#!/usr/sbin/ez-ipupdate -c

###  The following lines were generated by debconf

service-type=gnudip
server=gnudip2.yi.org
user=MYUSERNAME:MYPASSWORD
host=.MYHOST.yi.org
interface=ppp0
run-as-user=ez-ipupd
cache-file=/var/cache/ez-ipupdate/default-cache

###  Changes below this line will be preserved on upgrades.

retrys=25

15-01-2006 18.05

FTP Server

My current FTP server is ProFTPD.
A wonderful addendum to apt-proxy is to fill a FTP server with the content of the Debian installation-CDs. Then the apt-proxy only has to deal with security updates.

If you would like to offer a public FTP-server check out the following configuration. It enables public readable files and file upload. Uploaded files can't be downloaded again to prevent unauthorised file-sharing.

DefaultTransferMode             binary

ServerName                      "My FTP server"
#ServerType                     inetd
ServerType                      standalone

Port                            21

Umask                           022  022

# This ONLY works in standalone mode
MaxInstances                    10

# Set the user and group that the server normally runs at.
User                            nobody
Group                           nogroup

# Forbit normal user logins
<Limit LOGIN>
  DenyAll
</Limit>

# A basic anonymous configuration, (extra upload directories)
<Anonymous ~ftp>
  User                          ftp
  Group                         nogroup
  UserAlias                     anonymous ftp
  RequireValidShell             off
#  MaxClients  10

  # Allow anonymous logins
  <Limit LOGIN>
   AllowAll
  </Limit>

  # Limit WRITE everywhere in the anonymous chroot
  <Directory *>
   <Limit WRITE>
     DenyAll
   </Limit>
  </Directory>

  # Allow write to incoming dir
  <Directory ~ftp/incoming>
    <Limit READ WRITE>
      DenyAll
    </Limit>
    <Limit STOR>
      AllowAll
    </Limit>
   </Directory>

  # Allow /pub_world from everywhere
  <Directory ~ftp/pub_world>
   <Limit WRITE>
    DenyAll
   </Limit>   
   <Limit READ DIRS>
    Order  deny,allow
    AllowAll
   </Limit>
  </Directory>

</Anonymous>

15-01-2006 19.22

Deutsch English
Thorsten Gunkel's Homepage --> IT --> Software --> Software Documentation
Sitemap / Search My Linux Server Interactive Programs Networks / Communication Debian Commandos Gentoo Commandos Tiny Linux / Unix Commandos Windows / Not Unix Revision / Version Control Makefiles Design Patterns C / C++ Java JavaScript LaTeX SQL / Datenbanken TinyCA Use Exim With Multiple Smarthosts Mutt and Slrn HOWTO Fill a FTP Server With the Debian Installation CDs