Ubuntu adding/archiving on to new hard drive

So you just got a new bigger hard drive and want to use it to replace the main storage device on your Ubuntu Linux server, or just add the extra storage to it.

First powerdown (may not be necesariy with SATA) connect new drive and power up.
loggin and get to the terminal.

Find out the new device name, likely to be some thing along the lines of /dev/sdb
$ sudo fdisk -l
----
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sda1 1 121601 976760032 83 Linux

Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xe53eaeaf

Device Boot Start End Blocks Id System
/dev/sdb

Not Formatted
----

It is the unformatted drive that we had to look out for.
Next Creat a partition table, It is a storage device so only want the one partition.

$ sudo fdisk /dev/sdb
press n to create new partition choose primary, the defaults should be good for use the full disk.
press w to save changes to disk.

This has created partition /dev/sdb1, which requires formatting for ext3 do:
$ sudo mke2fs -j /dev/sdb1

Creat a point to mount it, I only want this to be temprorayt but if it is for permenent installation choose the names carfully.
$ sudo mkdir -p /mnt/insertHostName/tempdrive

Mount it (One off)
$ sudo mount -t ext3 /dev/sdb1 /mnt/insertHostName/tempdrive/

To mount the drive on sytem restarts do:
$ sudo vim /etc/fstab
added this line
/dev/sda1 /mnt/hostname/partitionname ext3 rw,auto,async,errors=remount-ro 0 1
then reload the fstab (mount the new drive)
$ sudo mount -a

Now start the Archiving process (Copying the files from old drive on to new).
The -a flag is very good for this it will Recursively copy maintaining permissions, Access dates and recreate links.
$ sudo cp -a /mnt/insertHostName/oldDrive/* /mnt/insertHostName/tempdrive/

once the copy is complete you can compare the space used available using:
$ df -h
>Filesystem Size Used Avail Use% Mounted on
>/dev/sda1 925G 866G 12G 99% /mnt/insertHostName/oldDrive
>/dev/sdb1 1.4T 866G 451G 66% /mnt/insertHostName/tempdrive

Low Power Server Recommendations

Every one needs a low power server, my current recommendations are:

Option [1]
If you only require a Network drive this is a really good (cheap/simple) option (it does include some basic server functionality)
Icy Box Dual Disk with Gigabit NAS £100 http://www.scan.co.uk/ WikiSupport

Option [2]
However if you require custom services (Squeeze box, etc) then building your own is a good option:
Some thing based on an atom in a mini-itx form factor like: Jetway JNC92 £105
As much DDR2 memory as allowed (mini-itx often limits to 2GB)
Note: DDR2 is much more power efficient than the original DDR standard.

2GB DDR2 533MHz for the Jetway JNC92 is about £25

A cheap case like this from ebuyer would look reasonable for £25

I also installed Ubuntu on to an 8Gb USB flash drive for about £20. For wear and tear next time I would just use a laptop drive (£35 for a 120 GB or £50 for 320Gb), which could host the always on server parts like websites, torrents, ftp etc. and let the main storage drives spin down. If you are planning on this with only 2 sata sockets available you may want to put the laptop boot drive on a usb interface (£8 ebuyer, cant find summvision device)

Note:
I spent about £100 on a Jetway C7 1.2GHz J7F2WE1G2E not realising it could not take the Jetway expansion cards. As they make a very nice 3Socket 1GBit LAN Card. Turning your machine into a low power Gigabit switch as well (if you configure the OS to bridge the interfaces).

Option [3]
A nice alternative to putting together your own itx machine is the BBS2 for around £380 with the 2G RAM upgrade and VAT. Although 1 Drive must be the Operating System and since it uses its own mounting system low power 2.5" laptop drive can not be used for this task. It can then be set up with hardware raid on the remaining 4 channels, but Software raid is pretty good on Linux and Windows Home Server (Apparently)

The only really advantage of this over building your own system is the 5 Drive bays and Shiny case, which comes at a real price. Not forgetting if you built your own itx system you can easily add external usb drive to it, or get it to mount and share additional NAS drives like [1] or put in a large case and get some thing like this 12 port sata controller £600 or a 4 port sata controller £75

Note: The case recommended in [2] only has room for 2 hard drives using Raid1 or adding extra capacity the bootable drive if separate from the storage would have to be external.

Again the drive could be external in [3] if you wanted to use a laptop drive in 3.

Follow Up Post
Why put a Laptop drive in a desktop?

Samba and Finder (OS X)

Recently I have had problems with finder not being able to copy and paste files from my network shares. I could still use quick look, preview etc.

The problem seemed to be specifically with finder although this article seemed to suggest it was todo with samba file locking Maybe finder got a bug in the last mac update and this is just a work around.

Any way adding this option seems to help finder copying files over samba (option goes in the smb.conf file on the server)
[global]
#Stop Mac Finder thinking it can not copy files
locking = no

Ubuntu Media Codecs

To install codecs to play back mpeg 4 video files I followed the instructions here http://linuxowns.wordpress.com/2008/02/11/media-and-ubuntu/ for Ubuntu 8.04

$sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list
$wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update

#DVD support
$sudo apt-get install libdvdcss2

#Some mpeg 4 codecs
$sudo apt-get install w32codecs

#Other codecs
$sudo apt-get install non-free-codecs

Remote X Forwarding

Forwarding the GUI (Graphical User Interface) from an application on a remote (not locallhost) server is known as X Forwarding. As X11 is the unix/linux display stuff.

a typical noddy example is the xclock which can be lanched from any terminal mac users might have to use the X11 terminal instead of terminal.app.

so to make sure that my (remote) server was setup to forward X over ssh check that the config file contains the forwarding option
$sudo vim /etc/ssh/sshd_config
----
X11Forwarding yes

Then when I ssh on the local machine to the remote machine include the -X option
$ssh -X me@remote_server

Setting up an RSS with rTorrrent

So I though t that I would have a go at setting up an RSS feed to put torrent files in to a watch folder for rTorrent to pick up.

http://code.google.com/p/rssdler/ Is meant to be very good at this

The requiremnets are this program pyhton above 2.4 and mechanize.
for my ubuntu server.

$ mkdir ~/bin
$cd ~/bin
$wget http://rssdler.googlecode.com/files/rssdler-0.4.0a.tar.gz
Once that is finished
$python --version
>Python 2.5.2
So that is OK
$apt-cache search mechanize
>python-mechanize - stateful programmatic web browsing
$sudo apt-get install python-mechanize

Now Build rssdler
$cd ~/bin
$tar -xvzf rssdler-0.4.0a.tar.gz

Now erm check the instructions!
http://code.google.com/p/rssdler/wiki/InstallInstructions

Looks like this is also required
$sudo apt-get install python-feedparser

Make dir for runtime files
$mkdir ~/.rssdler

Install rssdler
$cd ~/bin/rssdler040
$sudo python setup.py install

now apparantely it will just work by calling
$rssdler -r

eek no config file found
$cd ~/.rssdler
copy the sampe config.txt from here http://code.google.com/p/rssdler/wiki/CommentedConfig

Still does needs some work before it will do something useful. Will update soon.
####
Update: got it working my config file with all the blotted comments removed.
$cd ~/.rssdler
$cat config.txt | grep -iv '#'
----
[global]
downloadDir = /mnt/starbug1/terra1/samba/10-test/
workingDir = /home/morgy/.rssdler
log = 5
logFile = downloads.log
verbose = 5

scanMins = 2
runOnce = True
urllib = False

[testrss]
link = http://rss.thepiratebay.org/0
----

Convert debain/ubuntu to static IP

My Ubuntu Mini-ITX server was by default being assined an IP through DHCP from the router. If my router could assign IP based on the MAC address I would not have to assign a STATIC IP on the machine itself but could control it from a central location. Since I dont have this most of my machine have a static IP assigned. For Debain and Ubuntu systems this is quite easy.
Following instructions are for eth0 (the first real network socket)

$sudo vim /etc/network/interfaces

This text should already be present
----
auto lo
iface lo inet loopback

if these lines are present delete/comment out (#at the beginning of the line is a commet)
----
auto eth0
iface eth0 inet dhcp


Add the following (replace the x's with the assigned ip)
----
#### Manually Added Below Morgy ####
auto eth0
iface eth0 inet static
address 192.168.1.x
netmask 255.255.255.0
gateway 192.168.1.x
network 192.168.1.0
broadcast 192.168.1.255

Now restart networking
$ sudo /etc/init.d/networking restart


A more complete tutorial

Moving the /home dir

I am currently working on a low power mini-itx server. The operating system and boot partition is a 8GB flash drive, with a 1TB SATA drive for storage which can be powered down when not being used.

The OS is Ubuntu, running samba. I will be backing up my laptop to the server and so lots of data will be going into the homes directory. Therefore I need to move the /home/ directory to the 1TB Drive.

The partitioning/formatting/mounting steps taken for /dev/sda.

Partitioning http://tldp.org/HOWTO/Partition/fdisk_partitioning.html
$ sudo fdisk /dev/sda
option n to edit partition table.
option w to write the new partition.

Format ext3
$ sudo mke2fs -j /dev/sda1

Mounting, edit /etc/fstab
$ sudo mkdir -p /mnt/hostname/partitionname
$ sudo vim /etc/fstab

added this line
/dev/sda1 /mnt/hostname/partitionname ext3 rw,auto,async,errors=remount-ro 0 1

then
$ sudo mount -a

The new partition is now mounted.
Now move the homes directory
$ sudo cp -ax /home /mnt/hostname/partitionname/
$ sudo mv /home /home.old
$ sudo ln -s /mnt/hostname/partitionname/home /