Weekend Project: Home-baked Hand-made File Server

Before I begin, I should give you a short warning.

This post does not pretend to be a comprehensive guide about file server creation. It is more than likely that I have missed some steps. Some steps are described only on a high level, which sometimes is not enough for proper set up.

This is rather an attempt for myself to systemitize knowledge I obtained during my own project, with a purpose to enhance and probably reuse it later.

This was not a truly one-weekend project for me. It lasted a few weeks, one evening at a time. But I think you can do everything in one weekend, if not in one evening.

How it all started

A long time ago I got a present. It was an NAS: Network Attached Storage device. In layman terms, this is an external hard disk, which you can attach to your router and access via network, rather than via USB cable. It has a benefit compared to usual external drives, if you have wireless connection at home. Wherever you are at home, all your files are just one click away.

After a few years of usage, I started to notice that there are some glitches with the drive. Files started to disappear. This is not the behaviour you would expect from the device you use as a storage for family photos and videos, is it?

That's why the decision was made to replace old NAS with something new. Because I am now a more or less experienced Linux user, I decided to base my new solution on Linux, rather than on embedded Samba/FTP of an external device.

This approach also gives you an option to use the same computer not only as a file server, but for other purposes, too. The most obvious is to use it for the bit-torrent client, which is always on-line, regardless of the status of each of other home computers.

Hardware

Used laptop HP Compaq nc6000: Pentium M 1.6 GHz, 2Gb RAM, 60 Gb HDD, wireless and LAN connectivity.

2 x external HDD, in this case Western Digital Elements, 1Tb each

Installing the OS

My initial plan was to use CentOS as the operating system for the computer. Unfortunately, I realized that the processor in the laptop does not support pae, and pae is now a requirement for RedHat, hence CentOS distributions.

The only way to use CentOS on that laptop would be the re-compilation of the distribution image with a new kernel. That would also mean a new kernel compilation for each upgrade. That would be too complicated of a task for the purpose.

That's why I selected Debian.

It was not a pure server installation of Debian, but rather a Debian XFCE installation. Why? Because I would like to log into that server and use a graphical environment for some tasks. As I said, one of them would be the usage of the torrent client.

The process of installing Debian is well documented. There are no pitfalls on the route. The only recommendation, which I'd like to give you: connect your computer to LAN during the installation. The results with LAN and without LAN are radically different: Debian downloads lots of the packages from the network during the installation. I learned it from my own experience.

Once the OS is installed, you may wish to configure it to your own taste, adding and removing packages. You are unlikely to use much of the desktop software, because it is not the purpose of the server, is it?

External connectivity

Once your server is up and running as a standalone computer, your next step is to enable external connection to it.

The most popular way to connect to remote Unix-based computers is SSH. This gives you a terminal-based access using either usual terminal with command ssh <server name>, or specially built clients like PuTTY.

In order to enable an SSH connection to a remote computer, you need to install an SSH server. The most obvious choice here is the openssh-server package, which is available in Debian repositories.

After the installation of OpenSSH server, you may need to configure your SSH passwords, but that is not always required.

A little trick which you can already do at this stage: if your server has X-server installed (and I obviously had X-server, because XFCE was running on my server), you can run your graphical applications on the server, but show their windows on the local client. To do so, use ssh -X <username>@<server name> command in the Terminal window, and then start your application: gedit, vlc or whatever else. If you use the same username on local and remote computers, then you can drop it from the command. Although, be aware that you can't run sudo in this mode. You need to login as ssh -X root@<server name> for this.

External connectivity to the desktop

It is OK for the server to run in command-line only mode. But only if this is a real server.

In my example, I plan to use a remote laptop for running graphical applications, too. That's why external access to a full-featured desktop is required.

There are different options to open a remote desktop of the X-server enabled computer. My choice was VNC (Virtual Network Computing)

In order to run VNC on your server, you need to complete 5 steps:

enable auto-login of your user on the server. This potentially can lead to security issues, since you always have a user logged-in. To some extent, this can be resolved by using the password-protected screensaver.
enable XDMCP on the server. Login screen setup of XFCE has this configuration.
install the VNC server. There is a wide choice of them. I used the vnc4server package.
configure VNC passwords.
start VNC server, and include it in your startup procedure. I have done this by inclusion of command vnc4server -geometry 1024x650 :1 in the XFCE's startup sequence.

When these steps are done, you can then run a VNC client on the remote machine and connect to your server's graphical environment.

Couple of comments here:

Your remote graphical connection is not what you see on your server desktop.
Your connection to VNC session on remote server works like <server name>:<desktop number>. If you have not configured this specially, :1 is your default desktop. If you look above, I specified this as parameter on my vnc4server start command.

There are plenty of VNC clients, for either Windows or Linux. Most distributions have one or more clients by default. For example, Mageia 1 KDE comes with KRDC, Xubuntu with Vinagre. For my Windows machine, I used UltraVNC.

Sharing the disks

The next step is to enable the file-sharing functionality for the server.

I don't use internal hard drives for sharing, but rather my external USB drives. They came formatted as NTFS, and I decided not to change this. Because Debian does not like auto-mounting of NTFS USB drives, I decided to include them into fstab.

The next step was installation of Samba. That's a trivial task, which you can do either from Synaptic or simple command line.

During the Samba installation, you need to answer some configuration questions: workgroup, netbios name and so on. Don't be afraid, because you'll be able to change them later without any problems.

Samba configuration for my purposes was an easy task. Simply because I did not want any sophisticated user access rights management, quotas and so on. The home file server should allow everyone to read and write whatever s/he wants.

I could probably do this with simple editing of the Samba configuration file. But why do it manually, if there is a graphical way?

There are not many, but still some tools which allow you to configure Samba in a graphical environment. After looking at some of them, I decided to use SWAT. SWAT is not a graphical application per se. Rather, it is a web server which transforms web-page information into the Samba configuration file and back. Once installed, SWAT is available from any machine in the network using the http://<server>:901 link in the browser. Although, only root user has rights to change the configuration.

I added my shared folder on the external USB disk to the list of Samba shares, and it was immediately available on the network.

Now, everything was working.

There is a little note about SWAT. Even though SWAT showed nmbd and winbindd services as not running, Samba was still responding to the requests from either Linux and Windows machines.

Backup

Backups are crucial, when you are talking about important personal data, family photos and video archives and so on. That is why I went for two identical external hard drives.

The best way to make backup copies in Linux is, obviously, the rsync command. This package does not come with default Debian distribution, so you need to install it.

Then, I used command rsync -avh <main> <backup> to run the actual backup.

Usage of rsync gives you an additional benefit compared to the simple command cp. Of course, you get a list of files which were backed up during the previous rsync session, i.e. new files appeared on <main> after previous backup. But rsync also lists files which only exist on <backup>, but not on <main>. It gives you an option to track missing files and restore them from the backup, if necessary. Or delete from the backup location, too, if you want to free up the space.

But... How do you know if this command actually runs, and what the results are? For me, the easiest way is to get e-mail notifications in my inbox. That's why I installed another package on my Debian-based server: ssmtp. I have written about it already. In my case, I used almost the same script as mentioned in the linked article, with few changes. Most of the changes are due to the fact that my article was based on Xubuntu experience, rather than on Debian.

In detail, the changes are:

cp command was replaced with rsync, as I have mentioned above
I specified the full path to ssmtp: /usr/sbin/ssmtp. Simple ssmtp command did not work for me in Debian, although it worked well in Xubuntu before.
I added the user to group ssmtp to give proper permissions. Again, this was only required in Debian, not in Xubuntu.

And finally, you need to make backups regularly. Don't forget backups! Schedule them!

Daemon crond is the tool for scheduling tasks in Linux. Of course, I could edit some configuration files to start my script automatically. But the syntax of crond configuration file is not the most user-friendly. That's why I had to install yet another package in my Debian XFCE. Unfortunately, XFCE does not have any native scheduling tools, and I reverted to a GNOME application this time: GNOME Schedule. It is in the Debian repositories, of course. Then I was able to schedule my task using an interface which operates in hours, minutes and days, rather than spaces and dashes.

So, the job's done. The server is up and running. A few tests have been done which show how files are copied in the backup session, what is shown for deleted files, how the e-mail is sent and so on. The server is ready for use.

Here are my notes.

As I promised, this is not a comprehensive list of steps, but rather a short guide which has lots of room for improvements. If you have any recommendations to make it better, please share your advice in the comments below.

The main, area in which I am looking for advice is the ability to auto-start XFCE in the VNC session. I currently have startxfce4 line in the ~/.vnc/xstartup script, but I still get the vanilla X screen after the server boot. It can be easily fixed using PuTTY with two commands:


vnc4server -kill :1 
vnc4server -geometry 1024x650 :1

But that's not ideal. I asked about this on LinuxQuestions, have still received no reply. Will you be able to help?

This post was edited by djohnston.