Why do I need this?
Initially I planned to use device as NAS only. I really need very limited functionality (in fact – FTP and HTTP) so I was expected that vendor firmware will completely satisfy my needs. It was not the case.
First of all I found that firmware is not very stable – sometime device is not booting at all without any visible reasons. Second issue was that there is no firewall on this box. Its hard to believe – box is running on Linux but without ability to setup firewall! In combination with many services running on the device, without any ability to disable them it makes highly insecure system. Web gui itslelf was also buggy, with a lot of needless things (they have a “facebook” and “youtube” icons!), but without very important functions, like Firewall or IPv6. Using undocumented trick I was able to get the root, but after all I decided that it would be easer to start with normal distribution then to fix this crap.
Box itself looks as 2U server with a 12 units for the drive caddies. It have 4 Ethernet sockets, USB and serial interface. For me it was surprising that there is no VGA interface at all – this is untypical for x86 server. CPU is Intel Core 2 Duo running at 3.0Ghz. Device starts booting even without disks at all, so I was expecting to find some internal storage. It was easy – 1Gb Apacer USB flash disk is inside, plugged directly on the motherboard. All disks are connected using 2 Marvell 88SE6440 controllers (btw, it should be SAS compatible). Server have 2 power supply units with a hot-swap replacement.
As I already mentioned – there is no VGA card, so to install Debian on this box I tried to use serial console. I used USB → Serial adapter + NULL modem cable.
To install Debian on this box I used null-modem cable connected to USB → serial adapter. On my laptop I used minicom (but I think any other terminal should work fine). Terminal settings are 115200, 8n1, with minicom it should be “minicom -o -b 115200 -D /dev/ttyUSB0”. Then I am recommending to leave only 1 disk in the first caddy and use it as installation target. Later you will be able to add all your other disks. But with one disk it will be much easer setup our OS.
To setup OS you will need USB flash, because there is no CDROM in the server. It also should be possible to boot server from network, but I decided to use USB. I am recommending you to prepare 2 flash disks – one with Debian netinst and another one with SystemRescueCd.
Run terminal and turn on the server. After few seconds you should see booting progress in the terminal window. Press F2 to enter into BIOS. There are not too much options in it, we need only “boot order”. I am recompensing to exclude all devices from list, this will force server to boot from USB. Then insert USB flash with SystemRescueCd and save BIOS settings. Server will restart and will try to boot from inserted USB device. If everything is fine you should see SystemRescueCd boot menu in the terminal window. You will need to modify default options to use serial console. To do this press “Tab” and add to the boot prompt “console=ttyS0,115200” and then enter. You should see boot screen and zsh after some minutes. On this stage I am recommending to do 2 things:
- Backup internal 1Gb disk content (just in case) using dd command (I used dd+ssh).
- Remove partition table from the hard disk (I used gdisk tool).
Then you will need to reboot and insert USB with Debian installer. After boot prompt use “install console=ttyS0,115200” to start headless text install. I decided to create System 20Gb RAID1 partition for the Debian, which will be replicated on all drives. Also I created 4Gb RAID1 partition for swap (just in case). I did everything from installer, using manual partitioning. After restart I added HDD in the BIOS and found that system is not booting. Reason was Debian installer – grub was installed to the /dev/sda (USB flash) instead of disk, which was mapped as /dev/sdc. I booted from SystemRescueCd again and installed grub using chroot and grub-install. Also I am recommending to run update-initramfs on this stage, because for some reason md module was not activated on boot.
Solving problems after installation
On this stage Debian should boot normally. But when I trying to add more disks I found that system hangs with disk-related messages shortly. I found a lot of similar reports on the web (e.g. http://kerneltrap.org/mailarchive/linux-scsi/2010/4/12/6884601) without any visible solution. After all I decided to boot from SystemRescueCD again to check if disks are working correctly. And I found no issues! So, probably bug was fixed somewhere in -3 kernel and not back-ported. It is possible to use 3.x kernels on Debian using backports source. I used recommendations from this blog (http://yumechanmiru.blogspot.com/2012/02/debian-squeeze-32-kernel-update.html) and after installing latest 3.2 kernel and system restart all disk related issues where resolved. Only problem left was very slow loading time – it was trying to load ata_piix (motherboard sata controller?). Because it is not in use I just added it to the /etc/modprobe.d/blacklist.conf (blacklist ata_piix) and updated initramfs (update-initramfs -k all -u). Linux started very fast (in about 10 seconds) after restart.
Update: I just found that with 3.2 kernel disks are handled by Linux ahci driver. It fine for me, I think it also possible to force AHCI driver on 2.6, but I don`t want to downgrade. If there is no SAS disks in the system it should be safe to blacklist mvs driver.
Adding more disks.
After boot you can add more disks into the NAS. Hot-swap works fine, messages about news disks will be in a kernel log soon. I used gdisk (“u” command) to clone partition table and then added new drives to system RAID1 using mdadm. Don`t forget to do “grub-install /dev/sdX” for every new disk to make your system bootable in case of first disk failure. Also I created large software RAID partition on every disk which will be used as storage and configured it as RAID-5 with XFS. There was nothing special in this so I will not cover this process here. On this stage we have Debian up and running, without any issues. Hooray!
Server monitoring and fine-tuning.
It is very important for every server, especially for NAS to monitor hardware status and temperatures. Without it data will never be safe. In the next post I will cover ix12 internals, including temperature sensors, different devices on I2C bus, GPIO devices, system leds, etc. Stay tuned 🙂