User Tools

Site Tools


linux:nas

A quick NAS setup @ Debian 8 "jessie"

Disclaimer: I am not responsible for any loss of data, security breaches, fraud, omissions, errors, misconfigurations, service interruptions, slowdowns, freezes, breakdowns, or diminishments of any kind resulting from, caused by or attributed to the following or any content on this site.

Please backup before you proceed, and don't run any command without knowing what it is for.
And PLEEAAASE use a UPS!

You also need to know that the logical name (/dev/sdX) are NOT always the same. It can quickly change doing a reboot! Always check their serial numbers before you format them.

SOFTRAID! (mdadm)

apt-get install mdadm parted lshw -y

Use the default answer (“all”) when asked.

RAID Min. devices Max. device failure Space available Speed gain*
RAID0 2 0 devicesize*devices (devices)x read & write
RAID1 2 devices-1 devicesize (devices)x read
RAID5 3 1 devicesize*(devices-1) (devices-1)x read
RAID6 4 2 devicesize*(devices-2) (devices-2)x read
RAID10 4 && (devices % 2)==0 devices/2 devicesize*(device/2) (device)x read & (device/2) write

* I found it on the internet. A few sites disagree on it, but this is the most common calculation.

Filesystem Max. file size Max. volume size
ext2 2TB 32TB
ext3 2TB 32TB
ext4 16TB 16TB (32bit/DEFAULT) / 1EB ('-O 64bit')
NTFS 256TB 256TB
XFS 8 EB 16 EB

Quick FAQ:

Q: Durrr, just use hardware RAID!?

A: NO! When the RAID controller dies after 20 years or whatever, just imagine the price of getting an identical one then!!

FIND THE CORRECT DISKS

Don't be like me and changing the parition table on the wrong drives.

lshw -class disk

This is also useful when you need to replace a disk, because you can see the serial number.

Create a RAID5

In this (tested) example, we use 3x6TB disks to create a RAID5 with.
PLEASE consider RAID6 if you have 4 or more disks!!
And RAID is NOT the same as backup! It will help to some degree, but you NEED a backup! (Hint: Crashplan, Amazon Glacier, etc.)

Since the discs are larger than 2TB, we need to switch the partition table to GPT instead of MBR.

WARNING: THIS WILL DESTROY ALL CONTENT ON THE CHOSEN DISKS!!

lshw -class disk && lsblk #Be sure you choose the correct drives! This can seriously fuck everything up if don't.
parted --script /dev/sda mklabel gpt
parted --script /dev/sdb mklabel gpt
parted --script /dev/sdd mklabel gpt
parted -a optimal /dev/sda mkpart primary 0% 100%
parted -a optimal /dev/sdb mkpart primary 0% 100%
parted -a optimal /dev/sdd mkpart primary 0% 100%
mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdd1 # Create RAID5 with these three disks.
cat /proc/mdstat
mkfs -t ext4 -E lazy_itable_init=1 -O 64bit,sparse_super,filetype,resize_inode,dir_index,ext_attr,has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize /dev/md0
mkdir /mnt/md0
mount -t ext4 /dev/md0 /mnt/md0
chown -R mathias:mathias /mnt/md0
echo "/dev/md0        /mnt/md0        ext4    defaults        0       0" >> /etc/fstab

Grow it! (Add an extra drive)

We have quickly realised, that your porn collection needs more than the 11TB you got from 3x6TB!

Now it's time to expand it with another 6TB disk.

WARNING: THIS WILL DESTROY ALL CONTENT ON THE CHOSEN DISK!!

lshw -class disk && lsblk #FIND AND BE 100% SURE YOU GET THE CORRECT /dev/sXX!!!!!! CHECK USING DISK SERIAL NUMBERS!
parted --script /dev/sde mklabel gpt
parted -a optimal /dev/sde mkpart primary 0% 100%
mdadm --add /dev/md0 /dev/sde1
mdadm --detail /dev/md0 # You can now see, a spare has been added.
mdadm --grow /dev/md0 --raid-devices=4 # Replace 4 with the (new) total amount of discs..
watch -n1 'mdadm --detail /dev/md0 && echo "\n" && cat /proc/mdstat' 
# I recommend waiting until it's done reshaping..
systemctl stop smbd
umount /dev/md0
e2fsck -f /dev/md0 -C 0 # Check for errors, if you're in a hurry this is not required. 
resize2fs /dev/md0 -p
e2fsck -f /dev/md0 -C 0 # Also not needed.. Feel free to skip this step unless you're paranoid..
mount -t ext4 /dev/md0 /mnt/md0
systemctl start smbd
mdadm --detail /dev/md0 && df -h /mnt/md0

Make it more secure! (RAID5 to RAID6)

We have quickly realised, how important your porn collection is. And we have therefore bought another 6TB disk (“device”) to make 2 disk failure possible.

WARNING: THIS WILL DESTROY ALL CONTENT ON THE CHOSEN DISK!!

lshw -class disk && lsblk #FIND AND BE 100% SURE YOU GET THE CORRECT /dev/sXX!!!!!! CHECK USING DISK SERIAL NUMBERS!
parted --script /dev/sdf mklabel gpt
parted -a optimal /dev/sdf mkpart primary 0% 100%
mdadm --add /dev/md0 /dev/sdf1
mdadm --detail /dev/md0 # You can now see, a spare has been added.
mdadm --grow /dev/md0 --level=6 --raid-devices=5 --backup-file=/root/mdadmbackup_md0_raid5

OH NO WE LOST A DRIVE! SHIT! F***!!!!!! PANIC!!!1

Our huge porn collections seems to be in danger, and we must therefor replace it as soon as possible with a new drive. So we can get the it all back to into an stable state again.

First, we remove the drive(s) from the RAID with this:

mdadm --detail /dev/md0 && echo "\n" && cat /proc/mdstat # Identify the broken drive, it's usually marked with a (F) for failed
lshw -class disk # Find the serial number for the broken drive, if it's still online. And remove it from the system. 
mdadm /dev/md0 -r failed # Remove the failed drives, output will be like: mdadm: hot removed 8:65 from /dev/md0
mdadm /dev/md0 -r detached # OR run this, if the drive didn't fail, but was just detached.

Now it's time to add the new drive(s)
WARNING: THIS WILL DESTROY ALL CONTENT ON THE NEW CHOSEN DISK!!

lshw -class disk && lsblk #FIND AND BE 100% SURE YOU GET THE CORRECT /dev/sXX!!!!!! CHECK USING DISK SERIAL NUMBERS!
parted --script /dev/sdg mklabel gpt
parted -a optimal /dev/sdg mkpart primary 0% 100%
mdadm --add /dev/md0 /dev/sdg1
watch -n1 'mdadm --detail /dev/md0 && echo "\n" && cat /proc/mdstat' # Now just sit back, and watch it while it's recovering. This step is not required, but cool to look at.

Watch it build/grow/whatever

Want to see some fancy stats for the rest of the day? Alrighty!

watch -n1 'mdadm --detail /dev/md0 && echo "\n" && cat /proc/mdstat'

Now crank that text font size up to BIG and chill ;)

If that “Failed Devices : 0” change to 1 or more, it's okay to panic.. BECAUSE YOU NEED TO PANIC AND GET A NEW DRIVE ASAP!!!! When you have the new drive, look at the section just above this one.

Encryptiiionnn

Setup /w RAID

parted --script /dev/sdc mklabel gpt
parted --script /dev/sdf mklabel gpt
parted --script /dev/sdg mklabel gpt
parted -a optimal /dev/sdc mkpart primary 0% 100%
parted -a optimal /dev/sdf mkpart primary 0% 100%
parted -a optimal /dev/sdg mkpart primary 0% 100%
mdadm --create /dev/md1 --level=5 --raid-devices=3 /dev/sdc1 /dev/sdf1 /dev/sdg1 
cat /proc/mdstat
cryptsetup -y -v luksFormat /dev/md1 # The encryption setup
cryptsetup luksOpen /dev/md1 md1luks # Run this after each boot
mkfs -t ext4 -E lazy_itable_init=1 -O 64bit,sparse_super,filetype,resize_inode,dir_index,ext_attr,has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize /dev/mapper/md1luks # The bigger the longer, have patience.
mkdir /mnt/md1
mount -t ext4 /dev/mapper/md1luks /mnt/md1 # Run this after each boot
chown -R mathias:mathias /mnt/md1

Setup /wo RAID

cryptsetup -y -v luksFormat /dev/sdc # The encryption setup
cryptsetup luksOpen /dev/sdc porn # Run this after each boot
mkfs -t ext4 -E lazy_itable_init=1 -O 64bit,sparse_super,filetype,resize_inode,dir_index,ext_attr,has_journal,extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize /dev/mapper/porn 
mkdir /mnt/pornmount
mount -t ext4 /dev/mapper/porn /mnt/pornmount # Run this after each boot

Unmount

umount /mnt/pornmount
cryptsetup luksClose porn

Stats

cryptsetup -v status md1luks

My open script

This is the script I run, manually after boot. I've disabled samba, so I know when things are down, and to avoid mistakes.

set -e
cryptsetup luksOpen /dev/md1 md1luks
mount -t ext4 /dev/mapper/md1luks /mnt/md1
systemctl start smbd

File sharing (samba)

apt-get install samba samba-common libcups2
mv /etc/samba/smb.conf /etc/samba/smb.conf.bak
nano /etc/samba/smb.conf

  • Samba can't read linux users password. So you need to do add one manually by using smbpasswd before they can login.
  • Samba can read and understand Linux groups, use I recommend adding “force group” when doing this. To make sure everyone will always be able to access the files.
  • Samba follows the file system permissions, but I've seen it giving read permission if not configured correctly. So remember to configure “valid users” correctly.
  • Remember to set MTU to 9000 if possible (if you don't know what MTU is, then don't). Also do a speedtest on your drives, before getting mad on your NIC for only giving you 189Mbyte/sec (@10gbps link).

GLOBAL (required!)

The smb.conf consists of multiple parts (all in the same file), first we need the [global] settings tag.
I have fine tuned this to be fast as possible! If you know any other tweak that makes it faster, let me know :)

[global]
workgroup = WORKGROUP
server string = Samba Server %v
netbios name = debian
security = user
map to guest = bad user
dns proxy = no
mangled names = no
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072
strict allocate = Yes
read raw = Yes
write raw = Yes
strict locking = No
min receivefile size = 16384
use sendfile = true
aio read size = 16384
aio write size = 16384

Then you can choose what you want next, based on what you want to do.

Everyone has access, f*** security

As long you don't add any other shares, that requires login. The client wont be asked to login with something.

mkdir /mnt/md0/anonymous
chmod -R 0755 /mnt/md0/anonymous
chown -R nobody:nogroup /mnt/md0/anonymous

[Anonymous]
path = /mnt/md0/anonymous
browsable =yes
writable = yes
guest ok = yes
read only = no

Specific user only

mkdir /mnt/md0/mathias
chmod -R 0700 /mnt/md0/mathias
chown -R mathias:mathias /mnt/md0/mathias

[mathias]
 path = /mnt/md0/mathias
 valid users = mathias
 guest ok = no
 writable = yes
 browsable = yes
 create mask = 0700
 directory mask = 2700

You can add more users under the “valid users”, they need to be seperated by space (“user1 user2 user3”). Just be sure to update chmod/chown for it, and use a common group (& “force group”.. see below) :)

Specific group

mkdir /mnt/md0/entertainment
chmod -R 0770 /mnt/md0/entertainment
chown -R mediauser:smbgroup /mnt/md0/entertainment

[entertainment]
 path = /mnt/md0/entertainment
 valid users = @smbgroup
 guest ok = no
 writable = yes
 browsable = yes
 force user = mediauser
 force group = smbgroup

Add (linux) users to group

First we to create a group

groupadd smbgroup

Create user, and make this the primary group

useradd mediauser -G smbgroup

Add existing user

usermod -a mathias -G smbgroup

Add (linux) users to Samba

Samba can't read the passwords set in Linux, and I also don't recommend having your top secret password floating over the network.

Luckily it's easy to add them using:

smbpasswd -a mathias

File sharing (nfs)

Server

apt-get install nfs-kernel-server portmap
nano /etc/exports # Modify and insert the lines below
# /mnt/md0/nfs/esxi 10.20.40.0/24(rw,sync,no_root_squash,no_subtree_check)
# /mnt/md0/sharedfolder 10.20.30.40(rw,sync,no_root_squash,no_subtree_check)
exportfs -ra
systemctl restart nfs-kernel-server

Client

apt-get install nfs-common
mkdir /mnt/localnfsfolder
nano  /etc/fstab # Modify and insert the lines below - BUT BE CAREFUL, THIS FILE CAN BREAK YOUR SYSTEM!
# 10.20.30.123:/mnt/md0/sharedfolder /mnt/localnfsfolder nfs rw,async,hard,intr,noexec 0 0
mount -a # or: mount /mnt/localnfsfolder

Windows

Get the UID and GID from the folder you wish to mount: ls -n /mnt/md0/sharedfolder

Add the UID and GID to the registry first, so you don't have to restart the Client For NFS service after install.

REG ADD HKLM\Software\Microsoft\ClientForNFS\CurrentVersion\Default /v AnonymousUid /t REG_DWORD /d 1000 
REG ADD HKLM\Software\Microsoft\ClientForNFS\CurrentVersion\Default /v AnonymousGid /t REG_DWORD /d 1001

Turn Windows features on or off > Client for NFS

mount -o anon 10.20.30.123:/mnt/md0/sharedfolder z:

10Gbps network

The following things might help you, I haven't fully tested them yet. But I have all of the following applied.

If you're doing an iperf3 from Windows, and it just seems wrong, remember to do parallel connections. Because Windows is Windows.. (iperf3 -c iphere -P 20)

Find the PCI id (in my case 80861528)and turn up for the bandwidth on it - I had no speed increase by this:

setpci -v -d 8086:1528 e6.b=2e

Add MTU 9000:

allow-hotplug eth2
iface eth2 inet static
  address 10.11.12.13
  netmask 255.255.255.0
  mtu 9000

Also make sure your switch is not limiting the MTU, I needed to set mine to 10k in L2 MTU before I could do 9k MTU.
If you want to quickly switch between MTU's, ifconfig eth0 mtu 9000 up is pretty useful

Apply some fancy things to /etc/sysctl.conf, if MTU9000 doesn't achieves the results you want (reload with sysctl -p):

net.ipv4.tcp_reordering = 16
net.ipv4.tcp_fack = 0
net.ipv4.tcp_dsack = 0
net.ipv4.tcp_allowed_congestion_control = htcp reno highspeed scalable lp
net.ipv4.tcp_congestion_control = highspeed
net.ipv4.tcp_low_latency = 1
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_adv_win_scale = 4
net.ipv4.tcp_mem = 16777216 33554432 67108864
net.ipv4.tcp_rmem = 8388608 16777216 33554432
net.ipv4.tcp_wmem = 8388608 16777216 33554432
net.ipv4.udp_mem = 2097152 8388608 16777216
net.ipv4.udp_rmem_min = 262144
net.ipv4.udp_wmem_min = 262144
net.core.rmem_default = 4194304
net.core.wmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 33554432
net.core.optmem_max = 4194304
net.core.somaxconn = 8192
net.core.netdev_max_backlog = 3000000

Backup

Crashplan

You seriously need a backup solution. I used to use Crashplan as my primary. But now they don't exists for home use anymore, so this section is just kept in case you go for their business version. And because it looks cool to have a lot of content on this page.
But please, do not set it to backup your movie collection or something that you can easily get again. It would be really sad if they started to prevent NAS's from using their service!

apt-get install libXtst-dev
wget https://download.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.5.2_Linux.tgz
tar zvfx CrashPlan_4.5.2_Linux.tgz
cd crashplan-install/
#enter
#enter
#q
#yes (enter)
#enter
#enter
#enter
#enter
#enter
#enter
#enter
#enter
#enter
#enter

# Make sure you have X11 forwarded, also please don't run this as root (I did it anyway).
/usr/local/bin/CrashPlanDesktop

# If the gui doesn't start, check /usr/local/crashplan/log/ui_output.log for more info.

rsync

I should write something useful here, but that must be some other day…
Until then, run rsync -a /source/folder /destination/folder to sync locally, if you want something to look at, then run rsync -av –progress /source/folder /destination/folder. To create a complete copy, you might want to add rsync -a –delete /source/folder /destination/folder to your cron, however, if you accidentally delete stuff at source. You're doomed. Do a test simulation with –dry-run

Applications

Oh yes, lets introduce some more possible security flaws to our system! <3

Syncthing

Your own personalized cloud! For the clients I recommend that you use Syncthing-GTK instead of Syncthing. It's much better when you have a gui, and want to be able to sync file changes as soon as they happen.

apt-get install curl
curl -s https://syncthing.net/release-key.txt | apt-key add -
echo "deb http://apt.syncthing.net/ syncthing release" |  tee /etc/apt/sources.list.d/syncthing.list
apt-get update
apt-get install syncthing
useradd syncthing -G smbgroup -d /mnt/md0/syncthing
mkdir /mnt/md0/syncthing
chmod -R 0770 /mnt/md0/syncthing
chown -R syncthing:smbgroup /mnt/md0/syncthing

Save this as /etc/systemd/system/syncthing.service:
[Unit]
Description=Syncthing
After=network-online.target

[Service]
Type=simple
User=syncthing
Group=smbgroup
UMask=007

ExecStart=/usr/bin/syncthing

Restart=on-failure

# Configures the time to wait before service is stopped forcefully.
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

systemctl start syncthing
systemctl status syncthing
systemctl enable syncthing

And append the following to /etc/samba/smb.conf:

[syncthing]
 path = /mnt/md0/syncthing
 valid users = mathias
 guest ok = no
 writable = yes
 browsable = no
 create mask = 0750
 directory mask = 2750
 force user = syncthing
 force group = smbgroup

chmod +x /etc/init.d/syncthing
/etc/init.d/syncthing start
sed -i 's/<address>127.0.0.1:8384<\/address>/<address>0.0.0.0:8384<\/address>/g' /mnt/md0/syncthing/.config/syncthing/config.xml
/etc/init.d/syncthing restart # Now visit your.nas.ip.adr:8384 and create login info
/etc/init.d/samba restart

Emby

Emby is a super awesome alternative to Plex. I would actually say it's better than Plex!
It use it to sync my Kodi watch status, and auto scrobble what I'm watching to Trakt.tv

#These two lines are not really needed, but will avoid future warning messages.
wget http://download.opensuse.org/repositories/home:emby/Debian_8.0/Release.key
apt-key add - < Release.key 

echo 'deb http://download.opensuse.org/repositories/home:/emby/Debian_8.0/ /' >> /etc/apt/sources.list.d/emby-server.list 
apt-get update
apt-get install emby-server -y --force-yes  
usermod -g smbgroup emby
systemctl start emby-server # http://ip.of.your.nas:8096/
systemctl status emby-server
systemctl enable emby-server

Deluge

A super great torrent client! With a neat web interface and full proxy support. You can even connect the desktop client directly too it if you don't want an webinterface.

apt-get update
apt-get install deluge deluged deluge-web
usermod -g smbgroup debian-deluged
update-rc.d deluge-daemon remove

[Unit]
Description=Deluge Bittorrent Client Daemon
After=network-online.target

[Service]
Type=simple
User=debian-deluged
Group=smbgroup
UMask=007

ExecStart=/usr/bin/deluged -d

Restart=on-failure

# Configures the time to wait before service is stopped forcefully.
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

[Unit]
Description=Deluge Bittorrent Client Web Interface
After=network-online.target

[Service]
Type=simple

User=debian-deluged
Group=smbgroup
UMask=007

ExecStart=/usr/bin/deluge-web

Restart=on-failure

[Install]
WantedBy=multi-user.target

sed -i 's/ENABLE_DELUGED=0/ENABLE_DELUGED=1/g' /etc/default/deluged
systemctl start deluged
systemctl status deluged
systemctl enable deluged
systemctl start deluge-web # http://your.nas.ip.here:8112 default password: deluge - after password change, do a F5 and connect to daemon
systemctl status deluge-web
systemctl enable deluge-web

Preference suggestions:

  • Those a different “Download to” other than where you actually them to end. I will recommend using /tmp. And then my “Move completed to” folder is my /mnt/md0/entertainment/linux-live-cds folder.
  • Specify an proxy, to keep your IP secret when you download all those Linux live cd ISO's
linux/nas.txt · Last modified: 2020/06/17 11:05 by mathias