In this post we are going to document how to make a raspberry use a root iSCSI volume while booting from SD card.

Disable swap

We do not want to use swap on a network drive. To disable swap

sudo systemctl disable dphys-swapfile

Reboot after disabling swap so that we can verify it is actually disabled.

free -h
               total        used        free      shared  buff/cache   available
Mem:           7.6Gi        95Mi       7.4Gi       1.0Mi       100Mi       7.4Gi
Swap:             0B          0B          0B

the 0B in output says swap is disabled.

Install open-iscsi and initramfs-tools

sudo apt install open-iscsi initramfs-tools

After installing, enable open-iscsi

sudo systemctl enable open-iscsi
sudo systemctl start open-iscsi

and get the initiator name

sudo cat /etc/iscsi/initiatorname.iscsi

Generate new initrd file with iSCSI support

To make sure iSCSI gets started on bootup, we need to touch this file

sudo touch /etc/iscsi/iscsi.initramfs
sudo update-initramfs -v -k `uname -r` -c

Make sure the new initrd file is generated

> ls -l /boot/init*
-rwxr-xr-x 1 root root 9412234 Feb  1 16:15 /boot/initrd.img-5.15.61-v8+

Make sure the iSCSI module can successfully load.

sudo vi /lib/modules-load.d/open-iscsi.conf

and change ib_user to #ib_user This is a step I do not understand. I don’t know what changing the ib_user does, but my setup didn’t work without this change. .

Reboot and confirm that the modules can load successfully.

~ $ systemctl status systemd-modules-load
● systemd-modules-load.service - Load Kernel Modules
     Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static)
     Active: active (exited) since Wed 2023-02-01 16:18:17 PST; 55s ago
       Docs: man:systemd-modules-load.service(8)
             man:modules-load.d(5)
    Process: 138 ExecStart=/lib/systemd/systemd-modules-load (code=exited, status=0/SUCCESS)
   Main PID: 138 (code=exited, status=0/SUCCESS)
        CPU: 67ms

Warning: journal has been rotated since unit was started, output may be incomplete.

Attach the iSCSI disk to pi

Discover targets.

sudo iscsiadm -m discovery -t sendtargets -p xxx.xxx.xxx.xxx

Choose the target we want and login

sudo iscsiadm -m node -T targetIQN -p portalIP:3260 --login

Verify that the device shows up in /proc/partitions

cat /proc/partitions  | grep sd
   8        0  314572800 sda

Setup pi installation on the iSCSI device

The easiest way to do that is to copy the current installation over to the iSCSI drive. To do that

Format the drive

sudo mkfs.ext4 /dev/sda

Note the UUID of the device

sudo blkid /dev/sda

Mount the device

sudo mount /dev/sda /mnt

then copy the existing filesystem over to the block device

sudo rsync -avhP --exclude /boot --exclude /proc --exclude /sys --exclude /dev --exclude /mnt / /mnt/
sudo mkdir /mnt/{dev,proc,sys,boot,mnt}

Edit the /mnt/etc/fstab to mount the block device in the root partition

sudo vi /mnt/etc/fstab

In this file, remove the second line corresponding to mounting / from sdcard and add the line below.

This ends the iscsi setup.

Bootloader setup

This is the step that ties everything together Create backups for /boot/cmdline.txt and /boot/config.txt

sudo cp /boot/cmdline.txt /boot/cmdline.txt.bak
sudo cp /boot/config.txt /boot/config.txt.bak

Modify /boot/config.txt to add the below lines

initramfs <YOUR_INITRD_IMAGE> followkernel

for example

initramfs initrd.img-5.15.61-v8+ followkernel

Modify cmdline.txt to boot using iSCSI device as root

console=serial0,115200 console=tty1 ip=::::{hostname}:eth0:dhcp root=UUID={UUID} rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait ISCSI_INITIATOR={INITIATOR_IQN} ISCSI_TARGET_NAME={TARGET_IQN} ISCSI_TARGET_IP={PORTAL_IP} ISCSI_TARGET_PORT=3260 rw

Reboot

sudo reboot

Verify that the installation is fine by ssh to the device and

~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G     0  3.7G   0% /dev
tmpfs           782M  1.2M  781M   1% /run
/dev/sda        295G  1.5G  278G   1% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
/dev/mmcblk0p1  255M   40M  216M  16% /boot
tmpfs           782M     0  782M   0% /run/user/1000

References

  1. https://jacobrsnyder.com/2021/01/20/network-booting-a-raspberry-pi-with-docker-support/
  2. https://mikejmcfarlane.github.io/blog/2020/09/12/PXE-boot-raspberry-pi-4-from-synology-diskstation#final-step-on-the-synology
  3. https://linuxhit.com/raspberry-pi-pxe-boot-netbooting-a-pi-4-without-an-sd-card/#8-install-raspbian-on-an-sd-card-and-install-needed-tools