Integrating installation of Internet Archive as a server

That changed something - a very strange listing with ??? for the permissions for usb0
/var/log/messages says …

Jul 15 19:03:10 rachel kernel: [    6.429796]  sda: sda1 sda2
Jul 15 19:03:10 rachel kernel: [    6.441169] sd 0:0:0:0: [sda] Attached SCSI disk
Jul 15 19:03:10 rachel kernel: [    6.848106] 8021q: 802.1Q VLAN Support v1.8
Jul 15 19:03:10 rachel rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd.  [v8.1901.0]
Jul 15 19:03:10 rachel rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="311" x-info="https://www.rsyslog.com"] start
Jul 15 19:03:11 rachel usbmount[337]: /dev/sda does not contain a filesystem or disklabel
Jul 15 19:03:11 rachel kernel: [    7.001680] Adding 102396k swap on /var/swap.  Priority:-2 extents:1 across:102396k SSFS
Jul 15 19:03:11 rachel kernel: [    7.054892] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
Jul 15 19:03:11 rachel kernel: [    7.054995] brcmfmac: power management disabled
Jul 15 19:03:11 rachel usbmount[392]: executing command: mount -texfat -osync,noexec,nodev,noatime,nodiratime /dev/sda2 /media/usb0
Jul 15 19:03:11 rachel kernel: [    7.342151] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Jul 15 19:03:11 rachel kernel: [    7.342158] 8021q: adding VLAN 0 to HW filter on device eth0
Jul 15 19:03:11 rachel kernel: [    7.357068] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jul 15 19:03:11 rachel mount.exfat: volume was not unmounted cleanly
Jul 15 19:03:11 rachel kernel: [    7.765384] fuse init (API version 7.27)
Jul 15 19:03:11 rachel usbmount[392]: executing command: run-parts /etc/usbmount/mount.d
Jul 15 19:03:16 rachel usbmount[393]: executing command: mount -tvfat -osync,noexec,nodev,noatime,nodiratime,gid=1000,uid=1000,umask=007 /dev/sda1 /media/usb1
Jul 15 19:03:16 rachel kernel: [   12.328417] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
Jul 15 19:03:16 rachel usbmount[393]: executing command: run-parts /etc/usbmount/mount.d
Jul 15 19:03:17 rachel kernel: [   13.439103] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready

Are you trying with multiple USB drives plugged in? I would stick with 1 until one works.

It looks like one mounted. What do the “lsblk”, “blkid”, and “ls /media/usb0” commands show?

If the permissions are wrong you can set those in usbmount.conf under FS_MOUNTOPTIONS. vfat filesystems don’t support permissions so they have to be set when mounting.

FS_MOUNTOPTIONS="-fstype=vfat,gid=pi,dmask=0007,fmask=0117"

“volume was not unmounted cleanly” can generally be ignored. It should still work. If all else fails you can always do “sudo mount /dev/sda1 /media/usb0 -o uid=pi,gid=pi” to mount the drive and see if it works.

1 Like

I’ve just got one drive plugged in - I switched to a Toshiba - exfat, because I thought it might be the drive giving problems.

pi@rachel:~ $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    0  1.8T  0 disk 
├─sda1        8:1    0  200M  0 part /media/usb1
└─sda2        8:2    0  1.8T  0 part /media/usb0
mmcblk0     179:0    0 59.6G  0 disk 
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 59.4G  0 part /
pi@rachel:~ $ blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="B6BB-0F0E" TYPE="vfat" PARTUUID="29a2fca6-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="638417fb-7220-47b1-883c-e6fee02f51ac" TYPE="ext4" PARTUUID="29a2fca6-02"
pi@rachel:~ $ ls /media/usb0
ls: cannot access '/media/usb0': Transport endpoint is not connected
pi@rachel:~ $ 

I edited FS_MOUNTOPTIONS based on one of the links you gave me, but I just noticed there was a
line break after the first group, I’m going to try replacing the vfat entry with the one you gave me and restart

FS_MOUNTOPTIONS="-fstype=ntfs-3g,nls=utf8,umask=007,gid=1000
-fstype=fuseblk,nls=utf8,umask=007,gid=1000 -fstype=vfat,gid=1000,uid=1000,umask=007"

Making that change didnt seem to fix anything /media/usb0 still shows empty even though it shouws something in lsblk

pi@rachel:~ $ ls -al /media/usb1
ls: cannot access '/media/usb1': Transport endpoint is not connected
pi@rachel:~ $ ls -al /media
ls: cannot access '/media/usb1': Transport endpoint is not connected
total 33
drwxr-xr-x 10 root root 4096 Jul 14 13:10 .
drwxr-xr-x 22 root root 4096 Jul 11 20:36 ..
lrwxrwxrwx  1 root root    4 Jul 14 13:10 usb -> usb0
drwxrwx---  2 root pi    512 Dec 31  1969 usb0
d?????????  ? ?    ?       ?            ? usb1
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb2
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb3
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb4
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb5
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb6
drwxr-xr-x  2 root root 4096 Jul 14 13:10 usb7

If this works on IIAB can you get their install script and see what they do?

I cant figure out the install script (complex ansible scripts) - but have been looking at usbmount.conf

I would just format it to ext4 and use it with Linux if you’re on a time crunch. The issue seems like it’s just around mounting other file systems. If you have to use it outside of linux I would try formatting it to NTFS instead of exFat and follow those NTFS instructions in that link.

Hi @jamesk , I had to put demonstrating Rachel on hold for Dweb camp and demo on IIAB instead.

Mounting disks seems to be the real issue, I have a disk that works fine on IIAB, but not on Rachel

The usbmount.conf files aren’t quite the same

diff usbmount.conf.pi usbmount.conf.rachel
17c17
< FILESYSTEMS="vfat ext2 ext3 ext4 hfsplus exfat fuseblk ntfs"
---
> FILESYSTEMS="vfat ext2 ext3 ext4 hfsplus ntfs fuseblk exfat"
49c49,50
< FS_MOUNTOPTIONS=""
---
> # Note the gid on Rachel for pi is 1000
> FS_MOUNTOPTIONS="-fstype=ntfs-3g,nls=utf8,umask=007,gid=1000 -fstype=fuseblk,nls=utf8,umask=007,gid=1000 -fstype=vfat,gid=pi,dmask=0007,fmask=0117"

I’ve tried rebooting with the empty FS_MOUNTOPTIONS and it doesnt seem to make a difference.

Can you link me to the IIAB image you use? I’m going to download it and see what they do.

Thanks - the exact instructions I use are at https://github.com/internetarchive/dweb-mirror/blob/master/INSTALLATION-iiab-raspberry.md

Note - that uses the Raspbian desktop image, and I think their script process is making changes.

I just installed Raspbian Desktop to start with and it doesn’t have usbmount installed by default.

pi@raspberrypi:~ $ sudo apt-get install usbmount

Reading package lists… Done
Building dependency tree
Reading state information… Done
The following additional packages will be installed:
liblockfile-bin liblockfile1 lockfile-progs pmount
Suggested packages:
cryptsetup
The following NEW packages will be installed:
liblockfile-bin liblockfile1 lockfile-progs pmount usbmount
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 142 kB of archives.
After this operation, 634 kB of additional disk space will be used.
Do you want to continue? [Y/n]

so that confirms this is just about the usbmount setup and not the distro. I’ll have to dig through the iiab installer to find out exactly where they do the usbmount setup to see what the difference is. Their installer is pretty convoluted.

Agreed - I added an ansible script to it based on a template from one of the others, but I don’t understand the installer overall.

It looks like they use templates to install packages and files and those are in .yaml files, One interesting thing is that it looks like they add other repositories to the sources list. I wonder if they use a usbmount from an ubuntu or other repo.

Hi James - I just got back to Australia, and wondered if you found anything out ?

I couldn’t find anything different in IIAB’s configuration that would make it work other than that I think they add some alternative sources for packages in their install process. I just got a Raspberry Pi 4 yesterday so I’m looking at everything again now. I’ll let you know if I find anything but I think an alternative to usbmount is probably necessary for multiple reasons.

Okay so the issue is 100% the usbmount package Raspbian Buster provides. The final comment from this solution that builds the package from the git works for me. From a fresh install of Raspbian Buster this is my process.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git
sudo apt-get install debhelper
sudo apt-get install exfat-fuse
cd /var/tmp
git clone https://github.com/rbrito/usbmount.git
cd usbmount
sudo dpkg-buildpackage -us -uc -b
cd ..
sudo apt install  ./usbmount_0.0.24_all.deb
change /etc/usbmount.conf file systems to FILESYSTEMS="vfat ext2 ext3 ext4 ntfs-3g ntfs exfat hfsplus fuseblk"
sudo reboot now

This worked for me with exfat and the drive gets mounted to /media/usb0 or /media/usb1. No extra settings or files required. Hope that works for you.

James

Thanks - I’ll give this a go

No problem. It worked for me with the RACHEL-Pi image for the Pi4/Buster I made but there does seem to be a boot issue where it won’t boot if you’ve got Raspbian on a MicroSD and a USB drive plugged in at boot. That’s an old bug that I thought was gone but I know they’re working on the USB boot support for the 4 so we’ll see. You may have to plug the drive in after boot.

Minor point - it should be: change /etc/usbmount/usbmount.conf ...

I think you also need apt-get install ntfs-3g if ntfs-3g is to be included in the FILESYSTEMS list.

Apart from that, moving to that newer usbmount seems to have nailed it - thanks.

I need to do a clean install, just to be sure.

I have notes

Download the temporary Raspbian Buster + Rachel image - 
Temporarily at ...
http://rachelfriends.org/downloads/public_ftp/rachelpi_64EN/rachelpi_2019/rachel-pi_kolibi_buster_unofficial.7z

Is that still the best place to be getting it?