When loading modules on my Rachel server, it takes an unusually LONG time. It seems that loading a module within Kolibri is faster than loading the same module outside of Kolibri. I have a 250/250Mbps Internet connection. Is there anything that can be done to speed this up?
When I have the Rachel server deployed in the field, is there a faster way to install and update the modules? Of example, would it be possible for me to simply attach an external USB hard drive that contains all the updated content and have the RACHEL server update the content and add new modules from the attached USB hard drive?
Hi @LarryDT - when you say âloadingâ do you mean getting the module onto RACHEL, or accessing a module from a tablet or laptop?
I assume you mean getting the module on RACHEL, which leads me to my next question, when you say unusually, are you comparing to how long it used to take to load RACHEL modules or something else?
Many times, the limiting factor is your bandwidth download, our serverâs bandwidth uploading, or the HDD on the device. The last of these is most prevalent when adding large modules with lots of files, worldmap for example. You can speed this up by opting for an SSD in the device.
It is possible to add modules to a USB and then plug the USB into RACHEL, however, it requires understanding how to SSH into a linux shell. This can be quite complicated. We hope to add a feature in the near future to other for point and click type installations.
Hi Jeremy, Thanks for the fast reply. Yes, I am referring to loading modules. I have a download speed at my house of 250Mbps. I am currently setting up a Rachel PI (tried both a Raspberry Pi 3 and 4 with similar results). I am using a fast SD card (256GB). I loaded the latest Rachel-PI Kolibri image. After burning the image, I selected all the modules to load (except the ones I know donât work). It only averages a few hundred kbps (occasionally may get to 2Mbps). I started it more than two days ago and so far the SD card has a total of 128G used (started with about 40G, so has downloaded around 80-90GB in the last 55 hours). My concern is when the Rachel Servers we will be using in Uganda need to be updated (or new modules added), we will have to spend an unreasonably long time in town at the Internet Cafe for the updates. I have a Rachel-Plus 3.0 on order and will try a similar experiment with it.
This is why I was wondering if it would be faster to somehow plug a USB Drive into the Rachel-Plus to update content. We have people visiting the school ever couple months and we could send a USB drive with the latest content with them and use it to update the Rachel-Plus (saving a lot of time waiting for the update at the Internet Cafe in town). It would be nice if there was an options in settings where we could select if the modules are to be updated from the servers on the internet or from a locally attached drive.
You mentioned it is possible but complicated. If you have any instructions for how this is done, maybe I can have one of my programmers look at it and write me a script or something.
I have noticed this before with my connection when downloading modules but Iâve always assumed that it was due to rsync transferring many individual files like Jeremy mentioned. It explains the connection speed fluctuating constantly, but my connection never gets over 2MB so itâs hard for me to test why yours wouldnât get higher on larger files. You may want to look at downloading modules through oer2go.org or the FTP and using WinSCP to copy them over. There is a tutorial included in the latest RACHEL-Pi Kolibri image that shows how to do that. Some tests that would help determine what causes this would be
Download a module from the World Possible FTP to see if the connection gets higher than 2mb.
Use an rsync command provided when you look at the details of the modules on oer2go.org on another system. For instance, http://oer2go.org/viewmod/en-coreknowledge has ârsync -Pavz rsync://dev.worldpossible.org/rachelmods/en-coreknowledge ./â. Those are essentially the same command as what runs when you install a module from the admin interface, so if you run that on the same connection but using something like Ubuntu on a desktop PC, it would tell me whether or not this is an issue with the Raspberry Pi.
Since the RACHEL-Pi just requires an SD card, if youâre planning to update them in the field every few months it will probably be easier and more reliable to just update one SD card that has all of the new and old content and make copies of that card to deploy to each RACHEL-Pi. This way you only have to download the content once and on a preferred connection so you donât have to rely on internet cafes which can be expensive and slow in some places. If you plan on maintaining Kolibri users that might require migrating Kolibri data between cards. @giakonda or @SHOMI might be able to give some insight on how they do their deployments.
I will look into this a bit further today to see if some settings change the speeds.
@jeremy after looking at this more, I think this might be an issue with the World Possible servers. Iâve tested on the Pi, Windows Desktop, and Ubuntu Desktop. Using wget on a zip, downloading a zip in browser, or using rsync the speeds fluctuate greatly even when itâs transferring a large file. It increases to my max speed and then fluctuates back down to a stall and then back up again. Iâve tested the same with other sites without issue.
I have done some more testing per your suggestion. I downloaded (just using FileZilla FTP on my computer and connected via GigE LAN and 250Mbps broadband) the rachelUSB_32EN_4.0.zip file which is approximately 20.78GB. It transferred at an average speed of 3.5MB/s (28 Mbps). It took 1 hour and 36 minutes. I then went to speedtest.tele2.net and downloaded the 10GB file. It would have taken around 45 minutes at the same speed as the Worldpossible FTP site. It only took 9 minutes and had an average speed of 19.2MB/s (153.6Mbps). So, it does appear that the Worldpossible FTP site is much slower.
I then tested the Raspberry Pi using a 4B with the latest Buster load. It averaged around 1.2MB/s (10 Mbps). The 4B has a 1G Ethernet interface and was also connected to my GigE LAN with 250Mbps broadband. The Raspberry Pi appears to be more of a limitation than the speed of the Worldpossible FTP site (although that seems to be slow also). A faster Worldpossible FTP site wonât speed up the loading of modules on my Rachel-Pi.
I think I probably need to wait and see how the RACHEL-Plus performs before coming to any conclusions. I will only be using the Rachel-Pi for training purposes. It will be more important that the Rachel-Plus is able to load fast. I will look into the loading and updating of modules with a USB drive, since most internet access is charged by the MB and could be expensive also for the updates.
When you tested with the Pi4 did you do a module install through the RACHEL admin interface or using wget on ssh? The Pi4 should be able to download at much faster speeds and based on your connection, should do better than mine does at 2mb. Can you try the following and let me know the results on your pi
I have not downloaded any modules in the past 2 months, so I am not sure about the fluctuations you see.
For my deployment, I am using the RPI-3B+ with a MicroCenter Inland 128 GB SSD in and external aluminum enclosure, No Microsd on the RACHEL server. I have one running and a spare with the instructor.
I am using a 1:2 USB 2.0 flash drive duplicator (Best Duplicator and Startech). The duplicator can accept the USB cable from the SSD as the source and then copy to 2 target SSD. It takes about 75 minutes to replicate my SSD on either of my duplicators.
I am not sure of your desktop types, but thought I would add this in case.
When I make an RPI 3B+ desktop image to the microsd with what I need, I back it up to a USB flash drive as well. I can make desktop duplicates the same as the RACHEL SSD but much faster due to the size. Microsdâs and SSD are so cheap, I keep several spares with the instructors supplies and a few with me to keep things updated and ready for another trip.
When I did the test with the Pi4, I booted to a clean Buster load and used the Buster GUI and FileZilla so that it would replicate exactly what I did on my windows desktop.
Each of the resource centres in Siavonga, Monze and Chilanga have sd card duplicators so one download can be easily duplicated and passed on to other schools. They also have ssd duplicators so once Raspbian has been set up on a laptop that too can be passed on to other schools. Note: This method requires laptops with very similar hardware, we use fujitsu lifebooks in all our schools. You can not do this with Windows because microsoft prevents this form of copying. Another advantage of using Linux. Teansfer of pupil data is easily done using LibreOffice and the csv format. Details of how to do this are available on the kolibri Community site or at giakonda.org.uk/resources
I hope this helps
Kind Regards Howard