I am trying to upgrade Kolibri on a RACHEL-Plus 3.0 using the zz-kolibri-upgrade module. It stalls (as expected) but it remains stalled even after waiting several hours. I have done this successfully on several other RACHELs but for this particular RACHEL it just gets stuck and will not move past the stalled state.
Hi @gary.friesen - Can you please confirm the following
- The RACHEL version your device is on
- The Kolibri version your device is on
- The information shown when you press “details” on the stalled upgrade
- The RACHEL and Kolibri versions of the successful upgrades.
James
-
RACHEL installer = 3.0.2, OS = Ubuntu 16.04.4 LTS, Hardware = i686
-
Kolibri version = 0.9.2
3, Details when it stalls out:
zz-kolibri-upgrade stalled
0.05kB/s
command: rsync -Pavz --del rsync://dev.worldpossible.org/rachelmods/zz-kolibri-upgrade …/modules/
runtime: 13:18:12 stalled
files_done: 4 out of 4 ( 100% )
data_done: 55272 out of 32104 ( 172% )
data_rate: 0.05kB/s
latest output:
Running: /media/RACHEL/rachel/modules/zz-kolibri-upgrade/finish_install.sh
- For the successful upgrades, the Kolibri version was 0.9.2 and the RACHEL was the same as above.
Thank you. Unfortunately there’s nothing here that helps with what went wrong. Can you please log in to the device over SSH using putty and run the command
’
- sudo systemctl status kolibri
Any output information from that may help.
Are you doing a full fresh upgrade of everything on these devices? 3.0.2 is very old and we have newer recoveries with updated Kolibri among other changes.
Hi James, I’m just how getting back to this project. 
Here is the system status for Kolibri:
oot@CMAL-a788:~# sudo systemctl status kolibri
● kolibri.service - LSB: kolibri daemon, an offline education platform
Loaded: loaded (/etc/init.d/kolibri; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2022-01-05 16:06:05 UTC; 2 days
Process: 811 ExecStart=/etc/init.d/kolibri start (code=exited, status=1/FAILUR
Jan 05 16:06:04 CMAL-a788 kolibri[811]: backup = dbbackup(version_file_conte
Jan 05 16:06:04 CMAL-a788 kolibri[811]: File "/usr/lib/python3/dist-packages/k
Jan 05 16:06:04 CMAL-a788 kolibri[811]: for line in db.connections[“default”
Jan 05 16:06:04 CMAL-a788 kolibri[811]: File "/usr/lib/python3.5/sqlite3/dump.
Jan 05 16:06:04 CMAL-a788 kolibri[811]: query_res = cu.execute(q)
Jan 05 16:06:04 CMAL-a788 kolibri[811]: sqlite3.DatabaseError: database disk ima
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Control process exited, c
Jan 05 16:06:05 CMAL-a788 systemd[1]: Failed to start LSB: kolibri daemon, an of
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Unit entered failed state
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Failed with result 'exit-
lines 1-15/15 (END)…skipping…
● kolibri.service - LSB: kolibri daemon, an offline education platform
Loaded: loaded (/etc/init.d/kolibri; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2022-01-05 16:06:05 UTC; 2 days ago
Process: 811 ExecStart=/etc/init.d/kolibri start (code=exited, status=1/FAILURE)
Jan 05 16:06:04 CMAL-a788 kolibri[811]: backup = dbbackup(version_file_contents)
Jan 05 16:06:04 CMAL-a788 kolibri[811]: File “/usr/lib/python3/dist-packages/kolibri/core/deviceadmin/utils.py”, li
Jan 05 16:06:04 CMAL-a788 kolibri[811]: for line in db.connections[“default”].connection.iterdump():
Jan 05 16:06:04 CMAL-a788 kolibri[811]: File “/usr/lib/python3.5/sqlite3/dump.py”, line 55, in _iterdump
Jan 05 16:06:04 CMAL-a788 kolibri[811]: query_res = cu.execute(q)
Jan 05 16:06:04 CMAL-a788 kolibri[811]: sqlite3.DatabaseError: database disk image is malformed
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Control process exited, code=exited status=1
Jan 05 16:06:05 CMAL-a788 systemd[1]: Failed to start LSB: kolibri daemon, an offline education platform.
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Unit entered failed state.
Jan 05 16:06:05 CMAL-a788 systemd[1]: kolibri.service: Failed with result ‘exit-code’.
~
HI @gary.friesen - It looks like on this image, the database was malformed during the upgrade. It’s a known issue that has some discussion here
Before you try any of the solutions, if there is important data in your database on that device, make sure to back up the database to your PC. It’s at the folder /.data/.kolibri/db.sqlite. If using WinSCP to transfer files, hit CTRL + H to unhide hidden folders and see /.data.
The options they suggest are first trying the command
kolibri manage migrate
If that doesn’t fix it, the more serious commands they suggest are there, but I don’t want to repeat them here in case someone goes that route and gets into more issues. If you have questions around that though I’d be happy to hel.
If you are on 3.0.2 and trying to upgrade the device in a factory state without installed modules/content/user data, I highly suggest using our latest USB recovery which has Kolibri 0.14.7 among other things.
James
I copied db.sqlite3 to my laptop. It was 476MB in size. Does this file contain ALL the user content? The school created quite a few courses and I don’t want to lose any of their work.
I used SSH (PuTTY) and ran kolibri manage migrate. This is the result:
280 packages can be updated.
220 updates are security updates.
New release ‘18.04.6 LTS’ available.
Run ‘do-release-upgrade’ to upgrade to it.
*** /dev/mmcblk0p2 will be checked for errors at next reboot ***
Last login: Fri Jan 7 18:33:29 2022 from 10.10.10.209
root@CMAL-a788:~# kolibri manage migrate
/usr/lib/python3/dist-packages/kolibri/dist/tzlocal/unix.py:158: UserWarning: Ca n not find any timezone configuration, defaulting to UTC.
warnings.warn(‘Can not find any timezone configuration, defaulting to UTC.’)
INFO Running Kolibri with the following settings: kolibri.deployment.default .settings.base
Traceback (most recent call last):
File “/usr/bin/kolibri”, line 9, in
load_entry_point(‘kolibri==0.14.7’, ‘console_scripts’, ‘kolibri’)()
File “/usr/lib/python3/dist-packages/kolibri/dist/click/core.py”, line 764, in call
return self.main(*args, **kwargs)
File “/usr/lib/python3/dist-packages/kolibri/dist/click/core.py”, line 717, in main
rv = self.invoke(ctx)
File “/usr/lib/python3/dist-packages/kolibri/dist/click/core.py”, line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/usr/lib/python3/dist-packages/kolibri/utils/cli.py”, line 272, in invoke
initialize()
File “/usr/lib/python3/dist-packages/kolibri/utils/cli.py”, line 359, in initialize
conditional_backup(kolibri.version, version)
File “/usr/lib/python3/dist-packages/kolibri/utils/cli.py”, line 99, in conditional_backup
backup = dbbackup(version_file_contents)
File “/usr/lib/python3/dist-packages/kolibri/core/deviceadmin/utils.py”, line 111, in dbbackup
for line in db.connections[“default”].connection.iterdump():
File “/usr/lib/python3.5/sqlite3/dump.py”, line 55, in _iterdump
query_res = cu.execute(q)
sqlite3.DatabaseError: database disk image is malformed
root@CMAL-a788:~# ^C
Hi @gary.friesen - You will likely want to back up the entire .kolibri folder to be sure all content is saved. The database backup is only for running the migrate command. Content itself is in the other folders.
Another option provided in that post is to run
sqlite3 db.sqlite3 ".dump" | sqlite3 new_db.sqlite3
You would then need to remove the old db.sqlite3 and rename new_db.sqlite3 to db.sqlite3 and restart.
I have removed the Kolibri upgrade package until we resolve the issue as I suspect more users are having this problem when upgrading Kolibri. Some users may also be installing it by mistake. An older working Kolibri is better than breaking it trying to upgrade. Moving forward we want people to upgrade either via a completely new image restoration, or the zip upload utility that exists in RACHEL 3.1.2+. That gives a log of the output of the installation and more options for fixes than a simple download.
I ran “sqlite3 db.sqlite3 “.dump” | sqlite3 new_db.sqlite3” and it returned the prompt immediately (see below). Nothing appeared to happen. I searched the entire kolibri folder and could not find the new_db.sqlite3 file.
login as: root
root@10.10.10.10’s password:
Welcome to CMAL100
280 packages can be updated.
220 updates are security updates.
New release ‘18.04.6 LTS’ available.
Run ‘do-release-upgrade’ to upgrade to it.
*** /dev/mmcblk0p2 will be checked for errors at next reboot ***
Last login: Sun Jan 9 03:32:45 2022 from 10.10.10.209
root@CMAL-a788:~# sqlite3 db.sqlite3 “.dump” | sqlite3 new_db.sqlite3
root@CMAL-a788:~#
I backed up the entire Kolibri folder. Should I delete all the en-kolibri modules and try downloading it again from the Add Modules area? Or is there something else I should try first?
Did you run the command in the same directory as the db.sqlite file? You will have to be in /.data/.kolibri/ if it’s there.
Another option is to try "sudo rm -rf /.data/RACHEL/.kolibri/process_cache and then rebooting. This may also help
OK, we’re making progress! The tip from two days ago worked (using “dump” command issued from the correct folder). Kolibri is now working and it is upgraded to version 0.14.7. However, there are two db.sqlite3 files. The 310,272 file works fine but there is no content in it. The 485,120,000 file does not work, and this is the one that has all the content. Here is a screen shot of the folders and files in the ,kolibri folder:
Hi @gary.friesen - Partially good news!
When you have tried renaming the 485 db.sqlite3_back to db.sqlite3 and running, what is the error you get now with the 485 database? The same malformed issue?
When I try to run Kolibri with the 485 db.sqlite3 database this is what I see:
When I run the smaller db.sqlite3 database it allows me to login:
That just says whether Kolibri is running or not, but isn’t enough detail to know why it isn’t running. I assume the old db is still throwing the “malformed database” error and causes Kolibri to stop. Something was likely corrupted during Kolibri’s upgrade process.
You could try using the 485 db and deleting the process_cache folder in /.data/RACHEL/.kolibri, then rebooting. I don’t believe this is it, but I have seen it mentioned.
Another option would be to reinstall the old Kolibri 0.9 and then trying the database/content with that again.
Unfortunately that migrate/dump command seems like it’s not recovering any of the original data in that command. Was the content they created using Kolibri Studio Online or locally only? If it’s using Kolibri studio, I would just re-import the channels.
I deleted the process_cache folder, rebooted, and that did not work. It seems like we’re out of options. I would like to try reinstalling the old Kolibri 0.9. How do I do that?
The only way I can think to do that is to use a 3.0.2 recovery USB without deleting the hard drive. There is an option for this, but I will have to test to make sure I give you the right instructions. Please make sure to back up all folders to your PC from /.data/ on the device in the mean time just in case. I will get back to you with instructions as soon as possible.
Did they create their channels in Kolibri Studio online?
No, they had no internet access in South Sudan so they created their lessons using Kolibri content already available on the RACHEL (mostly KA-Lite).
Are there any other devices with the same lessons or is unique to this device?