Add Kiwix .Zim file to RACHEL-Pi

Please note, RACHEL-Plus does not requires this process:

Before completing the following steps, you must install Kiwix-Serve on RACHEL-Pi. By default, English RACHEL-Pi does not include Kiwix, only Spanish and French RACHEL-Pi do.

  1. Download the zim files you’d like from kiwix: http://wiki.kiwix.org/wiki/Content

  2. Place the downloaded content into a module folder usually named by a language-descriptor format of two characters (e.g. en-wikipedia) which should have data/content/ and data/index/ subfolders inside it. (/var/www/modules/en-wikipedia)

  3. Add to the en-wikipedia folder an index.htmlf or rachel-index.php file to add navgivation to the kiwix content on the RACHEL homepage (/var/www/modules/en-wikipedia/rachel-index.php)

  4. Restart kiwix using the following command: perl /var/kiwix/bin/rachel-kiwix-start.pl (or reboot the machine) – that will automatically update the kiwix library to include any newly installed kiwix modules.

Has anyone actually gotten the Wikipedia full novid zim file to work?
When I try it says invalid header and will not load the zim file.
The smaller Simple English Wikipedia zim file loads just fine.
I verified the md5 sum on it and it did transfer correctly.

Apparently the newer zim files will not work with the included kiwix-serve binary in rachel pi. Have to use the newest kiwix-serve binary.

@Blueskies were you able to just get/install/update the kiwix to latest version? Just seeing if a need to redo an image or not. I used teh ES Pi image and removed teh ES wikipedia to install the PT wikipedia for a Brazile deployment and seem to have the same issue you ran into. The module shows but the search does not work and can not see any other that the search form page.

This is actually a really great point. I’m not sure that RACHEL-Pi in English has Kiwix installed at all. I’m not sure why it would, as I don’t believe it gets distributed with any Kiwix packages by default.

I am pretty sure your right. I know it is in the ES image and many of the servers I deploy are spanish countries but since not much content exists in portuguese, they get by on spanish and I add in the PT wiki and PT Khan.
**I failed to mention this one is for Brazil

To check if Kiwix is actually installed, if you go to http://10.10.10.10:81 does it show the Kiwix management page? Also, if you can SSH in or run commands, if you do “sudo systemctl status kiwix” it will say something like “unit kiwix.service could not be found.” if it is not installed.

In case it isn’t installed I’ve edited an installer script to just install the latest Kiwix-serve. These are the instructions.

  • Download kiwix.zip and unzip it,
  • Place the kiwix folder on the /boot drive of your MicroSD. if you plug it into a windows computer with a card reader it should show up.
  • Connect your pi to the internet.
  • Either SSH into the pi or plug a keyboard into the pi and connect it to a monitor.
  • Run command “cd /boot/kiwix”.
  • Run command “sudo ./kiwix.py”
  • Once it’s complete it should say installed successfully.
  • Delete the kiwix folder either in windows or with the command “sudo rm -rf /boot/kiwix”

Kiwix should be installed. If you can’t connect it to internet I can add the kiwix-serve package to the zip and make the installer run it locally for you.

If Kiwix still doesn’t work after adding zims, try rebooting or sorting modules in the admin interface and hitting save. This makes it look for zims in the modules directory. Make sure the module is not hidden in the sort page as this will cause it to be skiped and the zim file won’t be added.

If that doesn’t work, there is a bug that is fixed by going directly to the link for the zim that I’ve detailed a bit in this readme.. Once you’ve gone to the link and refresh the page in your browser the zim will show from then on. I hope one of these things helps.

James

I am having trouble updating .zim file. Kiwix server not run well when I update the .zim file downloaded from this link (http://download.kiwix.org/zim/wikipedia_my_all_maxi.zim).

Thanks

Hi @soehtet,

The problem is that the RACHEL-Pi images are running version 0.9.0 of kiwix-tools to be compatible with the .zim files from existing RACHEL modules that use separate indexing files. These are the ones on the servers. Newer zims don’t use separate index files. I am looking at ways around this so modules can be built automatically on the Pi when a new zim is downloaded and it can run the latest kiwix-tools version 3.1.2-3. This will be a separate RACHEL-Pi image and I’ll make it available when it’s ready.

James

I’m waiting your RACHEL-Pi image. Thank you!

1 Like

Hi @jamesk, I don’t know what zim files are, should I?

Hi @giakonda,

zim files are the format that Wikipedia releases their full copies of content as. In the past they were split into multiple files, for example en-wikipedia_2017.zima, en-wikipedia_2017.zimb, en-wikipedia_2017.zimc, etc. These files are now released as one single .zim file, so en-wikipedia_2017.zim.

Here’s a bit more info about how that affects RACHEL. There are a set of tools called kiwix-tools that RACHEL uses to serve wikipedia content ( kiwix-serve ) and manage the Kiwix library ( kiwix-manage ) by adding zims to a library.xml file. Zim files previously used separately generated index files for the search functionality. The latest versions of those tools no longer take a provided index anymore when adding zims to the library with kiwix-manage. Older zims ( .zima, zimb, etc ) can be added but since kiwix-tools doesn’t take the index, they won’t be indexed properly for search with the new kiwix-tools. So in the near future I will update kiwix-tools to the latest version and create a way to directly generate wikipedia modules for RACHEL based on a .zim file.

Hope that explains it
James

Hi @jamesk It does thank you. It will be some time before I could do anything like that though, unfortunately.

1 Like