Daniel Lange: Upgrading Limesurvey with (near) zero downtime
Limesurvey is an online survey tool. It is very powerful and commonly used in academic environments because it is Free Software (GPLv2+), allows for local installations protecting the data of participants and allowing to comply with data protection regulations. This also means there are typically no load-balanced multi-server szenarios with HA databases. But simple VMs where Limesurvey runs and needs upgrading in place.
There’s an LTS branch (currently 3.x) and a stable branch (currently 4.x). There’s also a 2.06 LTS branch that is restricted to paying customers. The main developers behind Limesurvey offer many services from template design to custom development to support to hosting (“Cloud”, “Limesurvey Pro”). Unfortunately they also charge for easy updates called “ComfortUpdate” (currently 39€ for three months) and the manual process is made a bit cumbersome to make the “ComfortUpdate” offer more attractive.
Due to Limesurvey being an old code base and UI elements not being clearly separated, most serious use cases will end up patching files and symlinking logos around template directories. That conflicts a bit with the opaque “ComfortUpdate” process where you push a button and then magic happens. Or you have downtime and a recovery case while surveys are running.
If you do not intend to use the “ComfortUpdate” offering, you can prevent Limesurvey from connecting to http://comfortupdate.limesurvey.org
daily by adding the updatable
stanza as in line 14 to limesurvey/application/config/config.php
:
-
[…]
-
// Use the following config variable to set modified optional settings copied from config-defaults.php
-
‘config’=>array(
-
// debug: Set this to 1 if you are looking for errors. If you still get no errors after enabling this
-
// then please check your error-logs – either in your hosting provider admin panel or in some /logs directory
-
// on your webspace.
-
// LimeSurvey developers: Set this to 2 to additionally display STRICT PHP error messages and get full access to standard templates
-
‘debug’=>0,
-
‘debugsql’=>0, // Set this to 1 to enanble sql logging, only active when debug = 2
-
// Mysql database engine (INNODB|MYISAM):
-
‘mysqlEngine’ => ‘MYISAM’
-
, // Update default LimeSurvey config here
-
‘updatable’ => false,
-
)
-
);
The comma on line 13 is placed like that in the current default limesurvey config.php
, don’t let yourself get confused.
Every item in a php array must end with a comma. It can be on the next line.
The basic principle of low risk, near-zero downtime, in-place upgrades is:
- Create a diff between the current release and the target release
- Inspect the diff
- Make backups of the application webroot
- Patch a copy of the application in-place
- (optional) stop the web server
- Make a backup of the production database
- Move the patched application to the production webroot
- (if 5) Start the webserver
- Upgrade the database (if needed)
- Check the application
So, in detail:
1. Create a diff between the current release and the target release
For Limesurvey it is essential that you keep a version of the currently running release before you have applied your templates and other customizations.
If you missed that step or are doing the upgrade like this the first time, the Limesurvey github releases page is a useful source.
In this example we are upgrading from release 4.2.6 to 4.3.0. So we skipped 4.2.7.
I assume you’re working in a /root/install
directory for this step. You should definitely work outside the webroot.
You should have a 4.2.6
directory containing the unpacked Limesurvey 4.2.6 zip or tarball.
Download the 4.3.0 release, mkdir 4.3.0
and (for example) unzip -d 4.3.0/ limesurvey4.3.0+200616.zip
.
Create the diff with
# diff -N -u -r 4.2.6/ 4.3.0/ > mydiff
-N
allows to create new files as a diff from /dev/zero and to remove files similarly
-u
creates a “unified diff”, so a diff with some context that will allow to apply patches even to files that you changed locally
-r
makes the diff recurse through subdirectories
2. Inspect the diff
As a first step look at the size of the diff:
$ wc -l mydiff
49896 mydiff
That looks reasonable, it’s a web app .
If you get 8.5m lines, you have managed to get one directory wrong and the whole app is removed and re-created.
Now read through the diff and inspect for changes that may collide with your local modifications or that look fishy.
A list of the files that got created or removed may be useful to inspect at minimum:
$ diff <(cd 4.2.6; find . -type f) <(cd 4.3.0; find . -type f)
2185,2188c2185,2188
< ./limesurvey/third_party/jquery/jquery-migrate-3.1.0.js
< ./limesurvey/third_party/jquery/jquery-3.4.1.min.js
< ./limesurvey/third_party/jquery/jquery-3.4.1.js
< ./limesurvey/third_party/jquery/jquery-migrate-3.1.0.min.js
—
> ./limesurvey/third_party/jquery/jquery-3.5.1.min.js
> ./limesurvey/third_party/jquery/jquery-3.5.1.js
> ./limesurvey/third_party/jquery/jquery-migrate-3.3.0.min.js
> ./limesurvey/third_party/jquery/jquery-migrate-3.3.0.js
10014a10015,10016
> ./limesurvey/locale/ti/index.html
> ./limesurvey/locale/ti/ti.mo
10058a10061
> ./limesurvey/tests/data/surveys/survey_archive_265351_listParticipants.lsa
10079a10083
> ./limesurvey/tests/data/surveys/survey_archive_821351.lsa
10115a10120
> ./limesurvey/tests/unit/helpers/QuexmlPDFTest.php
10120a10126
> ./limesurvey/tests/unit/helpers/RemoteControlListParticipantsTest.php
12241d12246
< ./limesurvey/application/controllers/admin/questionedit.php
12277a12283
> ./limesurvey/application/controllers/QuestionEditorController.php
12835,12836d12840
< ./limesurvey/application/views/admin/survey/Question2/view.php
< ./limesurvey/application/views/admin/survey/Question2/_jsVariables.php
13414a13419,13420
> ./limesurvey/application/views/questionEditor/view.php
> ./limesurvey/application/views/questionEditor/_jsVariables.php
13438a13445
> ./limesurvey/application/views/layouts/title_bar.php
13442a13450,13451
> ./limesurvey/application/views/layouts/sidemenu.php
> ./limesurvey/application/views/layouts/layout_questioneditor.php
3. Make backups of the application webroot
cd /var/www
cp -ra limesurvey limesurvey_backup_${date “+%y%m%d”} # create backup
cp -ra limesurvey limesurvey_new # create tree to patch
4. Patch a copy of the application in-place
cd limesurvey_new
patch –verbose -N -p 2 < /root/install/mydiff
The --verbose
is important so you see deleted files. -N
forces a forward-patch as everything else would not make sense.
-p 2
strips two directories off the beginning of the paths in the diff. That makes sense as 4.3.0/limesurvey
is not needed when you have limesurvey_new
as your working directory.
The output will look like this:
$ patch –verbose -N -p 2 < /root/install/mydiff
Hmm… Looks like a unified diff to me…
The text leading up to this was:
————————–
|diff -N -u -r 4.2.6/limesurvey/application/commands/DemomodeCommand.php 4.3.0/limesurvey/application/commands/DemomodeCommand.php
|— 4.2.6/limesurvey/application/commands/DemomodeCommand.php 2020-06-02 14: 10: 32.000000000 +0200
|+++ 4.3.0/limesurvey/application/commands/DemomodeCommand.php 2020-06-16 08: 09: 50.000000000 +0200
————————–
patching file application/commands/DemomodeCommand.php
Using Plan A…
Hunk #1 succeeded at 104.
Hmm… The next patch looks like a unified diff to me…
The text leading up to this was:
————————–
|diff -N -u -r 4.2.6/limesurvey/application/config/config-defaults.php 4.3.0/limesurvey/application/config/config-defaults.php
|— 4.2.6/limesurvey/application/config/config-defaults.php 2020-06-02 14: 10: 32.000000000 +0200
|+++ 4.3.0/limesurvey/application/config/config-defaults.php 2020-06-16 08: 09: 50.000000000 +0200
————————–
patching file application/config/config-defaults.php
Using Plan A…
Hunk #1 succeeded at 97.
Hunk #2 succeeded at 417.
Hunk #3 succeeded at 594.
[..]
Hmm… The next patch looks like a unified diff to me…
The text leading up to this was:
————————–
|diff -N -u -r 4.2.6/limesurvey/third_party/jquery/jquery-migrate-3.3.0.min.js 4.3.0/limesurvey/third_party/jquery/jquery-migrate-3.3.0.min.js
|— 4.2.6/limesurvey/third_party/jquery/jquery-migrate-3.3.0.min.js 1970-01-01 01: 00: 00.000000000 +0100
|+++ 4.3.0/limesurvey/third_party/jquery/jquery-migrate-3.3.0.min.js 2020-06-16 08: 09: 50.000000000 +0200
————————–
patching file third_party/jquery/jquery-migrate-3.3.0.min.js
Using Plan A…
Hunk #1 succeeded at 1.
done
Removing file application/controllers/admin/questionedit.php
Removing file application/views/admin/survey/Question2/_jsVariables.php
Removing file application/views/admin/survey/Question2/view.php
Removed empty directory application/views/admin/survey/Question2
Removing file third_party/jquery/jquery-3.4.1.js
Removing file third_party/jquery/jquery-3.4.1.min.js
Removing file third_party/jquery/jquery-migrate-3.1.0.js
Removing file third_party/jquery/jquery-migrate-3.1.0.min.js
If you see any hunks rejected (.rej
files created), find the reason and fix it before continuing.
5. (optional) stop the web server
If you run a stupid webserver, a complex setup, external opcode caches or just have high load (survey writes likely), you may want to stop the httpd now.
For “normal loads” (a few hundred answers to surveys per day) and when using Apache you can typically just move the webroot.
In any case even with stopping the httpd, the downtime should be very short.
6. Make a backup of the production database
Make a backup of the production database. In our case this is as simple as calling backup-mysql
.
If you don’t have a specific way to backup your database
mysqldump --quick --extended-insert --skip-comments limesurvey | gzip -9 > /root/install/limesurvey_database_${date "+%y%m%d"}.sql.gz
should be a good start.
7. Move the patched application to the production webroot
mv limesurvey limesurvey_old && mv limesurvey_new limesurvey
This will overwrite the old webroot and is two fast atomic operation on sane filesystems (e.g. ext4), so this is rather safe™ to do.
You can as well patch the production webroot in place. Esp. if you have stopped the httpd. You should have made a backup in step 3. You did, didn’t you?
Note: If you want a fully atomic replacement of the application webroot, you have to use symlinks for the old and new tree and use mv -T
, see Richard Crowley’s Things UNIX can do atomically for a discussion.
8. (if 5) Start the webserver
If you stopped the webserver, start it again.
9. Upgrade the database (if needed)
Depending on whether the upgrade needed database schema upgrades, you want to inspect the Admin UI at:
https://limesurvey.example.org/admin
The 4.2.6 -> 4.3.0 upgrade does not need any database changes.
10. Check the application
As a good last step – as always – take a critical look at the application whether everything runs as it should.
Are the images loading, can you do a test survey?
If so, all fine and until the next upgrade.