This website uses cookies to improve your experience and deliver relevant information.

on Friday, 19 May 2017

Performing a swing-migration using Docker to update Mautic from 1.4 to 2.8 and update PHP versions

Recently we had a need to perform a ‘swing migration’ on a Mautic instance which was running 1.4 - we needed to migrate it to the latest version of Mautic, and also update PHP on the server.

Mautic has some in-built checks which are designed to prevent you from updating your Mautic version when there is an incompatible PHP version on the server. This means that if you are running Mautic 1.4, and you have a PHP version which is below the minimum required level for the next update, the only way to update Mautic is to upgrade your PHP version, and hope that Mautic will survive!

This wasn’t really an option for us in this instance, as our Mautic installation was in heavy production use, and the server had multiple other applications running which prevented us from updating PHP without extensive testing - so we needed to move it to our newer servers.

After much head scratching, our DevOps guru Mike came up with a strategy which would allow us to perform a swing migration - leaving the live environment untouched but allowing us to suck it down into a test environment with the same PHP version on the new server, and another environment with the new version of PHP, pointing at the same data.

The way we achieved this was through containerisation - in our case we use Docker containers.

Most of the resources we found online about using Mautic were related to spinning up a brand new Mautic instance, but in our case we had an active instance which had been in use for nearly two years, so this wasn’t really an option for us.

The scripts we created allow us to do the following:

  1. Set up the environments and ssh connections
  2. Spin up Docker containers for PHP, MySQL, Cron
  3. Suck down the current instance - files and database
  4. Dump the live database into a script
  5. Create a .env file as a ‘template’ for Docker
  6. Map the local filesystem into the Docker containers
  7. Map the scripts directory into the MySQL container
  8. Fire up the MySQL database container which imports the database scripts
  9. Run script to build Mautic, import database and configure settings

Server architecture

It’s probably helpful to start with an overview of the server architecture we are working with.

Our architecture is configured to leverage containerisation, using the Docker API extensively to build containers, update configuration files and keep our proxy (Nginx) up to date with what’s going on in our containers.

Our current setup is to have a container for each application - so in the case of Mautic we have the following containers:

  • PHP 7.x
  • MySQL
  • Cut-down minimal Linux container (for running Cron tasks)
  • Mailhog (mail server for internal testing)

We have a naming convention such that containers are given the name of the client, followed by the application, and then the script name.

For example:

  • viryagroup-mautic-php7
  • viryagroup-mautic-mysql55
  • viryagroup-mautic-cron
  • viryagroup-mautic-mailhog

This means that we can easily identify the purpose of each container and reduces the risk of modifying the wrong containers.

We host our file structure locally on the server hardware for live instances rather than containerising the file systems.  We do things slightly differently for our demo instances which we will explore in a future blog post.

Getting started

The first task is to create a directory for the application - in this case Mautic - within which we create three sub-folders: -

  • viryagroup-mautic 
    • web_data
    • db_data
    • db_scripts

As a brief explanation of the structure:

viryagroup-mautic

This is the naming convention we use in the directory structure - the name of the client, followed by the application.

web_data

This will be where the filesystem for the application will reside

db_data

This is where the database filesystem for the application will reside

db_scripts

This is where we locate any scripts that are required.

Setting up the environments

We use virtualhosts to set up each domain, so these need to be set up within your instance as appropriate.

Downloading the current instance

Once the directories are created, we import the current instance into the folder web_data.

As an example, we would pull from: /var/www/html/mautic into: /viryagroup-mautic/web_data

Download a database dump

Once the files are imported, the next task is to import the database. We make a database dump into /viryagroup-mautic/web_data which will be used to resurrect the data once the containers are built.

Create .env file

Next, we create the .env file based on the configuration from the files downloaded from the live instance. The .env file includes all the configuration settings required to reinstate the database.

Map file system

Once the .env file is created, we map the filesystem directory for the virtual host, so that it points at the appropriate location. In this case, we would map: /var/www/html into: /viryagroup-mautic/web_data

Build required containers

PHP containers

At this point we need to build two Docker containers - as we need to spin up our Mautic instance in the current PHP version, run the update, and then check that it works OK with the latest PHP version.

In our case, we have the following containers:

  • viryagroup-mautic-php53
  • viryagroup-mautic-php7

MySQL container

At this point we were not doing any major version changes with MySQL, so we only need to build one MySQL container: viryagroup-mautic-MySQL

Mailhog container

Mailhog is a great tool for testing, especially if you want to ensure no mail goes out from development areas or until you are ready. This is a simple one really - spin up the container and configure: viryagroup-mautic-mailhog

Linux container

As Mautic relies heavily on cron jobs, after researching the different ways to manage this in a containerised environment we decided to build a simple, cut-back Linux container to manage cron jobs on a per-account basis. We build it thus: viryagroup-mautic-cron

Once all the containers are set up and running, we can kick off with preparing the migration.

What we are wanting to achieve is:

  • Database containing current live database PHP 5.3.x container pointed at viryagroup-mautic-web_data
  • PHP 7.x container pointed at viryagroup-mautic-web_data

To kick things off, we map the MySQL container to viryagroup-mautic-db_data and kick-start the import by loading up the database dump within viryagroup-mautic-db_scripts.

This builds the database from the live dataset, which will be connected to by the PHP containers.

Build Mautic

Our buildMautic.sh script then sets about building Mautic and running the various tasks that are needed.

Updating Mautic

We now have the ability to fire up Mautic using PHP 5.3, and PHP 7.x - using the same fileset and database.

What we need to do is:

  • Check Mautic in the PHP 5.3 environment, ensure everything is as expected
  • Run Mautic update to latest version at command line (as we were working with a large data set, so command line update is the recommended route)
  • Check Mautic in the PHP 7.x environment, ensure everything is as expected

One of the huge benefits we found through using containerisation and the scripts we wrote was that when inevitably we hit some problems, within a few clicks we could nuke the instance, pull it down again, rebuild, and try again. All of this could be done without touching the live dataset.

Bumps we hit along the way

We encountered multiple issues with regards to zero-dated date/time fields which needed fixing before we could complete the migration - these were flagged up in logs when we tried to update. We also hit a few snags with foreign characters, fields with invalid data (a historical data import had thrown every job one person had held through their entire life as a pipe-separated list in the ‘role’ field!  Oops!).

Once we had achieved a working installation on the latest version of PHP and Mautic, we were able to apply and test other improvements such as implementing SSL, before we pushed the Mautic instance live. This environment then became ‘live’ and we were able to use the same processes to clone down and keep in sync a development and staging area.


We hope this has been helpful for people who might need to be considering similar swing-migrations - if you need some help with the containerisation and scripts, drop us a line.  Comments or questions? Leave a message below!

 

We support shared hosting, VPS and dedicated servers Host with us today!

Virya Group provides a range of solutions to your technology needs