13/Aug2021

Magento 2 in DTAP Environments

Larger companies usually have multiple people involved in a Magento 2 project, including several developers, project managers, internal customers, end customers and other steakholders.
All these people benefit from a properly maintained and properly running Magento 2 webshop.

To properly maintain the quality of a webshop there's multiple factors in place besides the development process itself, including but not limited to the proper use of version control, project managmeent, release management, unit testing, functional testing and more.

To facilitate this, companies usually use multiple environments to run the project on, which allows developers to efficiently develop code without others constantly being stopped in their tracks while code is still buggy or in active development.

This is where DTAP usually comes into play. This short blog is ment as a technical guide into how to quickly and effectively manage a DTAP environment for your Magento 2 projects.


What is DTAP?

 

DTAP is an abbreviation for "Development, Testing, Acceptation and Production" and resembles the 4 environments which the project takes to ensure new versions of are properly developed and tested without the end users and customers having to worry about unfinished results.


DTAP Best Practises

 

Combining Testing and Acceptation to remove overhead

In small to medium sized team, the Testing and Acceptation server is most likely combined into one environment. This is because there's usually not a party involved who would require a separate Acceptation environment. Therefor a D(TA)P environment is very often just 3 environments.

A situation where 4 environments is used, is when a project team tests on the Testing environment, while other steakholders test on Acceptation prior for a new version to make it to production.

 

Automating deployment to remove human error and security risks by access control

There are many tools and services which you can use to automate the deployment process. This process is called Continuous Deployment (usually in conjunction with Continuous Integration). We at GoGento use Buddy (https://buddy.works/) and we have worked with many other tools to automate our deployment process. When code is committed to version control and pushed, our system deploys this code to the right environments without developer interaction.

If you need more information about Continuous Deployment for your projects, you can always either contact us or Google terms you just learned.

If you do need to manually deploy, make sure you only let people on the servers which you fully trust. If you 

 

Environment Equality will reduce the risk of environment specific issues

Try to keep in mind that when you maintain code in any of the DTAP environments, it needs to be a representation of your production environment to ensure the flow to the next steps is as smooth as possible. Try to run the same versions of Linux, PHP, NGINX/Apache, MySQL, ElasticSearch, Varnish, Redis and other systems you may use in each environment.


Best Practises for Magento 2 in different environments

 

Development

  1. Use a database dump which you can reimport from production in case of failures. There will be many cases where a "setup:upgrade" command will run in a permanent database upgrade and having a backup to restore back to is certainly not a luxury.
  2. Disable email communications so that customers are not spammed with fake emails when you didnt realise you were sending them in the first place
  3. Set all your payment providers in testing / sandbox mode
  4.  Unset your Google Tag Manager information to prevent fake statistics
  5. Enable template hints
  6. Enable developer mode (bin/magento deploy:mode:set developer

 

An example deploy script for development

 

#!/bin/bash

bin/magento maintenance:enable
bin/magento setup:upgrade
bin/magento index:reindex
bin/magento cache:enable
bin/magento cache:clean
bin/magento maintenance:disable

 

Testing and Acceptation

  1. Just like in Development, use a database dump which you can restore to when testing is completed. Make sure to refresh this DB dump often so that your tests resemble the production environment.
  2. Either disable email communications or reroute them to a location where you catch them (ex. MailHog) to prevent customers from getting them.
  3. Set all your payment providers in testing / sandbox mode
  4.  Unset your Google Tag Manager information to prevent fake statistics
  5. Enable template hints
  6. Enable Production mode to ensure the enviroment is as close to production as possible (bin/magento deploy:mode:set production)
  7. When running static-content:deploy, ensure you add -j<amount of cpu cores in the system> to speed this process up drastically

An example deploy script for Testing and Acceptation

 

#!/bin/bash

bin/magento maintenance:enable
bin/magento deploy:mode:set production --skip-compilation
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy -j6 <space separated locales used in your store>
bin/magento index:reindex
bin/magento cache:enable
bin/magento cache:clean
bin/magento maintenance:disable

 

Production

  1. Obviously you want to use production mode here
  2. Use ElasticSearch for your indexes 
  3. When running static-content:deploy, ensure you add -j<amount of cpu cores in the system> to speed this process up drastically

 

An example deploy script for production

 

#!/bin/bash

bin/magento maintenance:enable
bin/magento deploy:mode:set production --skip-compilation
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy -j6 <space separated locales used in your store>
bin/magento index:reindex
bin/magento cache:enable
bin/magento cache:clean
bin/magento maintenance:disable

 

Bonus Script: Switching environments easily

When you work in these environments a lot, rapid development and rapid testing becomes a factor.
Store a recent database dump in your version control, for example, you can create a db folder in your project and store the SQL file there.

Store the example script below, or an variant of it into your project root. Then you can very easily switch (reset) your environments to the latest database dump

#!/bin/bash

if [ -z "$1" ]
then
        echo "Please specify environment: development, testing, acceptation";
        exit 1;
fi

if [ $1 == 'development' ]
then
        mysql --execute="DROP DATABASE yourproject"
        mysql --execute="CREATE DATABASE yourproject"
        mysql --execute="GRANT ALL PRIVILEGES ON yourproject.* TO [email protected]'%' IDENTIFIED BY 'yourproject'"
        mysql --database="yourproject" < db/db.sql
        ./bin/magento app:config:import
        ./bin/magento config:set web/unsecure/base_url http://dev.yourprojectdomain.com/
        ./bin/magento config:set web/secure/base_url https://dev.yourprojectdomain.com/
        ./bin/magento config:set admin/url/custom https://dev.yourprojectdomain.com/admin/
        ./bin/magento config:set weltpixel_googletagmanager/general/gtm_code ""
        ./bin/magento config:set weltpixel_googletagmanager/general/gtm_nonjs_code ""
        ./bin/magento config:set dev/debug/template_hints_storefront 1
        ./bin/magento config:set dev/debug/template_hints_blocks 1
        ./bin/magento config:set dev/debug/template_hints_storefront_show_with_parameter 1
        ./bin/magento deploy:mode:set developer
        ./bin/magento setup:upgrade
        ./bin/magento cache:clean
        chown -R www-data:www-data *
        ./bin/magento index:reindex
fi

if [  $1 == 'testing' || $1 == 'acceptation' ]
then
        mysql --execute="DROP DATABASE yourproject"
        mysql --execute="CREATE DATABASE yourproject"
        mysql --execute="GRANT ALL PRIVILEGES ON yourproject.* TO [email protected]'%' IDENTIFIED BY 'yourproject'"
        mysql --database="yourproject" < db/db.sql
        ./bin/magento app:config:import
        ./bin/magento config:set web/unsecure/base_url http://$1.yourprojectdomain.com/
        ./bin/magento config:set web/secure/base_url https://$1.yourprojectdomain.com/
        ./bin/magento config:set admin/url/custom https://$1.yourprojectdomain.com/admin/
        ./bin/magento config:set weltpixel_googletagmanager/general/gtm_code ""
        ./bin/magento config:set weltpixel_googletagmanager/general/gtm_nonjs_code ""
        ./bin/magento config:set dev/debug/template_hints_storefront 1
        ./bin/magento config:set dev/debug/template_hints_blocks 1
        ./bin/magento config:set dev/debug/template_hints_storefront_show_with_parameter 1
        ./bin/magento deploy:mode:set developer
        ./bin/magento setup:upgrade
        ./bin/magento cache:clean
        ./bin/magento index:reindex
fi