Live deployment, ci, drupal
•
9 likes•3,432 views
1. The document discusses setting up a continuous integration workflow for Drupal projects using tools like Jenkins, Drush, and Vagrant. 2. It identifies problems with current development practices like code being merged without testing and different environments between dev and production. 3. The workflow proposed uses scripts to automate rebuilding development and production environments from source control, running tests, and deploying code.
Report
Share
Live deployment, ci, drupal
- 1. Live Deployment. Continuous Integration. Drupal. Andriy Podanenko, @podarok, ProPeople.
- 2. Technologies used 1. sh/bash scripting 2. drush/drupal/php 3. Jenkins server master/slave CI server. 4. mysql advanced configuration 5. vagrant virtualization, puppet scripting (ruby) 6. java (if You need to extend basic Jenkins’es plugins) (optional) 7. apache/nginx/php5-fpm/mysql (Full LAMP/MAMP stack) 8. phantomjs/javascript scripting 9. git/GitHub 10. code sniffer stack (php codesniffer, jshint, scss lint, twig lint etcetera)
- 3. The problems ● Code merged to *master without real testing ● Configs from dev site do not pushed to stage ● Different dev/stage/prod server environments ● Code review is not comprehensive. ● Visual regression needs a lot of love. ● Deploy is a horror. ● Client make changes that brake upgrade path. ● Update/upgrade path not tested.
- 4. Demo/Live content. ● drush si build_profile_name... ● rebuild.sh script with all steps for getting dev build ● After first release for content management ● liverebuild.sh script for getting live DB from hosting server to meet latest content/config changes. ● helps with visual regression testing when there is frontend’s pull request for review and content/config changed.
- 5. rebuild.sh script example(in repo!!!) #!/bin/sh service memcached restart chmod +w sites/default/settings.php && rm -rf sites/default/settings.php drush -vy si sitename --db-url=mysql://drupal:drupal@127.0.0.1:/drupal --account-name=admin --account-pass=pass chmod +w sites/default/settings.php && rm -rf sites/default/settings.php && cp sites/default/settings_devel.php sites/default/settings.php cd sites/all/modules/contrib/guzzle composer update cd ../../../../../ pwd cd sites/all/modules/custom/salespush composer update cd ../../../../../ drush some_custom_command drush en devel -y drush cc all
- 6. liverebuild.sh script example #!/bin/sh rm -rf livedb.sql* wget http://URL_TO/livedb.sql.gz gunzip -f livedb.sql.gz mysql --force -e "drop database IF EXISTS livedrupal;create database IF NOT EXISTS livedrupal;use livedrupal;source livedb.sql;" rm -rf sites/default/settings.php cp sites/default/settings_live.php sites/default/settings.php drush -y updatedb drush upwd admin --password=dEvPasS create-solr-instance inst_x 7 create-solr-instance inst_i 7 drush en -y live_solr_settings drush cc all drush -dvy en migrate_master drush -dvy updatedb
- 7. MySQL fast restoring from dump SET foreign_key_checks = 0;SET UNIQUE_CHECKS = 0; SET AUTOCOMMIT = 0; source dbdump.sql; SET foreign_key_checks = 1;SET UNIQUE_CHECKS = 1; COMMIT; ------------------------------------------------------------------------------------ innodb_file_per_table = 1 tmp_table_size = 160M max_heap_table_size = 160M innodb_file_format = Barracuda innodb_file_format_max = Barracuda innodb_flush_log_at_trx_commit = 2 query_cache_size = 160M table_cache = 800 innodb_buffer_pool_size = 900M
- 8. GitHub ads (PR matters) - Jenkins - GitHub PR builder plugin (triggers a build job in Jenkins for a PR’s hash) - create a build from scratch using cloned repo from PR, demo content or even live db if any. - Create a comment at PR’s thread with links to build/job results
- 9. How it looks like in PR’s comment
- 10. Sniffers hell (Code Review matters)... ● PHP CodeSniffer ● JSHint ● SCSS lint ● yslow.js + phantomjs... ● Drupal SimpleTest ● PHPUnit tests ● Performance tests (custom made)
- 11. PHP CodeSniffer log example FILE: .../smartling.admin.inc -------------------------------------------------------------------------------- FOUND 3 ERRORS AND 7 WARNINGS AFFECTING 10 LINES -------------------------------------------------------------------------------- 20 | ERROR | global variables should start with a single underscore | | followed by the module and another underscore 346 | WARNING | Unused variable $s_locale. 451 | ERROR | global variables should start with a single underscore | | followed by the module and another underscore 464 | WARNING | Unused variable $need_fix. 618 | WARNING | Do not use the raw $form_state['input'], use | | $form_state['values'] instead where possible 740 | ERROR | global variables should start with a single underscore | | followed by the module and another underscore 762 | WARNING | Unused variable $need_fix. --------------------------------------------------------------------------------
- 12. JSHint log example demo.js: line 5, col 5, Missing "use strict" statement. demo.js: line 5, col 26, 'hideUnusedFn' is defined but never used. demo.js: line 26, col 25, 'hideUnused3' is defined but never used. demo.js: line 10, col 27, 'unusedVariable' is defined but never used. demo.js: line 12, col 23, 'hideUnused' is defined but never used. demo.js: line 9, col 43, 'unusedArg' is defined but never used. demo.js: line 17, col 25, 'anotherUnusedArg' is defined but never used. demo.js: line 16, col 27, 'unusedArgAfterUsed' is defined but never used. demo.js: line 23, col 26, 'anotherUnusedArg2' is defined but never used. demo.js: line 1, col 34, 'unusedGlobalVar' is defined but never used. 10 errors
- 13. SCSS lint log example _base.scss:568 [W] Class `footer-article-icon-Case-study` in selector should be written in all lowercase as `footer- article-icon-case-study` _base.scss:725 [W] Merge rule `.pane-block` with rule on line 612 _base.scss:971 [W] Merge rule `.hp-marquee` with rule on line 966 _base.scss:982 [W] Selector should have depth of applicability no greater than 3, but was 4 _colors.scss:14 [W] Color `#666666` should be written as `#666` _comments.scss:10 [W] `border: 0;` is preferred over `border: none;` _comments.scss:171 [W] URLs should be enclosed in quotes _reset.scss:25 [W] Use `//` comments everywhere _reset.scss:26 [W] Each selector in a comma sequence should be on its own line _slider.scss:17 [W] Properties should be sorted in order, with vendor-prefixed extensions before the standardized CSS property
- 14. yslow.js graph example $ phantomjs yslow.js --info grade --format tap --threshold '{"yminify": 90}' example.com TAP version 13 1..24 ok 1 B (88) overall score not ok 2 C (72) ynumreq: Make fewer HTTP requests ok 3 C (70) ycdn: Use a Content Delivery Network (CDN) ok 4 A (100) yemptysrc: Avoid empty src or href not ok 5 F (12) yexpires: Add Expires headers ok 6 A (100) ycompress: Compress components with gzip ok 7 A (100) ycsstop: Put CSS at top ok 8 A (100) yjsbottom: Put JavaScript at bottom ok 9 A (100) yexpressions: Avoid CSS expressions ok 10 N/A (-1) yexternal: Make JavaScript and CSS external # SKIP score N/A not ok 11 C (70) ydns: Reduce DNS lookups ok 12 A (90) yminify: Minify JavaScript and CSS ok 13 A (100) yredirects: Avoid URL redirects ok 14 A (100) ydupes: Remove duplicate JavaScript and CSS ok 15 A (100) yetags: Configure entity tags (ETags) ok 16 A (100) yxhr: Make AJAX cacheable ok 17 A (100) yxhrmethod: Use GET for AJAX requests ok 18 A (100) ymindom: Reduce the number of DOM element
- 15. PHPUnit/SimpleTest tests example ./runtests.sh Configuration read from /var/www/pr/*****/build717/app/phpunit.xml.dist F..... Time: 219 ms, Memory: 13.00Mb There was 1 failure: 1) HPBundleAuthBundleTestsControllerDefaultControllerTest::testIndex Failed asserting that two strings are equal. --- Expected +++ Actual @@ @@ -'Authentication required.' +'"Argument 2 passed to SymfonyBundleTwigBundleExtensionAssetsExtension::__construct() must be an instance of SymfonyComponentRoutingRequestContext, none given, called in /var/www/pr/*****/build717/app/cache/test/appTestDebugProjectContainer.php on line 3496 and defined"' /var/www/pr/***/build717/src/**/Bundle/AuthBundle/Tests/Controller/DefaultControllerTest.php:21 FAILURES! Tests: 6, Assertions: 10, Failures: 1.
- 17. Full Demo
- 18. Useful links ● https://www.drupal.org/coding-standards ● http://www.squizlabs.com/php-codesniffer ● http://jshint.org/ ● https://github.com/causes/scss-lint ● https://github.com/asm89/twig-lint ● https://github.com/jenkinsci/ghprb-plugin ● https://github.com/podarok/ghprb-plugin (Extended version) ● http://puphpet.drupal.ua/ (Custom version for Drupal devs) ● http://www.vagrantup.com/ ● http://drush.ws/
- 19. Future plans Add twig templates linting Visual Regression tool Performance tests
- 20. Thank You Questions? Andriy Podanenko, Software Architect, DevOp. http://dgo.to/@podarok http://druler.com/contact http://wearepropeople.com