How To: Setting up your continuous integration server for PHP projects...

STATUS: WORK IN PROGRESS

To make short things shorter: this gonna be a long post.

In this post want to describe how to set up a continuous integration server or environment. in the same way i set it up for myself, so this post ist also (maybe mainly) a reminder to myself.

Actually, there are two widely use CI implementations for PHP projects used:

• phpUnderControl, an extension for cruise control
• jenkins, the open source fork moved out of formally known hudson ci server

Both implementations are java based systems, which brings in the mainly functionality of a build system for CI.
More details should be taken from the project websites.

I'm going to describe the set up with both implementations. Why? You could run only the one of them you need, or both if you think it's cool. The other point is, the decision which one you prefer should not depend on how easy one solution is to install. Use the one which best fits your needs. So both are described here, test them, and give feedback to the maintainers if you are happy or unhappy with it.

The CI Server covers the following build and software tools:

• phpunit
• php_codebrowser
• phploc
• phpcpd
• selenium
• php_depend
• php_codesniffer
• php_pmd
• phpdcd

I've tried to use headlines for each part. Some parts are used for both implementation, some are not. These parts have the used implementation in the headline.

For usual, i install optional software in the /opt directory. Replace this directory with your own if you prefer another directory, for example /usr/local

1. The requirements
First, i've set up a System with actually debian version (squeeze). I'm alwys installing debian from the smaller netboot.iso and install all packagaes via aptitude, not via the install dialogs, where it asks if you plan a web server, dns server and so on.
(Just because i just had some troubles with these pre defined package sets in history. Thats why i'm reloading apache configuration changes always with restart instead of reload, too. If you want to know why, ask for it.)
For now, squeeze is the stable debian release.
After installation always my first packages i'm going to install are less, wget, ssh.

2. Who knows the Java and sex practise comparison?
For both implementations, we need Java. the most uncomplicated Java implementation on my way as a developer isbuild by Sun. So i suggest to use the Sun Java implementation. But this is only my opinion. If you think different, its your way. This how to covers my way.
To take advantage of debians packages system, we change the used repositories.
Change your /etc/apt/sources.list as follows:

sudo vi /etc/apt/sources.list

# deb http://ftp.de.debian.org/debian/ squeeze main

deb http://ftp.de.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.de.debian.org/debian/ squeeze main contrib non-free

deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free

# squeeze-updates, previously known as 'volatile'
deb http://ftp.de.debian.org/debian/ squeeze-updates main contrib non-free
deb-src http://ftp.de.debian.org/debian/ squeeze-updates main contrib non-free

Fire

sudo aptitude update

to bring your package lists up to date.

3. Anything else?
There are now some packages which we need to install.

sudo aptitude install sun-java6-jre ant ant-optional rake php-pear php5-dev make php5-curl php5-imagick sun-java6-jdk subversion subversion-tools postgresql mysql-server php5 php5-mysql php5-pgsql php5-sqlite git unzip

- sun-java6-jre and sun-java6-jdk are required for jenkins and cruisecontrol and also for selenium.
- ant ant-optional are used for the build process of a project and also selenium startup
- rake there is another way to start up selenium with rake
- php-pear php5-dev make php5-curl php5-imagick php5 are required for installing xdebug via pecl and the PHP quality assurance tools from pear
- subversion subversion-tools git to checkout project sources PHP quality assurance tools
- php5-mysql php5-pgsql php5-sqlite postgresql mysql-server are optional for project for example

4. [phpUnderControl] Selenium Grid standalone
If you plan to take advantage of frontend tests via selenium (and you should do it), we use selenium grid. With grid, we can manage multiple browser on multiple clients. This speed up tests a lot.

cd /opt
wget http://release.seleniumhq.org/selenium-grid/selenium-grid-1.0.8-bin.zip

unzip selenium-grid-1.0.8-bin.zip

cd selenium-grid-1.0.8

ant sanity-check

You can test the functionality by starting a hub:

ant launch-hub

If no error occures, stop selenium with ctrl^c. With rake you can easily start selenium grid as deamon in background. You need a pid subdirectory in the selenium directory.

mkdir pid

Start the daemon with

rake hub:start BACKGROUND=true

You can stop the deamon with a post to the selenium lifecycle manager

wget -q -t1 --post-data action=shutdown http://localhost:4444/lifecycle-manager

If you need to pass arguments to selenium, you can pass them to the rake start command

rake hub:start BACKGROUND=true SELENIUM_ARGS="-multiWindow -debug"

Both, start and stop command, can easily be taken to configura a start--stop script in /etc/init.d/.

vi /etc/init.d/selenium-grid

#!/bin/bash

case "$1" in
  start)
    rake hub:start BACKGROUND=true SELENIUM_ARGS="-multiWindow -debug"
    ;;
  stop)
    wget -q -t1 --post-data action=shutdown http://localhost:4444/lifecycle-manager
    ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop}" >&2
    exit 3
    ;;
esac

exit 0

chmod +x /etc/init.d/selenium-grid

Thats it. Until now, we have a selenium hub, but the test should run on remote controls. Instructions for remote controls are described later.

The concepts of selenium grid are explained on the projects website http://selenium-grid.seleniumhq.org/

5. The PHP tools
Currently, the most used PHP tools for quality assurance by myself are

• Xdebug http://xdebug.org/ (thanks to Derick Rethans)
• PHPUnit http://www.phpunit.de/manual/current/en/index.html and https://github.com/sebastianbergmann/phpunit/ (thanks to Sebastian Bergmann)
• PHP_Depend http://pdepend.org/ (thanks to Manuel Pichler)
• PHPMD http://phpmd.org/ (thanks to Manuel Pichler)
• phploc https://github.com/sebastianbergmann/phploc (thanks to Sebastian Bergmann)
• PHP_CodeSniffer http://pear.php.net/package/PHP_CodeSniffer/ (thanks to Greg Sherwood)
• phpcpd https://github.com/sebastianbergmann/phpcpd (thanks to Sebastian Bergmann)
• phpdcd https://github.com/sebastianbergmann/phpdcd (thanks to Sebastian Bergmann)
• PHP_CodeBrowser https://github.com/mayflowergmbh/PHP_CodeBrowser (thanks to Sebastian Bergmann and Mayflower GmbH)

Except Xdebug, all tools can be installed with pear.

sudo pecl install xdebug

sudo pear channel-discover pear.phpunit.de

sudo pear channel-discover components.ez.no

sudo pear channel-discover pear.symfony-project.com

sudo pear channel-discover pear.phpmd.org

sudo pear channel-discover pear.pdepend.org

sudo pear upgrade-all

sudo pear install --alldeps phpunit/PHPUnit

sudo pear install pdepend/PHP_Depend

sudo pear install phpmd/PHP_PMD

sudo pear install phpunit/phploc

sudo pear install PHP_CodeSniffer

sudo pear install phpunit/phpcpd

If you want to know which packages are available on a channel, try pear remote-list -c phpunit for example.

6. [phpUnderControl] Never gonna give you up...
The installation of phpUnderControl works easy with the pear installer:

sudo pear channel-discover pear.phpundercontrol.org

sudo pear install --alldeps phpuc/phpUnderControl-beta

PhpUnderControl is an extension for Cruise Control (current latest version is 2.8.4). So the next step:

sudo cd /opt

sudo wget http://downloads.sourceforge.net/project/cruisecontrol/CruiseControl/2.8.4/cruisecontrol-bin-2.8.4.zip

sudo unzip cruisecontrol-bin-2.8.4.zip

sudo mv cruisecontrol-bin-2.8.4 cruisecontrol

sudo phpuc install /opt/cruisecontrol

sudo useradd cruisecontrol

sudo chown -R cruisecontrol cruisecontrol

You can now start your Cruise Control and phpUnderControl instance with

sudo /opt/cruisecontrol/cruisecontrol.sh

and stop the instance with

wget http://localhost:8000/invoke?operation=halt&objectname=CruiseControl+Manager%3Aid%3Dunique

There is an example of a startup script I often use. I changed the port cruise control runs on to get no conflicts with jenkins later:

vi /etc/init.d/cruisecontrol

#!/bin/sh
#content of /opt/cruisecontrol/init script
# chkconfig: 345 99 05
# description: CruiseControl build loop (see /home/tools)

# CruiseControl Unix Startup Script Version 2.1
#
# based on http://confluence.public.thoughtworks.org/display/CC/RunningCruiseControlFromUnixInit
# adapted for multiple projects
# also based on the file attached to the above page created by Jerome Lacoste

#
# CruiseControl startup: Startup and kill script for Cruise Control
#


###################################################################################################
# USER CONFIGURATION
#
# Fill in these values for your Cruise Control setup

# What user will Cruise Control run as?  The user will need permission to write and modify files
# in the next entries.
CC_USER=cruisecontrol


# Where is the CC startup script located?
CC_INSTALL_DIR=/opt/cruisecontrol

# In what directory is the config.xml file located for CC?
# default: CC_WORK_DIR=$CC_INSTALL_DIR
CC_WORK_DIR=$CC_INSTALL_DIR

# Where will the cruisecontrol.log file be located?
# default: CC_LOGFILE_DIR=$CC_INSTALL_DIR
CC_LOGFILE_DIR=$CC_INSTALL_DIR


#######################
# ENVIRONMENT ADDITIONS

# Add environement variables here that are needed by your build.
# example:
export JAVA_HOME=/usr
#
# or like this for local variables ONLY used in this file:
#  JAVA_HOME=/usr/local/java


# Add path to additional executables needed for project build.  See PATH entry below for base config.
# No additional action taken when blank.
PATH_ADDITIONS=



##############################
# CRUISE CONTROL PORT SETTINGS

# Port for Jetty reporting application.  You can access it by going to http://localhost:8080
# default CC_WEBPORT=8080
CC_WEBPORT=8081

# JMX port for webapp and Java Management eXtensions (JMX).  You can access it by going to http://localhost:8080
# Change only if this port is in use as the webapp will also need modification.
# default CC_JMXPORT=8082
CC_JMXPORT=8082

# RMI port for control via Java's Remote Management Interface (RMI)
# Leave blank to disable.
CC_RMIPORT=



###################################################################################################
# DO NOT MODIFY ENTRIES BELOW THIS LINE

NAME=cruisecontrol
DESC="CruiseControl - continuous integration build loop"


PATH=/sbin:/usr/sbin:/usr/bin:/bin
# add additions if variable has text defined
if [ -n "$PATH_ADDITIONS" ]; then
    PATH=$PATH_ADDITIONS:$PATH
fi
export PATH


CC_DAEMON=$CC_INSTALL_DIR/cruisecontrol.sh

CC_CONFIG_FILE=$CC_WORK_DIR/config.xml

CC_LOG_FILE=$CC_LOGFILE_DIR/cruisecontrol.log

CC_COMMAND="$CC_DAEMON -configfile $CC_CONFIG_FILE -webport $CC_WEBPORT -jmxport $CC_JMXPORT -rmiport $CC_RMIPORT"

# overwrite settings from default file
if [ -f /etc/default/cruisecontrol ]; then
  . /etc/default/cruisecontrol
fi

# does the executable exist?
test -f $CC_DAEMON || (echo "The executable $CC_DAEMON does not exist!" && exit 0)

if [ `id -u` -ne 0 ]; then
        echo "Not starting/stopping $DESC, you are not root."
        exit 4
fi

# Get the PID output from the startup script
if [ -f $CC_INSTALL_DIR/cc.pid ]; then
 CC_PID=`cat $CC_INSTALL_DIR/cc.pid`
else
 echo "No cc.pid file found.  CC process may not be controllable from this script!"
fi


case "$1" in

  'start')
    cd $CC_INSTALL_DIR
    #echo "CC environtment at startup" > cc.startup.env
    #env >> cc.startup.env
    su $CC_USER -c "/bin/sh -c \"$CC_COMMAND >> $CC_LOG_FILE 2>&1\"" & RETVAL=$?
    echo "$NAME started with jmx on port ${CC_JMXPORT}"
    ;;

  'stop')
    if [ -n "$CC_PID" ] && ps -p ${CC_PID} > /dev/null ; then
     kill -9 ${CC_PID}
      $0 status
      RETVAL=$?
    else
      echo "$NAME is not running"
      RETVAL=1
    fi
    ;;

  'status')
    if [ -n "$CC_PID" ] && ps -p ${CC_PID} > /dev/null ; then
      echo $NAME \(pids $CC_PID\) is running
      RETVAL=0
    else
      echo "$NAME is stopped"
      RETVAL=1
    fi
    ;;

  'restart')
    $0 stop && $0 start
    RETVAL=$?
    ;;

  *)
    echo "Usage: $0 { start | stop | status | restart }"
    exit 1
    ;;
esac
#echo ending $0 $$....
exit 0;

chmod +x /etc/init.d/cruisecontrol

Now you can start and stop your CI instance with

/etc/init.d/cruisecontrol start
/etc/init.d/cruisecontrol stop

The webserver needs a few seconds to come up, so don't be eager and breathe deeply, then fire up your browser.

Thats it! For now...

7. [jenkins] together forever...
Parts in this step are taken from http://jenkins-php.org/ by Sebastian Bergmann. He also created the php-template project used later.

Jenkins can be installed with use of a repository:

sudo wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -
sudo echo "deb http://pkg.jenkins-ci.org/debian binary/" > /etc/apt/sources.list.d/jenkins.list
sudo aptitude update
sudo aptitude install jenkins

Thats it! Jenkins listens by default on port 8080.

To get useful results in the jenkins instance, we use some plugins, installed by the client tool jenkins provides:

wget http://localhost:8080/jnlpJars/jenkins-cli.jar
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin checkstyle
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin clover
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin dry
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin htmlpublisher
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin jdepend
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin plot
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin pmd
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin violations
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin xunit
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin redmine

java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin collapsing-console-sections
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin iphoneview
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin analysis-core
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin analysis-collector
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin selenium
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin piwikanalytics

java -jar jenkins-cli.jar -s http://localhost:8080 safe-restart

8. [phpUnderControl] The build.xml file

9. [phpUnderControl] cruise control config.xml