You may have a number of minions, each with their own database that needs backing up. In the old days (about three months ago), I had a number of client sites all sharing the same server and the same MySQL database instance.

I had a MySQL replication server keeping a copy of the database and would take daily backups of from the replication server (stop slave, backup, start slave) and then send this backup up to Amazon S3.

Since I have started using Salt, I have been able to create lots of smaller servers, most of them hosting a single web project (the idea of doing this without a configuration management product such as Salt was not very appealing.) The most important projects have a replication server, but most are just backed up ‘live’, accepting the moderate risk of loss of data integrity in the knowledge that it wouldn’t be very difficult to restore the database to a workable state.

So, each of these minions has a Salt state defined something like the following:

{% set s3Bucket = "bucketname" %}
{% set project = "projectname" %}
 
mysql-backup-directory:
  file.directory:
    - name: /var/db_dumps
 
mysql-backup-script:
  file.managed:
    - name: /etc/cron.daily/backup_mysql
    - makedirs: False
    - source: salt://_files/mysql/backup_mysql.jinja
    - template: jinja
    - context:
        backupName: {{ grains['id'] }}
        project: {{ project }}
        s3Bucket: {{ s3Bucket }}
    - require:
      - file: mysql_backup_directory
    - mode: 100

The backup_mysql file sends a copy of a database dump, stamped with the day of the week to a directory on the server, and then sends it to the Salt master for safe-keeping:

#!/bin/bash
 
TARGET_DIR="/var/database_backups"
SQL_FILEPATH="$TARGET_DIR/{{ backupName }}.sql"
ZIP_FILENAME="{{ backupName }}-`date '+%A'`.sql.zip"
ZIP_FILEPATH="$TARGET_DIR/$ZIP_FILENAME"
 
/usr/bin/mysqldump --all-databases --add-drop-database --add-drop-table --add-locks --single-transaction --quick --extended-insert --lock-tables --disable-keys -uroot > $SQL_FILEPATH
rm $ZIP_FILEPATH
zip -j $ZIP_FILEPATH $SQL_FILEPATH
rm $SQL_FILEPATH
 
salt-call cp.push $TARGET_FILEPATH

As you can see, it’s very simple to push a file to the Salt master, however, you will need to set the following variable in your /etc/salt/master config file on the master:

# Allow minions to push files to the master. This is disabled by default, for
# security purposes.
file_recv: True

Files will be pushed to the following directory on the master:

/var/cache/salt/master/minions/MINION-ID/files

Slightly less fiddly than rsync or scp and likely cheaper than S3 storage, particularly if your Salt master has spare disk capacity that you can’t reduce to cut costs.

{ 0 comments }

The last post gave an overview of how I am starting to implement deployment pipelines for all my projects.

This post will give an example of a Jenkins job that uses SaltStack to inject configuration data into source files for testing and for deployment.

The Jenkins box itself is described as a collection of Salt states which can be found in this GitHub repo. The states allow Salt to install Jenkins on a clean Ubuntu 14.04 server, install the necessary plugins, and create the jobs for each project that Jenkins will manage.

When Jenkins is running one of those jobs, it will make use of the same Salt master as was used to build the Jenkins box itself. It will as the Salt master for configuration data for the project it is building.

The Jenkins script bit

Here is an example of the “Execute script” section of a Jenkins job

cd build
sudo salt-call state.sls myproject.source-injections > salt.log
cat salt.log
grep "Failed: 0$" salt.log
sudo sh build.sh $BUILD_NUMBER $BUILD_ID

This assumes that previously in the job some source code was pulled into the workspace from a repository such as GitHub and that the source code has a build directory.

The Salt Call

The salt-call command makes a request to the Salt master to execute the state “myproject.source-injections” which will populate some Jinja templates will configuration values. These templates may include config files for the application itself (database usernames and passwords, for example), but may also include populating a build script template.

Here is an example of how the myproject.source-injections SLS file might look. This is an example from a Zend Framework 2 project. The files it is managing are:

  • config/autoload/local.php (project config)
  • composer.json file (populating with GitHub oAuth token which will be needed for Jenkins to call “composer install” non-interactively)
  • build/selenium/suite.html and build/selenium/master.html (Selenium test files)
  • build/build.sh (The build script itself)
{% set repo = 'myproject' %}
{% if 'jenkins' in grains['id'] %}
{% set sourceCodeRoot = '/var/lib/jenkins/jobs/' + repo + '/workspace' %}
{% else %}
{% set sourceCodeRoot = '/var/www/' + repo %}
{% endif %}
 
{{ sourceCodeRoot }}/config/autoload/local.php:
file:
- managed
- source: file://{{ sourceCodeRoot }}/config/autoload/local.jinja
- template: jinja
- mode: 755
{% if 'jenkins' not in grains['id'] %}
- require:
- git: clone_{{ repo }}
{% endif %}
 
{{ sourceCodeRoot }}/composer.json:
file:
- managed
- source: file://{{ sourceCodeRoot }}/composer.json.jinja
- template: jinja
- mode: {{ pillar['zf2']['configfiles']['mode'] }}
{% if 'jenkins' not in grains['id'] %}
- require:
- git: clone_{{ repo }}
{% endif %}
 
{{ sourceCodeRoot }}/build/selenium/suite.html:
file.managed:
- source: file://{{ sourceCodeRoot }}/tests/selenium/suite.html.jinja
- template: jinja
{% if 'jenkins' not in grains['id'] %}
- require:
- git: clone_{{ repo }}
{% endif %}
 
{% for testCase in pillar[repo]['selenium-test-cases'] %}
 
{{ sourceCodeRoot }}/build/selenium/{{ testCase }}.html:
file.managed:
- source: file://{{ sourceCodeRoot }}/tests/selenium/{{ testCase }}.html.jinja
- template: jinja
{% if 'jenkins' not in grains['id'] %}
- require:
- git: clone_{{ repo }}
{% endif %}
 
{% endfor %}
 
{{ sourceCodeRoot }}/build/build.sh:
file.managed:
- source: file://{{ sourceCodeRoot }}/build/build.sh.jinja
- template: jinja
- mode: 777
{% if 'jenkins' not in grains['id'] %}
- require:
- git: clone_{{ repo }}
{% endif %}

The build script

And here is an example build script. It is rough and ready, but does some good things. The first thing it does is to install the project dependencies and run the unit tests. If we fail here, we bail out and fail the build. If all is well, it then packages up the source code (this is the only artifact created in this build).

Now, the source is then sent over and installed on a test server. Selenium tests are run against the test server, and, if those pass, the source is installed on a second server (in this case a UAT server) and the Selenium tests are run against this second server. The application is then ready for its first use by a manual tester.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
#!/bin/bash
 
{% set key='myproject' %}
REPO={{pillar[key]['repository']}}
DOMAIN={{pillar[key]['domain']}}
UAT_URL={{pillar[key]['protocol']}}{{pillar[key]['httpurllogin']}}{{pillar[key]['domain']}}
TEST_URL={{pillar[key]['protocol']}}{{pillar[key]['httpurllogin']}}test.{{pillar[key]['domain']}}
 
runSelenium() {
 
	#################################################################################
	echo "Preparing to run Selenium tests against $1"
	#################################################################################
 
        cd build/selenium
 
	#################################################################################
	echo "Killing any old displays"
	#################################################################################
 
	rm /tmp/.X10-lock
	fuser -k 6010/tcp
	fuser -k 4444/tcp
 
	#################################################################################
	echo "Initiating display"
	#################################################################################
 
	Xvfb :10 -ac &
	export DISPLAY=:10
 
	#################################################################################
	echo "Running Selenium suite"
	#################################################################################
 
	java -jar /usr/bin/selenium-server.jar -multiwindow -htmlSuite "*firefox" "$1" "suite.html" "report.html"
 
	if grep '<td>passed</td>' report.html
	then
	    echo "Tests passed"
	else
	    echo "Cannot confirm that tests passed"
	    exit 1
	fi
 
	cd ../..
}
 
#################################################################################
echo "Stamping build"
#################################################################################
 
echo $1 > build_number.txt
echo $2 > build_time.txt
 
#################################################################################
echo "Installing Composer dependencies"
#################################################################################
composer install
 
#################################################################################
echo "Running phpunit tests"
#################################################################################
cd tests
phpunit
 
rc=$?
if [ $rc != 0 ] ; then
    exit $rc
fi
 
#################################################################################
echo "Tarring source"
#################################################################################
 
rm source.tar.gz
tar -zcf source.tar.gz ./*
 
#################################################################################
echo "Sending source to test site"
#################################################################################
 
scp -i /var/lib/jenkins/.ssh/id_rsa source.tar.gz jenkins@$DOMAIN:/var/www/test.$REPO
 
rc=$?
if [ $rc != 0 ] ; then
    exit $rc
fi
 
#################################################################################
echo "Untarring source on remote test site"
#################################################################################
 
ssh -i /var/lib/jenkins/.ssh/id_rsa jenkins@$DOMAIN "cd /var/www/test.$REPO && tar -zxf source.tar.gz"
 
#################################################################################
echo "Running Selenium test on test server"
#################################################################################
 
runSelenium $TEST_URL
 
#################################################################################
echo "Sending source to live site"
#################################################################################
 
scp -i /var/lib/jenkins/.ssh/id_rsa source.tar.gz jenkins@$DOMAIN:/var/www/$REPO
 
rc=$?
if [ $rc != 0 ] ; then
    exit $rc
fi
 
#################################################################################
echo "Untarring source on live site"
#################################################################################
 
ssh -i /var/lib/jenkins/.ssh/id_rsa jenkins@$DOMAIN "cd /var/www/$REPO && tar -zxf source.tar.gz"
 
#################################################################################
echo "Running Selenium test on live server"
#################################################################################
 
runSelenium $UAT_URL
 
#################################################################################
echo "Done"
#################################################################################

In the next post I will describe how to build the Jenkins box itself from SaltStack definitions which will include creating jobs such as the one described above as part of the box build.

{ 0 comments }

A project’s deployment lifecycle

I’ve recently reviewed and revamped my entire process for developing and deploying applications. I’m going to describe the process in this post and then over the next few weeks I will go into more technical details about how to set up the system.

The star of the whole show is SaltStack which manages the configuration for every environment in the process – if your Jenkins box goes wobbly, for example, you can get SaltStack to fire up a new fully-functioning Jenkins instance with all your remote server credentials and deployment keys set up. And if you screw up your development environment, just destroy your virtual machine and get SaltStack to provision a new one by re-installing all the necessary services and setting up your configuration files.

This process presents a decent level of automation, quality assurance and control. Your comments and suggestions will be very welcome.

I have started a public repo on GitHub which holds the Salt master state configuration and pillar structure and examples of project machine configs and Jenkins jobs.

The following diagram gives an overview of the deployment pipeline.

pipeline

A commit’s journey

Assuming that the system shown above is in place, I’ll talk through what happens when a developer commits a line of code.

Step 1 – Commit

The developer, having tested the code locally, commits to the repository and then pushes the updated repository to GitHub. If the commit was to the master branch it is now a candidate for release to production.

The development environment on which the developer is working has been created as a virtual machine using Vagrant and definitions of the environment held in the Salt master.

Step 2 – Build

Jenkins notices that the master branch has been updated and grabs the latest code and performs a build. The build consists of populating any Jinja templates that we have in our code base (e.g. config files requiring passwords), running any package managers such as Composer, compiling the code, or indeed any other activity required to put the code into a working state.

The Jenkins box is a Salt minion, that is to say, it was created from server configs held on the Salt master. As a result, we can easily set it to have access to any pillar data (passwords, etc…) that may be needed by any project whose environment is also managed by the Salt master.

Step 3 – Unit Test

If Jenkins manages to build the code without errors, it will run the unit tests. If the build has failed, or if the unit tests fail, Jenkins will turn the project red and we must give our full attention to fixing the problem, as no code is going to progress past this point until the problem is resolved.

Step 4 – Deploy to Test Environment

On successful completion of the unit tests, Jenkins will push the code out to a test server which will be a full production-like environment. All environments from development through to production are built using the same server definitions so we don’t need to configure each environment manually causing small but potentially baffling differences in their config.

Step 5 – Automated Acceptance Test

The Jenkins box will now run some Selenium tests against the newly-deployed code. If they pass, we now have a build that we are happy to put before a manual tester (i.e., either you or your client). This build will be stamped and identifiable so that we can choose it for deployment to any environment at any time.

Step 6 – To the Next Environment

Following manual testing you will have a release that is ready for the next stage which may well simply be a production server ready for the world to use. Manual testing will have been performed against an identiable build which can now be deployed to the chosen environment.

{ 2 comments }

Just a quick one. Here is an example of how to test a ViewHelper in Zend Framework 2.

In this case, the View Helper is relying on a view renderer (PhpRenderer) and a helper plugin (FormElement), as shown by the following line which uses them both.

1
echo $this->view->formElement($element);

Here is a simple and not particularly exploratory test that checks that my custom view helper is rendering fieldsets in the HTML output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
    /**
     * @outputBuffering disabled
     */
    public function testCanRenderFieldsets() 
    {
        $this->expectOutputRegex(
            '/<form(.*)<fieldset(.*)<\/fieldset>(.*)<fieldset(.*)<\/fieldset>(.*)<\/form>/'
        );
 
        $form = new NetsensiaForm();
        $form->addHidden('test', 'testvalue');
 
        $element1 = new Text('testelement1');
        $element1->setLabel('Test Element');
        $element1->setAttribute('icon', 'pencil');
 
        $element2 = new Text('testelement2');
        $element2->setLabel('Test Element 2');
        $element2->setAttribute('icon', 'pencil');
 
        $fieldset1 = new Fieldset('testfieldset1');
        $fieldset1->add($element1);
 
        $fieldset2 = new Fieldset('testfieldset2');
        $fieldset2->add($element2);
 
        $form->add($fieldset1);
        $form->add($fieldset2);
 
        $helpers = new HelperPluginManager();
        $helpers->setService('formElement', new FormElement());
        $view = new PhpRenderer();
        $view->setHelperPluginManager($helpers);
 
        $viewHelper = new BootstrapForm();
        $viewHelper->setView($view);
        $viewHelper($form, 'testform', '/');
    }

{ 0 comments }

In the previous post I gave my opinions on Behaviour-Driven Development. Here I’ll give an introduction to Behat and how to use it in conjunction with Mink to describe and test the behavior of web systems. In the next post I’ll explain how to integrate it with your development framework so that we can move beyond black-box and start testing the behaviour of the system internals.

PHP and Behat

Behat allows you to write tests like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Feature: Register
  In order to create an account
  As a user
  I need to be able to submit my details using a registration form
 
Scenario: Access the registration via Get Started button on homepage
  Given I am on the homepage
  When I follow "Get started"
  Then I should see "Create My Account"
 
Scenario: Submit user registration details with mismatched passwords
  Given I am on the registration page
  When I fill in "username" with "Chrismo"
  And I fill in "email" with "chris@chrismo.com"
  And I fill in "password" with "asdasd12"
  But I fill in "confirmpassword" with "adsasd11"
  And press "Create My Account"
  Then I should see "Passwords don't match"

The first part of the above script, the “Feature” description is a user story. It does not map to any test code, but describes the context of the scenarios to follow.

The second two sections of the script – the scenarios themselves – get turned into code that is run as tests.

As it happens, these aren’t great scenarios – perhaps they are too explicit in terms of how the user will interact with the system. We could just as easily write scenarios such as:

1
2
3
4
5
6
7
8
9
Scenario: Access the registration via from the homepage
  Given I am on the homepage
  When I follow the link to register on the site
  Then I should see the registration form
 
Scenario: Submit user registration details with mismatched passwords
  Given I am on the registration page
  When I fill in the registration form with mismatched passwords
  Then I should be informed that my passwords need to match

However, we’ll stick with our original scenarios for now. As you will see they take advantage of some existing test methods which saves us getting into too much coding detail right away.

In either case, the structure of the scenarios is the same, comprising three key sections:

Given
Describe the preconditions for the test

When
Actions and triggers.

Then
The post conditions of a successful test.

And/But
These keywords are all equivalent, and allow us to write tests in a more natural manner than if we were provided with just one key word. They can be used to add additional sentences to any of the three scenario sections.

What does the test code look like?
When you run the tests for the first time, Behat will attempt to map the components of your script to existing PHP methods in order to run the tests. Its standard output will show you which methods were used, and, where it can’t map to an existing method, it will tell you what you need to do. Something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
You can implement step definitions for undefined steps with these snippets:
 
Feature: Register
  In order to create an account
  As a user
  I need to be able to submit my details using a registration form
 
  Scenario: Access the registration form # features\register.feature:7
    Given I am on the homepage           # FeatureContext::iAmOnHomepage()
    When I follow "Get started"          # FeatureContext::clickLink()
    Then I should see "Your username"    # FeatureContext::assertPageContainsText()

  Scenario: Submit user registration details with mismatched passwords # features\register.feature:12
    Given I am on the registration page
    When I fill in "username" with "Chrismo"                           # FeatureContext::fillField()
    And I fill in "email" with "chris@chrismo.com"                     # FeatureContext::fillField()
    And I fill in "password" with "asdasd12"                           # FeatureContext::fillField()
    But I fill in "confirmpassword" with "adsasd11"                    # FeatureContext::fillField()
    And press "Create My Account"                                      # FeatureContext::pressButton()
    Then I should see "Passwords don't match"                          # FeatureContext::assertPageContainsText()

2 scenarios (1 passed, 1 undefined)
10 steps (3 passed, 6 skipped, 1 undefined)
0m0.468s
 
You can implement step definitions for undefined steps with these snippets:
 
    /**
     * @Given /^I am on the registration page$/
     */
    public function iAmOnTheRegistrationPage()
    {
        throw new PendingException();
    }

The FeatureContext class
As it happens, Behat was able to map all but one of scenario lines to an existing method.

iShouldSee() is provided by the Behat core the others are provided by the MinkContext class which is provided by the Mink library. The Mink library allows us to plugin browser controllers such as Selenium to allow the writing of tests that test the behaviour of web-based applications.

Without going into full detail (I’ll do that in the next post), you will need a class called FeatureContext to provide these methods, and, in this case, one that inherits from MinkContext.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;
use Behat\MinkExtension\Context\MinkContext;
 
class FeatureContext extends MinkContext
{
    /**
     * @Given /^I am on the registration page$/
     */
    public function iAmOnTheRegistrationPage()
    {
        $this->getSession()->visit($this->locatePath('/register'));
    } 
}

Who writes the tests?
While the aim is to use language that feels natural and understandable to any team member, it still requires some formalising. Ultimately a coder is going to have to ensure that the script components map correctly to tests in code, but the team as a whole should define the vocabulary together and be able to write tests jointly and severally.

In the next post I’ll describe step-by-step how to set up Behat and Mink and also how to bootstrap a Zend Framework 2 application so that we have access to our system’s internals which allows us to go deeper than pure black-box testing.

Rather than wait for me however, I would recommend spending a few minutes reviewing the speaker deck by Ryan Weaver as it is an excellent introduction to Behat and how to configure it.

{ 0 comments }

When testing Zend Framework 2 controllers, you may find yourself wanting to test a path within an action that requires a user to be logged in. We do this using PHPUnit mock objects.

I use a standard base test controller inherited from PHPUnit_Framework_TestCase. The pertinent methods are shown below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<?php
namespace Netsensia\Test;
 
use PHPUnit_Framework_TestCase;
use TestSuite\Bootstrap;
use Zend\Mvc\Router\Http\TreeRouteStack as HttpRouter;
use Zend\Http\Request;
use Zend\Mvc\MvcEvent;
use Zend\Mvc\Router\RouteMatch;
use Netsensia\Adaptor\Auth\UserSessionModel;
use Zend\Test\PHPUnit\Controller\AbstractControllerTestCase;
 
class NetsensiaControllerTest extends AbstractHttpControllerTestCase
{
    ...
 
    protected function mockLogin()
    {
        $userSessionModel = new UserSessionModel();
        $userSessionModel->setUserId(1);
        $userSessionModel->setName('Tester');
 
        $authService = $this->getMock('Zend\Authentication\AuthenticationService');
        $authService->expects($this->any())
                    ->method('getIdentity')
                    ->will($this->returnValue($userSessionModel));
 
        $authService->expects($this->any())
                    ->method('hasIdentity')
                    ->will($this->returnValue(true));
 
        $this->getApplicationServiceLocator()->setAllowOverride(true);
        $this->getApplicationServiceLocator()->setService('Zend\Authentication\AuthenticationService', $authService);
    }
 }

You’ll notice the mockLogin() method, which creates a mock authentication service and uses it to override the existing service. We tell this mock object to return the instance of a user identity of our choosing and to return true when asked if an identity exists. Then, anywhere in the code that is being tested, it will appear as if a user is logged in, assuming that the test for user existence is querying the authentication service returned by the service manager.

Within a controller test, we simply use it as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
use Zend\Http\Request;
use Netsensia\Test\NetsensiaControllerTest;
 
class LocaleControllerTest extends NetsensiaControllerTest
{
    ...
    ...
 
    public function testCanSwitchLocaleWhenLoggedIn()
    {
        $this->mockLogin();
        $result = $this->dispatch('/locale/fr_FR');
        $this->assertRedirectTo('/user/profile');
    }
}

{ 8 comments }

Understanding magic bitboards in chess programming

Overview

This post relates to programming issues involved with creating bitboard chess engines. It was originally rewritten to coincide with the release of the Android version of Rival Chess in 2011 and published on rivalchess.com.

For further reference, the back catalog of Rival Chess code going back to 1992 is available on my GitHub account.

Up until 2010, Rival was using pretty straightforward move-generation techniques. I had always intended to dive into bitboards at some point but never had the time. The chance came, however, with the development of a new Java engine for an Android version of Rival.

I started that phase of development using rotated bitboards but quickly switched to magic bitboards after dedicating a few days to their full understanding.

I think generating chess moves using magic bitboards is one of the most awesome things you can do with computer code and this post is my attempt to explain the process in as simple a manner as possible, something I wish I had been able to read before I started.

I think it’s quite possible to understand how to use magic bitboards without a background in the wider concept of standard and rotated bitboards, which are not covered in any detail in this post.

Rival and bitboards

Throughout the code, Rival uses bitboards to represent chess board states. Board states are represented by 64-bit integers, with bit 63 representing square A8 and bit 0 representing square H1. For example, you can imagine a bitboard where every set bit represents a square occupied by an enemy piece. When you have various states represented like this, there are many manipulations and calculations you can perform to get valuable information, and get it quickly (admittedly, the code is Java so speed isn’t the main focus here).

What are magic bitboards?

Magic bitboards allow us to calculate the possible destination squares for sliding pieces with a simple calculation, rather than needing to loop through each square to determine if it is occupied, and then, if it is, to go back and set off in another direction, gathering possible moves as we go.

Given infinite memory, we could hold a huge lookup table in memory. We could index into this table using the piece type, the destination square, and a 64-bit integer representing the position of all the enemy pieces on the board. Unfortunately, this requires an impossibly large array.

With magic bitboards, we can indeed use that table-based-lookup idea, but instead of using a 64-bit number to represent the state of the board, we can use a much, much smaller number.

The trick is to determine the position of all the blockers on the board (friendly and enemy pieces on the horizontal or diagonals in question) and represent this as a 64-bit number. Then we multiply this by a magic number that is specific to the type of piece and the source square, perform a little shift on the result, and then we have a nice, small number that we can use as an index into the memory table that will return us another bitboard representing the available destination squares for our piece.

This will include moves that take our own pieces, so we perform another bitwise operation on the result to remove those destination squares and we’re done.

The magic of the magic number (and the related, magic number of shifts performed at the end) is that it is produces the correct index for all the various blocker configurations either by

1) Returning the same index for different blocker configurations that require the same result
2) Returning a different index which contains an identical result

Here is an example of how to use a magic calculation to generate a bitboard with bits set for each destination square available for the moves for a rook on square C3…

1
2
3
4
5
bbBlockers = bbAllPieces & occupancyMaskRook[C3]
 
databaseIndex = (int)((bbBlockers * magicNumberRook[C3]) >>> rookMagicShifts[C3])
 
bbMoveSquares = magicMovesRook[C3][databaseIndex] & ~bbFriendlyPieces

As an example, let’s say we want to use this forumula to find the moves of the white rook on C3 on the following board…

We would use…

All Pieces Rook Mask for C3 Occupancy Magic Number for Rook on C3 Magic Result Required Shifts for Rook on C3 Array Index
10010001
00000111
10000000
00010000
10110110
00000111
01100000
00001100
& 00000000
00100000
00100000
00100000
00100000
01011110
00100000
00000000
= 00000000
00000000
00000000
00000000
00100000
00000110
00100000
00000000
* 01101000
00100000
10000000
10000000
00000100
00000000
00100010
00000000
= 00010011
10010000
00011100
11000000
11010000
01000000
00000000
00000000
>>> 54 = 00000000
00000000
00000000
00000000
00000000
00000000
00000000
01001110
= 78

And then we use the index “78” to find the moves…

magicMovesRook[C3][78] ~bitboardFriendly Move Destinations
00000000
00000000
00000000
00000000
00100000
11011100
00100000
00000000
& 11111111
11111111
11111111
11111111
01101011
11111100
10011111
11110011
= 00000000
00000000
00000000
00000000
00100000
11011100
00000000
00000000

All Pieces The bitboard containing bits set to show the location of every piece on the chessboard.
Rook Mask for C3 Occupancy Mask For Rook on C3. Each “1” represents a blocker which would bring an end to the slides of the rook. Note that we don’t need to specify the edge squares because they are always blockers. The difference between a friendly blocker and an enemy blocker is sorted out at the end of the calculation.
Occupancy Has bits set for any piece, enemy or friendly, that sits along the attack rays of a rook on C3
Magic Number for Rook on C3 If memory were no issue, we could just use the occupancy value as an index into a huge 2^64 size array. But this is not possible, so here is the magic number whose purpose is to allow the generation of an index into a reasonable-sized array.

This number is generated by brute force trial and error. We can always find a number that will generate an index between 0 and 2^n (and if you’re really smart you can sometimes get a smaller index), where n is the number of bits set in the occupancy mask. A perfect mapping to unique indexes is probably impossible (I’m not a mathematician but I suspect it’s impossible) but luckily multiple occupancies can share the same index as they will produce the same moves.

Note that the number is fairly sparse (just a few 1s) – you’ll notice when generating magics by brute force that it is much, much quicker to try random sparse numbers than just any old number.

Magic Result This doesn’t mean much, at least not until we shift it a bit… You need “>>>” rather than “>>” when coding in Java as there is no unsigned long type.
Required Shifts for Rook on C3 The number of shifts needed to turn the magic result into an array index.
Array Index The index into the rook moves database.
magicMovesRook[C3][78] Using the index “78” from above.

As mentioned, more than one occupancy pattern would map to this result, but also this result may reside in multiple indexes – all that matters is that the magic number creates an index that can retrieve the correct result.

~bitboardFriendly The inverted bitboard of friendly pieces.
Move Destinations A bitboard with 1s set for each legal destination square for our rook on C3.

Using the coding of a8=63, h1=0, the following magic numbers will work for generating the sliding moves…

1
2
3
public static long magicNumberRook[] = {
        0xa180022080400230L, 0x40100040022000L, 0x80088020001002L, 0x80080280841000L, 0x4200042010460008L, 0x4800a0003040080L, 0x400110082041008L, 0x8000a041000880L, 0x10138001a080c010L, 0x804008200480L, 0x10011012000c0L, 0x22004128102200L, 0x200081201200cL, 0x202a001048460004L, 0x81000100420004L, 0x4000800380004500L, 0x208002904001L, 0x90004040026008L, 0x208808010002001L, 0x2002020020704940L, 0x8048010008110005L, 0x6820808004002200L, 0xa80040008023011L, 0xb1460000811044L, 0x4204400080008ea0L, 0xb002400180200184L, 0x2020200080100380L, 0x10080080100080L, 0x2204080080800400L, 0xa40080360080L, 0x2040604002810b1L, 0x8c218600004104L, 0x8180004000402000L, 0x488c402000401001L, 0x4018a00080801004L, 0x1230002105001008L, 0x8904800800800400L, 0x42000c42003810L, 0x8408110400b012L, 0x18086182000401L, 0x2240088020c28000L, 0x1001201040c004L, 0xa02008010420020L, 0x10003009010060L, 0x4008008008014L, 0x80020004008080L, 0x282020001008080L, 0x50000181204a0004L, 0x102042111804200L, 0x40002010004001c0L, 0x19220045508200L, 0x20030010060a900L, 0x8018028040080L, 0x88240002008080L, 0x10301802830400L, 0x332a4081140200L, 0x8080010a601241L, 0x1008010400021L, 0x4082001007241L, 0x211009001200509L, 0x8015001002441801L, 0x801000804000603L, 0xc0900220024a401L, 0x1000200608243L
    };
1
2
3
 public static long magicNumberBishop[] = {
        0x2910054208004104L, 0x2100630a7020180L, 0x5822022042000000L, 0x2ca804a100200020L, 0x204042200000900L, 0x2002121024000002L, 0x80404104202000e8L, 0x812a020205010840L, 0x8005181184080048L, 0x1001c20208010101L, 0x1001080204002100L, 0x1810080489021800L, 0x62040420010a00L, 0x5028043004300020L, 0xc0080a4402605002L, 0x8a00a0104220200L, 0x940000410821212L, 0x1808024a280210L, 0x40c0422080a0598L, 0x4228020082004050L, 0x200800400e00100L, 0x20b001230021040L, 0x90a0201900c00L, 0x4940120a0a0108L, 0x20208050a42180L, 0x1004804b280200L, 0x2048020024040010L, 0x102c04004010200L, 0x20408204c002010L, 0x2411100020080c1L, 0x102a008084042100L, 0x941030000a09846L, 0x244100800400200L, 0x4000901010080696L, 0x280404180020L, 0x800042008240100L, 0x220008400088020L, 0x4020182000904c9L, 0x23010400020600L, 0x41040020110302L, 0x412101004020818L, 0x8022080a09404208L, 0x1401210240484800L, 0x22244208010080L, 0x1105040104000210L, 0x2040088800c40081L, 0x8184810252000400L, 0x4004610041002200L, 0x40201a444400810L, 0x4611010802020008L, 0x80000b0401040402L, 0x20004821880a00L, 0x8200002022440100L, 0x9431801010068L, 0x1040c20806108040L, 0x804901403022a40L, 0x2400202602104000L, 0x208520209440204L, 0x40c000022013020L, 0x2000104000420600L, 0x400000260142410L, 0x800633408100500L, 0x2404080a1410L, 0x138200122002900L    
    };

Generating the required arrays and magic numbers

The occupancyMask arrays can be calculated easily enough, but here they are along with the code to calculate them… you’ll notice in the code that we avoid marking edge squares in the mask. This is because an edge square is always a blocker so does not need to be defined as such – a slider will always stop on top of the first blocker it encounters (captures of friendly pieces are filtered out at the end as you can see above.)

1
2
3
 public static final long occupancyMaskRook[] = {
        0x101010101017eL, 0x202020202027cL, 0x404040404047aL, 0x8080808080876L, 0x1010101010106eL, 0x2020202020205eL, 0x4040404040403eL, 0x8080808080807eL, 0x1010101017e00L, 0x2020202027c00L, 0x4040404047a00L, 0x8080808087600L, 0x10101010106e00L, 0x20202020205e00L, 0x40404040403e00L, 0x80808080807e00L, 0x10101017e0100L, 0x20202027c0200L, 0x40404047a0400L, 0x8080808760800L, 0x101010106e1000L, 0x202020205e2000L, 0x404040403e4000L, 0x808080807e8000L, 0x101017e010100L, 0x202027c020200L, 0x404047a040400L, 0x8080876080800L, 0x1010106e101000L, 0x2020205e202000L, 0x4040403e404000L, 0x8080807e808000L, 0x1017e01010100L, 0x2027c02020200L, 0x4047a04040400L, 0x8087608080800L, 0x10106e10101000L, 0x20205e20202000L, 0x40403e40404000L, 0x80807e80808000L, 0x17e0101010100L, 0x27c0202020200L, 0x47a0404040400L, 0x8760808080800L, 0x106e1010101000L, 0x205e2020202000L, 0x403e4040404000L, 0x807e8080808000L, 0x7e010101010100L, 0x7c020202020200L, 0x7a040404040400L, 0x76080808080800L, 0x6e101010101000L, 0x5e202020202000L, 0x3e404040404000L, 0x7e808080808000L, 0x7e01010101010100L, 0x7c02020202020200L, 0x7a04040404040400L, 0x7608080808080800L, 0x6e10101010101000L, 0x5e20202020202000L, 0x3e40404040404000L, 0x7e80808080808000L 
    };
1
2
3
public static final long occupancyMaskBishop[] = {
        0x40201008040200L, 0x402010080400L, 0x4020100a00L, 0x40221400L, 0x2442800L, 0x204085000L, 0x20408102000L, 0x2040810204000L, 0x20100804020000L, 0x40201008040000L, 0x4020100a0000L, 0x4022140000L, 0x244280000L, 0x20408500000L, 0x2040810200000L, 0x4081020400000L, 0x10080402000200L, 0x20100804000400L, 0x4020100a000a00L, 0x402214001400L, 0x24428002800L, 0x2040850005000L, 0x4081020002000L, 0x8102040004000L, 0x8040200020400L, 0x10080400040800L, 0x20100a000a1000L, 0x40221400142200L, 0x2442800284400L, 0x4085000500800L, 0x8102000201000L, 0x10204000402000L, 0x4020002040800L, 0x8040004081000L, 0x100a000a102000L, 0x22140014224000L, 0x44280028440200L, 0x8500050080400L, 0x10200020100800L, 0x20400040201000L, 0x2000204081000L, 0x4000408102000L, 0xa000a10204000L, 0x14001422400000L, 0x28002844020000L, 0x50005008040200L, 0x20002010080400L, 0x40004020100800L, 0x20408102000L, 0x40810204000L, 0xa1020400000L, 0x142240000000L, 0x284402000000L, 0x500804020000L, 0x201008040200L, 0x402010080400L, 0x2040810204000L, 0x4081020400000L, 0xa102040000000L, 0x14224000000000L, 0x28440200000000L, 0x50080402000000L, 0x20100804020000L, 0x40201008040200L     
    };
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
    public void generateOccupancyMasks()
    {
        int i, bitRef;
        long mask;
        for (bitRef=0; bitRef<=63; bitRef++)
        {
            mask = 0;
            for (i=bitRef+8; i<=55; i+=8) mask |= (1L << i);
            for (i=bitRef-8; i>=8; i-=8) mask |= (1L << i);
            for (i=bitRef+1; i%8!=7 && i%8!=0 ; i++) mask |= (1L << i);
            for (i=bitRef-1; i%8!=7 && i%8!=0 && i>=0; i--) mask |= (1L << i);
            occupancyMaskRook[bitRef] = mask;
 
            mask = 0;
            for (i=bitRef+9; i%8!=7 && i%8!=0 && i<=55; i+=9) mask |= (1L << i);
            for (i=bitRef-9; i%8!=7 && i%8!=0 && i>=8; i-=9) mask |= (1L << i);
            for (i=bitRef+7; i%8!=7 && i%8!=0 && i<=55; i+=7) mask |= (1L << i);
            for (i=bitRef-7; i%8!=7 && i%8!=0 && i>=8; i-=7) mask |= (1L << i);
            occupancyMaskBishop[bitRef] = mask;
        }
    }

And the shift arrays are a function of the maximum number of bits that can be set in the occupancy mask for that piece and that square (i.e. 63-maxBits).
It is possible that these shift values can be larger than 63-maxBits if a magic number is found that can generate a database index smaller than 2^maxBits – although I’ve not done this here, this is possible because many occupancy variations share the same eventual move destinations bitboard.

1
2
3
4
5
6
public static int magicNumberShiftsRook[] = {
    52,53,53,53,53,53,53,52,53,54,54,54,54,54,54,53,
    53,54,54,54,54,54,54,53,53,54,54,54,54,54,54,53,
    53,54,54,54,54,54,54,53,53,54,54,54,54,54,54,53,
    53,54,54,54,54,54,54,53,52,53,53,53,53,53,53,52
};
1
2
3
4
5
6
public static int magicNumberShiftsBishop[] = {
    58,59,59,59,59,59,59,58,59,59,59,59,59,59,59,59,
    59,59,57,57,57,57,59,59,59,59,57,55,55,57,59,59,
    59,59,57,55,55,57,59,59,59,59,57,57,57,57,59,59,
    59,59,59,59,59,59,59,59,58,59,59,59,59,59,59,58
};

To generate the magic numbers, we first need to populate a occupancyVariation and atttackSet array. The occupancyVariation array will contain all the occupancy variations for a piece on a given square.

In the case of a rook on C3, as there are 10 bits in the occupancy mask, there (2^10) = 1024 occupancy variations, and many of these variations will be effectively the same as one another as they represent the same available moves.

e.g.

This occupancy variation... ...is the same as this one... ...and can be represented as the follwing attack set (we set a 1 on the first blocker, except edge squares)
00000000
00100000
00100000
00100000
00000000
01010010
00100000
00000000
00000000
00000000
00000000
00100000
00000000
01010110
00100000
00000000
= 00000000
00000000
00000000
00100000
00000000
01010000
00100000
00000000

Calculating these occupancy variations and attack sets can be done as follows…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
    public void generateOccupancyVariations(boolean isRook)
    {
        int i, j, bitRef;
        long mask;
        int variationCount;
        int[] setBitsInMask, setBitsInIndex;
        int bitCount[] = new int[64];
 
        for (bitRef=0; bitRef<=63; bitRef++)
        {
            mask = isRook ? occupancyMaskRook[bitRef] : occupancyMaskBishop[bitRef];
            setBitsInMask = Bitboards.getSetBits(mask);
            bitCount[bitRef] = Bitboards.countSetBits(mask);
            variationCount = (int)(1L << bitCount[bitRef]);
            for (i=0; i<variationCount; i++)
            {
                occupancyVariation[bitRef][i] = 0; 
 
                // find bits set in index "i" and map them to bits in the 64 bit "occupancyVariation"
 
                setBitsInIndex = Bitboards.getSetBits(i); // an array of integers showing which bits are set
                for (j=0; setBitsInIndex[j] != -1; j++)
                {
                    occupancyVariation[bitRef][i] |= (1L << setBitsInMask[setBitsInIndex[j]]);
                }
 
                if (isRook)
                {
                    for (j=bitRef+8; j<=55 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j+=8);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef-8; j>=8 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j-=8);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef+1; j%8!=7 && j%8!=0 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j++);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef-1; j%8!=7 && j%8!=0 && j>=0 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j--);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                }
                else
                {
                    for (j=bitRef+9; j%8!=7 && j%8!=0 && j<=55 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j+=9);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef-9; j%8!=7 && j%8!=0 && j>=8 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j-=9);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef+7; j%8!=7 && j%8!=0 && j<=55 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j+=7);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                    for (j=bitRef-7; j%8!=7 && j%8!=0 && j>=8 && (occupancyVariation[bitRef][i] & (1L << j)) == 0; j-=7);
                    if (j>=0 && j<=63) occupancyAttackSet[bitRef][i] |= (1L << j);
                }
            }
        }
    }

Now we have enough information to generate the magic numbers…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
    public void generateMagicNumbers(boolean isRook)
    {
        int i, j, bitRef, variationCount;
 
        Random r = new Random();
        long magicNumber = 0;
        int index;
        long attackSet;
 
        for (bitRef=0; bitRef<=63; bitRef++)
        {
            int bitCount = Bitboards.countSetBits(isRook ? occupancyMaskRook[bitRef] : occupancyMaskBishop[bitRef]);
            variationCount = (int)(1L << bitCount);
            boolean fail;
            long usedBy[] = new long[(int)(1L << bitCount)];
 
            int attempts = 0;
 
            do
            {
                magicNumber = r.nextLong() & r.nextLong() & r.nextLong(); // generate a random number with not many bits set
                for (j=0; j<variationCount; j++) usedBy[j] = 0;
                attempts ++;
 
                for (i=0, fail=false; i<variationCount && !fail; i++)
                {
                    index = (int)((occupancyVariation[bitRef][i] * magicNumber) >>> (64-bitCount));
 
                    // fail if this index is used by an attack set that is incorrect for this occupancy variation
                    fail = usedBy[index] != 0 && usedBy[index] != occupancyAttackSet[bitRef][i];
 
                    usedBy[index] = attackSet;
                }
            } 
            while (fail);
 
            if (isRook)
            {
                magicNumberRook[bitRef] = magicNumber;
                magicNumberShiftsRook[bitRef] = (64-bitCount);
            }
            else
            {
                magicNumberBishop[bitRef] = magicNumber;
                magicNumberShiftsBishop[bitRef] = (64-bitCount);
            }
        }
    }

Now we just need to populate the magicMovesRook and magicMovesBishop arrays. This is easily done by going through each occupancyMask value and multiplying it by the relevant magic number to determine the index that will be generated at runtime. We then just put the correct move bitboard into this index of the array.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
    public void generateMoveDatabase(boolean isRook)
    {
        long validMoves;
        int variations, bitCount;
        int bitRef, i, j, magicIndex;
 
        for (bitRef=0; bitRef<=63; bitRef++)
        {
            bitCount = isRook ? Bitboards.countSetBits(occupancyMaskRook[bitRef]) : Bitboards.countSetBits(occupancyMaskBishop[bitRef]);
            variations = (int)(1L << bitCount);
 
            for (i=0; i<variations; i++)
            {
                validMoves = 0;
                if (isRook)
                {
                    magicIndex = (int)((occupancyVariation[bitRef][i] * magicNumberRook[bitRef]) >>> magicNumberShiftsRook[bitRef]);
 
                    for (j=bitRef+8; j<=63; j+=8) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
                    for (j=bitRef-8; j>=0; j-=8) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
                    for (j=bitRef+1; j%8!=0; j++) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
                    for (j=bitRef-1; j%8!=7 && j>=0; j--) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
 
                    magicMovesRook[bitRef][magicIndex] = validMoves;
                }
                else
                {
                    magicIndex = (int)((occupancyVariation[bitRef][i] * magicNumberBishop[bitRef]) >>> magicNumberShiftsBishop[bitRef]);
 
                    for (j=bitRef+9; j%8!=0 && j<=63; j+=9) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
                    for (j=bitRef-9; j%8!=7 && j>=0; j-=9) { validMoves |= (1L << j); if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) break; }
                    for (j=bitRef+7; j%8!=7 && j<=63; j+=7) { 
                        validMoves |= (1L << j); 
                        if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) 
                            break; 
                    }
                    for (j=bitRef-7; j%8!=0 && j>=0; j-=7) { 
                        validMoves |= (1L << j); 
                        if ((occupancyVariation[bitRef][i] & (1L << j)) != 0) 
                            break; 
                    }
 
                    magicMovesBishop[bitRef][magicIndex] = validMoves;
                }
            }
        }
    }

{ 2 comments }

There’s more than one way to do this, three of which are:

  • Define an interface for each dependency and define in your config the name of the concrete class that should be automatically instantiated by dependency injection.
  • Create a Factory class, implementing Zend\ServiceManager\FactoryInterface and reference the factory in your controller configuration.
  • Define a factory closure in Module.php

Here I’ll describe the third of these, as this is my preferred approach.

In your Module.php

1
2
3
4
5
6
7
8
9
10
11
12
public function getControllerConfig()
{
    return array(
        'factories' => array(
            'Course' => function(ControllerManager $cm) {
                return new Controller\CourseController(
                    $cm->getServiceLocator()->get('CourseService')
                );
            },
        ),
    );
}

Here, the ‘Course’ refers to the controller you define for your routes in module.config.php, e.g.

1
2
3
4
5
6
7
8
9
10
11
12
13
'delete-course' => array(
    'type'    => 'Segment',
    'options' => array(
        'route'    => '/delete[/:courseId]',
        'constraints' => array(
            'courseId' => '[0-9]+'
        ),
        'defaults' => array(
            'controller' => 'Course',
            'action'     => 'delete',
        ),
    ),
),

You should ensure that you don’t have ‘Course’ listed in module.config.php in your invokables section, e.g.

1
2
3
4
5
6
7
8
9
10
'controllers' => array(
    'invokables' => array(
        /****************************************************************************************
        DON'T DO THIS! - You should remove the reference you may already have in this section,
        otherwise, ZF2 will try to instantiate your controller without any arguments and you'll
        get an error.
        *****************************************************************************************/
        'Course' => 'Application\Controller\CourseController',
    ),
),

The ‘CourseService’, on the other hand, does need to be defined as an invokable in your module.config.php in the service_manager section, e.g.

1
2
3
4
5
6
7
8
9
10
11
12
'service_manager' => array(
    'abstract_factories' => array(
        'Zend\Cache\Service\StorageCacheAbstractServiceFactory',
        'Zend\Log\LoggerAbstractServiceFactory',
    ),
    'aliases' => array(
        'translator' => 'MvcTranslator',
    ),
    'invokables' => array(
        'CourseService' => 'Application\Service\CourseService',
    ),
),

Once this is done, the following code should assign an instance of CourseService to the member variable of the CourseController:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
class CourseController extends NetsensiaActionController
{
    /**
     * @var Application\Service\CourseService
     */
    private $courseService;
 
    public function __construct(
	   CourseService $courseService
    )
    {
        $this->courseService = $courseService;
    }
}

{ 1 comment }

While playing with the Zend Framework 2 Skeleton application, I decided finally to get to grips with using translation for all strings within my web applications.

One of the first things you notice while playing around with the ZF2 Skeleton is pieces of code like the following in the layout.phtml file

1
echo $this->translate('My high-quality, well-tested web application');

If you are like me, at this point you will vow to wrap all your strings in this apparently-magical function translate(). You will then forget to do so and end up going back through your code later to do it. At which point, you’ll decide to experiment with modifying one of the translation files that you spotted in the module/Application/language directory.  The ones that come in pairs like this:

fr_FR.mo
fr_FR.po

Poedit

The translations are held in the “po” version of these couplets.   You need not worry about the “mo” – that will be generated automatically.

Poedit is one of many editors that you can use to edit the “po” files.  Download it here.

One of the first things you’ll try to do after installing the software is to open up a PO file and start editing the source strings.

You won’t be able to! You’re not supposed to do this. You’re supposed to get Poedit to extract the strings for you by scanning your source code for all occurrences of the translate() function.

xgettext

To extract the strings, Poedit uses xgettext from the gettext binaries. It can be configured to use any parser, but you’ll find that the default settings are to use xgettext. You can see these settings by looking at File | Preferences | Parsers and selecting PHP.

Oh, and while you’re on that page, add *.phtml to the list of files to be parsed, so that you have *.php as well *.phtml.

Now, open your PO file, and choose Catalog | Properties | Source Paths and add the path within which you want to scan for text to be translated.

Now choose the Source Keywords tab and add translate as one of the Keywords (you can delete any other keywords as we are only extracting text being passed to the translate function).

Update your PO

At this point, you should be able to click the Update icon and your ZF2 code will be scanned, and your translation source texts extracted.

Your MO file will be created when you save your PO.

Setting the locale in Zend Framework 2

Here is one way you can set up your locale. In Module.php:

1
2
3
4
5
6
7
8
9
10
    public function onBootstrap(MvcEvent $e)
    {
        ...
        ...
 
        $translator = $e-&gt;getApplication()-&gt;getServiceManager()-&gt;get('translator');
        $translator
            -&gt;setLocale(\Locale::acceptFromHttp($_SERVER['HTTP_ACCEPT_LANGUAGE']))
            -&gt;setFallbackLocale('en_GB');        
    }

And, of course, you can use similar code to respond to a user request to change the current language.

{ 0 comments }

If you are using Jenkins to build multiple projects and you have tried using deployment keys, you will have noticed that you need a new deployment key for each project in services such as GitHub and BitBucket.  This creates a difficulty when configuring your Jenkins builds, because the jenkins user always pulls with the same SSH key.

There is a pretty easy solution to this, but I always seem to trip up when implementing it on a new server, so here it is, step by step.

1. Create deployment keys for your projects (don’t use a passphrase)

ssh-keygen -t rsa -f /var/lib/jenkins/.ssh/id_rsa_MY_FIRST_PROJECT
ssh-keygen -t rsa -f /var/lib/jenkins/.ssh/id_rsa_MY_SECOND_PROJECT

Ensure that your keys are owned by the jenkins user.

2. Add the public part of each key pair to its associated project

Login to your GitHub/BitBucket/Other account, then, for each repository find the Deployment Keys area and add the public part of your key pair for that repository.

3. Create an SSH config file in the jenkins user’s ~/.ssh directory

sudo vi /var/lib/jenkins/.ssh/config

4. In this file, add an entry for each project, such as those below

Host github-my-first-project
HostName bitbucket.org
User git
IdentityFile ~/.ssh/id_rsa_MY_FIRST_PROJECT

Host github-my-second-project
HostName bitbucket.org
User git
IdentityFile ~/.ssh/id_rsa_MY_SECOND_PROJECT

This has created per-project domain aliases for, in this case, bitbucket.org, with each domain alias being associated with its own private key.

5. Log in as the jenkins user and force the creation of the known hosts file

sudo su -l -p jenkins
ssh git@bitbucket-my-first-project

6. You can now set up your jenkins to pull from your repositories as

git@bitbucket-my-first-project:Username/my-first-project.git
git@bitbucket-my-second-project:Username/my-second-project.git

{ 2 comments }