I find myself using the client functionality of the gearman tool most often. If I am tasked with creating or updating a gearman worker I want to test that the worker actually works. The code that sends a job to the worker is normally part of the web application and I don't want to dig through the application trying to figure out what I need to do send the job out. I could just create a simple php script that creates a client and sends the job over, but the gearman tool already does this.

Example of using gearman as a client:

I use the gearman program as a worker less often. It is still useful for creating a simple worker to test my client code against. I can write my client code without a fully functional worker if the client code is not expecting a complex response.

Example of using gearman as a worker:

The gearman tool comes with the standard options for specifying a specific host and port. There are a number of other options that may be of use in specific circumstances. I encourage you to read them over by typing "gearman -H" on the command line.

Declaring a global functions in a gearman worker script may not seem like a big deal, but these things have a way of catching up to you. I personally ran into this when I suggested that HauteLook start using GearmanManager to manage the gearman workers. A side affect of this is that all gearman worker scripts now run in the same instance of PHP. There were a number of occasions when workers would fail to load on production because of the global naming conflicts.

I prefer to use register anonymous functions with my gearman workers. This keeps them out of the global scope and puts the logic right next to the worker registration call. This makes hard to test the logic inside the anonymous function, but I never test that logic. I treat the functions I register with gearman as controllers. I pass the workload off to a model class that is fully unit tested.

Here is a nice example:

I would update the Gearman examples on php.net, but I think a lot of people are still using PHP 5.2 (or even earlier). Providing multiple ways of registering the workers may confuse people too. Maybe next year.

The release of pecl/gearman 1.0 included a fix to the exception handling of gearman workers. An uncaught exception would go unnoticed by the gearmand server before pecl/gearman 1.0. The only way to work around this was to catch all exception and explicitly return a failure code to gearmand. I fixed this by making by detecting an exception with the pecl/gearman code and sending a special exception message to the gearmand server. Unfortunately, the API for sending the exception message was introduced in gearman 0.21.

I did not think much of the new gearmand requirement when fixing this bug. Since that time I have been introduced to the painful upgrade process of gearmand. I think this pain is mostly felt by those using Linux distributions with long term support. The first hurdle to get over is the fact that gearmand 0.21 requires the boost 1.41 libraries. A default Cent OS install only provides boost 1.39 libraries. There was a 3rd party that provided boost 1.41 libraries for Cent OS, but they used a non-standard installation directory. After some experimenting, I did find a fix: http://groups.google.com/group/gearman/msg/12bcc13691e132ae.

The good news is that upgrading to Cent OS 6 makes compiling gearman much easier with the inclusion of boost 1.41 libraries by default. The next problem comes when trying to submit a patch. Cent OS 5 comes with aclocal 1.9 and the gearman project requires aclocal 1.11. Again, upgrading to Cent OS 6 makes life easier as it comes with aclocal 1.11 by default. I learned the hard way that the gearman project also requires autoconf 2.64, but Cent OS 6 only provides autoconf 2.63. The 2.64 version of autoconf provides a macro named m4_ifnblank which is required to build the configure script used to compile gearman. You can see me answer my own question here: https://answers.launchpad.net/gearmand/+question/185592 regarding the autoconf issue.

These build issues have made it tough for users to upgrade to the pecl/gearman 1.x line. I think I will be forced to maintain the pecl/gearman 0.8.x line for some time with bug fixes as users struggle to get gearmand 0.21 on their Cent OS 5 servers. I have definitely learned to do more research on migration paths before requiring a newer dependency version.

This is a pretty common use case and I am sure most people would write the logic something like this:

There is nothing logically wrong with this code. It is a very readable and easy to understand way to write it. With the rise in popularity of JavaScript and other functional languages, some of us may take a different approach. Using the underscore framework (available for both PHP and JavaScript), you could also write it like this:

The use of a callback function should be familiar to anyone with at least a basic knowledge of JavaScript. The callback function is evaluating each item in the iterator to true or false. The major drawback to both of the code snippets is that the logic for determining whether or not an item is sold out is not able to be re-used. The use of procedural code in the first example and the use of an anonymous function in the second example also make it hard to test. We can improve the second example by not using an anonymous callback:

Now I can re-use this function and easily test it. We can use SPL FilterIterator in a very similar way to the functional example:

Now my logic for what constitutes a soldout item is isolated in a class. This coincides with the object oriented principle of "encapsulate what varies". I think the biggest stumbling block for using SPL is that it just seems too heavy. You might be wondering why you should go to the trouble of creating a new class to perform such a simple task. The procedural example above is faster to write. Some of us might be tempted to use the functional example with an anonymous function because it feels more "expressive". Now consider what happens when the ecommerce system introduces returns. The new formula for determining a soldout item is now:


availability = (purchased - sold) + returned


This isn't some hypothetical example, it actually happened. It was real easy to update the logic to handle returns and make sure the tests passed.

The last decision you have to make is whether or not to put the logic in a database query. If there is a huge performance boost by writing the logic into the query, it may be worthwhile. A query is still testable, but you have to setup some test data in the database in order to test it (which means you probably won't do write the test for it). It is harder to re-use queries, so the business logic for determining sold out items may be duplicated over a number of queries too. You also might want to consider what happens if you decide to alleviate the database load by putting the list of items for sale in a cache (like APC or memcache).

The SPL classes, especially FilterIterator, really start to shine when dealing with a dataset outside of our control. More and more platforms are becoming service based and we have no control over how the data comes back. Especially when you consider something like the Twitter API timeline response and trying to filter out any tweet that starts with "RT".

In order to make development of the gearman workers easier, I created a script to startup the gearman worker in a similar manner to the GearmanManager. The major benefit is that any sort of error or exception is shown right on the terminal screen. No digging through logs wasting time tying to figure out what happened. If you have logging already setup, it is now trivial to add a stdout writer to the logger when running php in cli mode.

One of the biggest complaints I heard from fellow developers was that they had a hard time trying to figure out if the worker actually received the job from the client. I added a logging to stdout that signals when a job is received and what a job is finished.

I setup a pear channel to make installation real easy. For installation instructions and some examples check out the github page page: https://github.com/hradtke/gearboot

Like Vic, I too have moved away from using ORM frameworks. One of my biggest problems with ORM frameworks is that they become a de-facto language within themselves. Instead of learning how to write proper SQL according to the SQL-99 standard, I now have to learn SQL and the syntax of the ORM. Things only get worse from there when trying to debug problems of performance and correctness.

There are some niche tasks where I still use an ORM despite the costs mentioned by Vic and myself. An ORM is great for creating dynamic queries. As we build more rich front-end web applications, it is very easy to build queries with user specified filters and sorting. Trying to do this manually is time-consuming and often bug prone. I find this setup particularly useful when building administrative pages that are looking at lists of things like users or orders. Performance is not a real big issue here and the queries themselves are usually only joining one or two tables. There is still the risk of a runaway query wreaking havoc on your master database. To combat this scare, I advocate a separate slave database to run queries like this.

Vic mentioned that he puts the SQL directory in the controllers, but I think this is a mistake. Controllers are tying the view to the model and should not contain any other logic. I prefer to keep business logic at the library level and data access logic in the model. The controller can consume library or model classes at will and pass them over to the view for output. This may seem like an extra step that is not needed, but it makes testing much easier. There is also the added benefit of being able to write a service layer or API with very little effort should the need arise.

All in all I think Vic makes a good argument. I think developers need to learn to embrace databases, SQL or NoSQL, instead of trying to abstract them away. Data is often the most important part of the business and it should be treated as such.