Native parallel PHP job queue - Kore Nordmann - PHP / Projects / Politics

Kore Nordmann - PHP / Projects / Politics

By Kore Nordmann, first published at Thu, 06 May 2010 17:42:50 +0200

Native parallel PHP job queue

To make use of multiple cores for some rather long processing operations I needed a way to fork multiple workers from a single PHP script multiple times lately. So I created a small project on github which implements this in a way, so that it should reusable by anybody. This is far from being rocket science, but still might be useful to someone out there.

Native job queue

I also implemented a ShellJobProvider (which implements JobProvider), which is constructed from an array of shell commands, which then are executed in parallel. A simple working example:

<?php require 'njq/environment.php'; $executor = new \njq\Executor(); $executor->run( new \njq\ShellJobProvider( array( 'echo 1 >> test', 'echo 2 >> test', 'echo 3 >> test', 'echo 4 >> test', 'echo 5 >> test', ) ), 4 ); ?>

The file test will then contain something like (the order might vary):

5 4 3 2 1

The 4 (second parameter of \njq\Executor::run) defines the number of parallel processes to spawn. This should not exceed the number of available cores in the most cases.

Requirements

The job queue requires PHP 5.3 and the PHP PCNTL extension.

Get it

I like github for dropping projects I do not want to maintain very actively, so this small piece of code is also hosted on github: http://github.com/kore/njq - happy forking. It, of course, has documentation and tests - as always.

If you liked this blog post, or learned something please consider using flattr to contribute back: .

Comments

Add new comment

Fields with bold names are mandatory.

eZ Components

eZ Components

Exploring PHP

Exploring PHP

Hire me

Amazon wishlist

Powered by