Native parallel PHP job queue
First published at Thursday 6 May 2010
Warning: This blog post is more then 14 years old – read and use with care.
Native parallel PHP job queue
To make use of multiple cores for some rather long processing operations I needed a way to fork multiple workers from a single PHP script multiple times lately. So I created a small project on github which implements this in a way, so that it should reusable by anybody. This is far from being rocket science, but still might be useful to someone out there.
Native job queue
I also implemented a ShellJobProvider (which implements JobProvider), which is constructed from an array of shell commands, which then are executed in parallel. A simple working example:
<?php
require 'njq/environment.php';
$executor = new \njq\Executor();
$executor->run(
new \njq\ShellJobProvider( array(
'echo 1 >> test',
'echo 2 >> test',
'echo 3 >> test',
'echo 4 >> test',
'echo 5 >> test',
) ),
4
);
?>The file test will then contain something like (the order might vary):
5
4
3
2
1The 4 (second parameter of \njq\Executor::run) defines the number of parallel processes to spawn. This should not exceed the number of available cores in the most cases.
Requirements
The job queue requires PHP 5.3 and the PHP PCNTL extension.
Get it
I like github for dropping projects I do not want to maintain very actively, so this small piece of code is also hosted on github: http://github.com/kore/njq - happy forking. It, of course, has documentation and tests - as always.
Subscribe to updates
There are multiple ways to stay updated with new posts on my blog: