First published at Thursday, 6 May 2010
Warning: This blog post is more then 12 years old – read and use with care.
Native parallel PHP job queue
To make use of multiple cores for some rather long processing operations I needed a way to fork multiple workers from a single PHP script multiple times lately. So I created a small project on github which implements this in a way, so that it should reusable by anybody. This is far from being rocket science, but still might be useful to someone out there.
Native job queue
I also implemented a
ShellJobProvider (which implements
JobProvider), which is constructed from an array of shell commands, which then are executed in parallel. A simple working example:
<?php require 'njq/environment.php'; $executor = new \njq\Executor(); $executor->run( new \njq\ShellJobProvider( array( 'echo 1 >> test', 'echo 2 >> test', 'echo 3 >> test', 'echo 4 >> test', 'echo 5 >> test', ) ), 4 ); ?>
test will then contain something like (the order might vary):
5 4 3 2 1
4 (second parameter of
\njq\Executor::run) defines the number of parallel processes to spawn. This should not exceed the number of available cores in the most cases.
The job queue requires PHP 5.3 and the PHP PCNTL extension.
I like github for dropping projects I do not want to maintain very actively, so this small piece of code is also hosted on github: http://github.com/kore/njq - happy forking. It, of course, has documentation and tests - as always.