{"_id":"compute-cluster","_rev":"447530","name":"compute-cluster","description":"Local process cluster management for distributed computation","dist-tags":{"latest":"0.0.9"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"time":{"modified":"2021-08-05T06:51:45.000Z","created":"2011-11-30T20:11:13.083Z","0.0.9":"2014-07-16T21:09:52.279Z","0.0.7":"2012-05-26T05:25:32.104Z","0.0.6":"2011-12-31T00:53:31.501Z","0.0.5":"2011-12-02T23:46:37.306Z","0.0.4":"2011-11-30T22:30:08.073Z","0.0.3":"2011-11-30T21:09:01.820Z","0.0.2":"2011-11-30T20:46:44.560Z","0.0.1":"2011-11-30T20:11:13.083Z"},"users":{"antixrist":true,"dgarlitt":true,"linxingfang":true,"kcando":true},"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"repository":{"type":"git","url":"https://github.com/lloyd/node-compute-cluster.git"},"versions":{"0.0.9":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.9","repository":{"type":"git","url":"https://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"bugs":{"url":"https://github.com/lloyd/node-compute-cluster/issues"},"licenses":{"type":"MIT","url":"https://raw.github.com/lloyd/node-compute-cluster/master/LICENSE"},"gitHead":"f3b6557969d69e03f62c6d86d63d71037f0f41ee","homepage":"https://github.com/lloyd/node-compute-cluster","_id":"compute-cluster@0.0.9","_shasum":"f862282b420bf76f3ff9756e00d16ed5333c1fed","_from":".","_npmVersion":"1.5.0-alpha-1","_npmUser":{"name":"dannycoates","email":"dannycoates@gmail.com"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"dist":{"shasum":"f862282b420bf76f3ff9756e00d16ed5333c1fed","size":7619,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.9.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.9.tgz"},"directories":{},"publish_time":1405544992279,"_cnpm_publish_time":1405544992279,"_hasShrinkwrap":false},"0.0.7":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.7","repository":{"type":"git","url":"git://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.7","optionalDependencies":{},"_engineSupported":true,"_npmVersion":"1.1.21","_nodeVersion":"v0.6.18","_defaultsLoaded":true,"dist":{"shasum":"3dc64830ef30a20b986f200c1efe13b5c5ab37ce","size":7633,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.7.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.7.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1338009932104,"_cnpm_publish_time":1338009932104,"_hasShrinkwrap":false},"0.0.6":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.6","repository":{"type":"git","url":"git://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.6","_engineSupported":true,"_npmVersion":"1.0.106","_nodeVersion":"v0.6.5","_defaultsLoaded":true,"dist":{"shasum":"8e7a52d9bfcd46ee65e2a084c32ce145ff6c3f79","size":10240,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.6.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.6.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1325292811501,"_cnpm_publish_time":1325292811501,"_hasShrinkwrap":false},"0.0.5":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.5","repository":{"type":"git","url":"git://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.5","_engineSupported":true,"_npmVersion":"1.1.0-alpha-2","_nodeVersion":"v0.6.3","_defaultsLoaded":true,"dist":{"shasum":"69cd4c74d00adbaf159d9725ab5ced2538fca27e","size":6595,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.5.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.5.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1322869597306,"_cnpm_publish_time":1322869597306,"_hasShrinkwrap":false},"0.0.4":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.4","repository":{"type":"git","url":"git://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.4","_engineSupported":true,"_npmVersion":"1.1.0-alpha-2","_nodeVersion":"v0.6.3","_defaultsLoaded":true,"dist":{"shasum":"123e3a97c7ec77c2bfbac725eb25c16718ea1478","size":6532,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.4.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.4.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1322692208073,"_cnpm_publish_time":1322692208073,"_hasShrinkwrap":false},"0.0.3":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.3","repository":{"type":"git","url":"git://github.com/lloyd/node-compute-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.3","_engineSupported":true,"_npmVersion":"1.1.0-alpha-2","_nodeVersion":"v0.6.3","_defaultsLoaded":true,"dist":{"shasum":"3a078a85c787e5c1d3b1e5a9135780c7920772fa","size":6265,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.3.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.3.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1322687341820,"_cnpm_publish_time":1322687341820,"_hasShrinkwrap":false},"0.0.2":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.2","repository":{"type":"git","url":"git://github.com/lloyd/node-process-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.2","_engineSupported":true,"_npmVersion":"1.1.0-alpha-2","_nodeVersion":"v0.6.3","_defaultsLoaded":true,"dist":{"shasum":"ba4c31d6527764d987cc0ca1335a9841051277c5","size":6265,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.2.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.2.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1322686004560,"_cnpm_publish_time":1322686004560,"_hasShrinkwrap":false},"0.0.1":{"author":{"name":"Lloyd Hilaiel","email":"lloyd@hilaiel.com","url":"http://lloyd.io"},"name":"compute-cluster","description":"Local process cluster management for distributed computation","version":"0.0.1","repository":{"type":"git","url":"git://github.com/lloyd/node-process-cluster.git"},"main":"lib/compute-cluster","engines":{"node":">= 0.6.2"},"dependencies":{"vows":"0.6.0"},"devDependencies":{},"scripts":{"test":"vows"},"_npmUser":{"name":"lloyd","email":"lloyd@hilaiel.com"},"_id":"compute-cluster@0.0.1","_engineSupported":true,"_npmVersion":"1.1.0-alpha-2","_nodeVersion":"v0.6.3","_defaultsLoaded":true,"dist":{"shasum":"53ddae16fc40a04d318b053d9800e78a28c95c55","size":6240,"noattachment":false,"key":"/compute-cluster/-/compute-cluster-0.0.1.tgz","tarball":"http://registry.cnpm.dingdandao.com/compute-cluster/download/compute-cluster-0.0.1.tgz"},"maintainers":[{"name":"chilts","email":"andychilton@gmail.com"},{"name":"dannycoates","email":"dannycoates@gmail.com"},{"name":"gene_wood","email":"npmjs.org@ewood.users.cementhorizon.com"},{"name":"lloyd","email":"lloyd@hilaiel.com"},{"name":"seanmonstar","email":"sean.monstar@gmail.com"}],"directories":{},"publish_time":1322683873083,"_cnpm_publish_time":1322683873083,"_hasShrinkwrap":false}},"readme":"## Distributed Computation for NodeJS\n\n[![Build Status](https://secure.travis-ci.org/lloyd/node-compute-cluster.png)](http://travis-ci.org/lloyd/node-compute-cluster)\n\nHow can you build a responsive and robust nodejs server that does some heavy\ncomputational lifting?  Some node libraries (like the awesome [node-bcrypt][])\ndo their own threading internally and combine that with an async API.  This\nallows libraries to internally thread their calls and use multiple cores.\n\n  [node-bcrypt]: https://github.com/ncb000gt/node.bcrypt.js\n\nWhile this is pretty awesome, it is significant work for library implementors,\nand as this pattern becomes rampant, the application author loses fine grained\ncontrol over the resource usage of their server as well as the relative priority\nof compute tasks.\n\nIf you just naively run computation on the main evaluation thread, you're blocking\nnode.js from doing *anything else* and making your whole server unresponsive.\n\n## The solution?\n\n`node-compute-cluster` is a tiny abstraction around a group of\nprocesses and the [built-in IPC][] introduced in NodeJS 0.6.x.  It provides a simple\nAPI by which you can allocate and run work on a cluster of computation processes.\nThis allows you to perform multiprocessing at a more granular level, and produce\na responsive yet efficient computation server.\n\n [built-in IPC]: http://nodejs.org/docs/v0.6.3/api/all.html#child_process.fork\n\n## Installation\n\n``` sh\n$ npm install compute-cluster\n```\n\n## Usage\n\nFirst you write your main program:\n\n``` js\nconst computecluster = require('compute-cluster');\n\n// allocate a compute cluster\nvar cc = new computecluster({\n  module: './worker.js'\n});\n\nvar toRun = 10\n\n// then you can perform work in parallel\nfor (var i = 0; i < toRun; i++) {\n  cc.enqueue({}, function(err, r) {\n    if (err) console.log(\"an error occured:\", err);\n    else console.log(\"it's nice:\", r);\n    if (--toRun === 0) cc.exit();\n  });\n};\n```\n\nNext you write your `worker.js` program:\n\n``` js\nprocess.on('message', function(m) {\n  for (var i = 0; i < 100000000; i++);\n  process.send('complete');\n});\n```\n\nAll done!  Now you're distributing your computational load across multiple processes.\n\n## API\n\n### Constructor - `new require('compute-cluster')(<options>);`\n\nAllocates a computation cluster.  Options include:\n\n  * `module` - **required** the path to the module to load\n  * `max_processes` - the maximum number of processes to spawn (default is `ciel(#cpus * 1.25)`)\n  * `max_backlog` - the maximum length of the backlog, -1 indicates no limit (default is 10 * max_processes)\n                    an error will be returned when max backlog is hit.\n  * `max_request_time` - the maximum amount of time a request should take, in seconds.  An error will be returned when we expect a request will take longer.\n\nExample:\n\n``` js\nvar cc = new require('compute-cluster')({\n  module: './foo.js',\n  max_backlog: -1\n});\n```\n\n### Event: 'error'\n\nAn error event will be emited in exceptional circumstances.  Like if a child crashes.\nCatch error events like this:\n\n``` js\ncc.on('error', function(e) { console.log('OMG!', e); });\n```\n\nDefault behavior is to exit on error if you don't catch.\n\n### Events: 'debug' or 'info'\n\nEvents raise that hold an english, developer readable string describing\nthe state of the implementation.\n\n### cc.enqueue(<args>, [cb])\n\nenqueue a job to be run on the next available compute process, spawning one\nif required (and `max_processes` isn't hit).\n\nargs will be passed into the process (available via `process.on('message', ...)`).\n\n`cb` is optional, and will be invoked with two params, `err` and `response`.\n`err` indicates hard errors, response indicates successful roundtrip to the\ncompute process and is whatever the decided to `process.send()` in response. \n\n### cc.exit([cb])\n\nKill all child processes, invoking callback (with err param) when complete.\n\n## LICENSE\n\nCopyright (c) 2011, Lloyd Hilaiel <lloyd@hilaiel.com>\n\nPermission to use, copy, modify, and/or distribute this software for any\npurpose with or without fee is hereby granted, provided that the above\ncopyright notice and this permission notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\nWITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\nANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF\nOR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n","_attachments":{},"homepage":"https://github.com/lloyd/node-compute-cluster","bugs":{"url":"https://github.com/lloyd/node-compute-cluster/issues"}}