Skip to content
This repository has been archived by the owner on Mar 23, 2023. It is now read-only.

Commit

Permalink
Updating documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
wa0x6e committed Jun 20, 2013
1 parent e3e52b7 commit d461ed4
Show file tree
Hide file tree
Showing 3 changed files with 114 additions and 56 deletions.
37 changes: 37 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# How to contribute

## Coding convention

This tool follows [PSR-2 Coding standard](https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-2-coding-style-guide.md).
## Testing

### Development workflow

My development workflow make use of grunt watch plugin, that will run phpunit automatically after each file save. Grunt is also used to generate the test coverage.

### Unit Tests

All pull request should not break existing tests. You're more than welcome to write additional tests. There is a grunt task to run the tests :

grunt phpunit

Without grunt, just run :

phpunit tests

Or you can also run tests after each file edition :

grunt watch

### Test coverage

Use the coverage grunt task to generate code coverage.

grunt coverage

Without grunt, use :

phpunit tests


The coverage reports is in build/coverage/index.html.
130 changes: 75 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#Fresque

Fresque is a command line tool to manage your php-resque workers.
#Fresque [![Build Status](https://travis-ci.org/kamisama/Fresque.png?branch=master)](https://travis-ci.org/kamisama/Fresque) [![Coverage Status](https://coveralls.io/repos/kamisama/Fresque/badge.png?branch=fix-travis)](https://coveralls.io/r/kamisama/Fresque?branch=fix-travis) [![Dependency Status](https://www.versioneye.com/package/php--fresque--fresque/badge.png)](https://www.versioneye.com/package/php--fresque--fresque)

> Fresque is a command line tool to manage your php-resque workers.
##Prerequisites

Expand All @@ -19,18 +18,18 @@ I'll assume in this tutorial that you have sufficient knowledge to start a worke

Fresque is a command line tool to manage your php-resque workers

# Starting a worker
# Start a worker without fresque
$ QUEUE=file_serve php resque.php

# Starting a worker with fresque
$ fresque start -q file_serve

It's more friendly, and provides more options, like `restart`, `stop`, etc …
Php-resque, and resque, by default doesn't provide an out-of-the-box way to stop your workers. You just kill the worker process. With Fresque, you'll enjoy stopping and restarting your workers at gogo. No more system process handling!
Php-resque, and resque, by default doesn't provide an out-of-the-box way to stop your workers. You have to directly kill the worker process. With Fresque, you'll enjoy stopping and restarting your workers at gogo. No more system process handling!

##Installation

Clone the git repo
### By cloning the git repo

$ git clone git://github.com/kamisama/Fresque.git

Expand All @@ -46,58 +45,67 @@ Finally, install dependencies

$ php composer.phar install

##Configuration
### Using Composer

A fresque.ini file is provided to set your workers default parameters, and other options used by fresque, such as redis connection options.
it's well documented, and you shouldn't have difficulties filling it.
If your application is already using Composer, just add Fresque in your composer dependencies

"require": {
"kamisama/fresque": "~1.2.0"
}

##Usage
and update the dependencies with `composer update`

*Some examples are available at the end of this page.*
##Configuration

A fresque.ini file is provided to set the workers default parameters, and other options used by fresque, such as redis connection options.
it's well documented, and you shouldn't have difficulties editing it.

It's often wise to start by adding the fresque executable to your system path.
You then just call
##Usage

For convenience, you should add the fresque bin (located in Fresque/fresque) to your system path.
You can just then call fresque with

$ fresque <command>

Or if you didn't add it,

$ cd /path/to/the/resque/executable
$ cd /path/to/fresque
$ ./fresque <command>

If installed as a composer dependency, it's also available in composer bin folder

$ vendor/bin/fresque <command>

There's a bunch of interesting commands available :

*Examples are available at the end of this section.*

* **start**

To start a new resque worker. By default, it will use the default configuration defined in you fresque.ini for the queue name, the pooling frequency, and other various options. You can override all of them with an option flag. Available options :
To start a new resque worker. By default, it will use the default configuration defined in you fresque.ini for the queue name, the pooling frequency, and other options. You can override all of them with an option flag. Available options :

> `-u` or `--user` : User running the php process. Should be the user running your php application, usually **www-data** for apache. Using a different user could lead to permissions problems in some cases.
> `-u` or `--user` : User running the php process. Should be the user running your php application, usually **www-data** for apache. Using a different user could lead to permissions problems.
>
> `-q` or `--queue` : A list of queues name, separated with a comma.
> `-i` or `--interval` : Pooling frequency. Number of seconds between each pooling.
> `-q` or `--queue` : A list of queues name polled by the worker, separated with a comma.
> `-n` or `--workers` : Number of workers working on the same queue. Uses pcntl to fork the process, ensure that you PHP is compiled with it.
> `-i` or `--interval` : Polling frequency. Number of seconds between each polling.
> `-l` or `--log` : Absolute path to the log file. You can set a different log for each worker. The `--user` must have permission to read and write to that file (and the parent folder, since the file will be created if non-existent).
> `-n` or `--workers` : Number of workers working on the same queues.

For creating multiple queues with different options, just run `start` again.
> `-l` or `--log` : Absolute or relative path to the log file. You can set a different log for each worker. The `--user` must have permission to read and write to that file (and the parent folder, since the file will be created if non-existent).
> Relative path is relative to the fresque folder.

* **stop**

To stop workers. Will wait for all jobs to finish, then stop the worker. If more than one worker is running, you'll have to choose the worker to stop from a worker menu.
To stop workers. Will wait for all jobs to finish, then stop the worker. If more than one worker is running, a list of worker will be displayed, to choose the worker to stop.

> `-f` or `--force` : Force shutdown, without waiting for the jobs to finish. All jobs will fail.
> `-w` or `--all` : Stop all workers, skipping the worker menu.
> `-f` or `--force` : Stop worker immediately, without waiting for the current job to finish processing. This will fail the current job.
> `-w` or `--all` : Stop all workers at once, skipping the worker menu.
* **pause**

To pause workers. Like with `stop`, a you'll be prompted with a worker list if more than one worker is available.
To pause workers. Similary to `stop`, you'll be prompted with a worker list if more than one worker is available.

> `-w` or `--all` : Stop all workers, skipping the worker menu.
Expand All @@ -117,11 +125,11 @@ To start a batch of pre-defined workers (set in your configuration file). See fr

* **stats**

Display total number of failed/processed jobs, as well as various stats for each workers.
Display total number of failed/processed jobs, as well as various stats for each workers and queues.

* **tail**

Tail the workers' logs. If you have more than one log file, you'll have to choose the log to tail from a log file menu.
Tail a worker's log. If you have more than one log file, you'll be prompted with list of log.

* **enqueue**

Expand All @@ -133,12 +141,11 @@ Add a job to a queue. Takes 3 arguments :
Will print the **Job ID** if the job is successfully enqueued.

*Successfully enqueuing a job does not means it will perform it successfully*

*Successfully enqueuing a job does not mean it will perform it successfully*

* **test**

Test your configuration. If no options are provided, it will test you *fresque.ini*. It accepts all options, to let you test them.
Test your configuration. If no options are provided, it will test your *fresque.ini*. It accepts all type of options.

Finally, there's some global options, that can be used for all commands. Default value in your config file will be used unless you use these.

Expand All @@ -155,27 +162,32 @@ Finally, there's some global options, that can be used for all commands. Default

##Examples

Let's start a worker with the default settings defined in the config file (fresque.ini)
Let's start a worker with the default settings defined in the config file (fresque.ini):

$ fresque start

Let's start another worker, named *activity*, with a pooling frequency of 1 second. Also. I want two workers working on this queue.
Let's start another worker, polling the *activity* queue, with a polling frequency of 1 second. Also, we'll want to have two workers working on this queue:

$ fresque start -q activity -i 1 -n 2

Then, I want another worker, that will work on the queues *default* and *activity* at the same time.
If we want another worker, working on the queues *default* and *activity* at the same time:

$ fresque start -q default,activity

Oh wait, I have another resque on another redis server. I want to log its activities in an other log file : remote.log
Oh wait, we have another resque on another redis server. we'll want to log its activities in another log file: remote.log

$ fresque start -s 192.168.1.26 -p 6390 -q myspecialqueue -l /path/to/remote.log

- -s 192.168.1.26 is the address of the redis server
- -p 6390 is the redis server port
- -q is the queuename
- -l is the path to the log file

If you have your config file elsewhere, and your php-resque lib elsewhere also

$ fresque start -c /path/to/my-config.ini -b /path/to/my/custom/php-resque

To view stats of your workers (to know how many you have, processed/failed jobs count, uptime)
To view stats of your workers (to know how many you have, processed/failed jobs count, uptime, etc ...)

$ fresque stats

Expand Down Expand Up @@ -232,13 +244,14 @@ It should output something like that
Remember that you can use the global options (-s, -p etc …) with any command

$ fresque stop -c /path/to/my-config.ini -s 192.168.1.26

Let's enqueue a job to the *activity* queue

$ fresque enqueue activity PageVisit 5 /index.php 158745693
$ fresque enqueue activity PageVisit "5,/index.php,158745693"

php-resque will then run this job, by instantiating the class `PageVisit`, then calling the `perform()` method with the arguments `5`, `/index.php` and `158745693`.
In order to instantiate the PageVisit class, php-resque should know where to find it. That should be done with you application autoloader (`--include`)

In order to instantiate the PageVisit class, php-resque should know where to find it. That should be done inside your application autoloader (either by using an autoloader, of an include/require).

Oh, and if you want to restart all your workers for whatever reasons

Expand All @@ -261,9 +274,7 @@ It'll spout something like that
Killing 33233 … Done
Killing 33223 … Done

See notes if it says **There were no active workers to kill …** but you're sure there are some.

###Starting all the workers at once
###Starting all your favorites workers at once

We've just created 6 workers, calling `start` 5 times (remember, the second `start` create 2 workers with `-n 2`). But there's a way to start all of them with only one command, useful when you have a lot of workers that you have to start each time.

Expand All @@ -276,24 +287,22 @@ Just set all your workers settings in the config file in the [Queues] section (w

###You can test your config with `test`

A testing tool for testing your configuration file is provided, just call `test`.
This will test the minimum requirements to run fresque :
A testing tool for testing your configuration file is provided, by calling `test`.
It will test the minimum requirements to run fresque :

* Your redis hostname and port are not null
* You can connect to the redis server with the specified hostname:port
* The log file exist and is writeable, or if it doesn't exists, the parent directory is writeable
* The path to the php-resque libraries is valid
* The path to your application autoloader is valid

This will not test the content of your application autoloader, so if there's something inside triggering a fatal error, fresque will not know, and return a *success*, but the worker will fail to start, like we just said above.

You can test more than the settings inside your config file, by passing options. An option will override the setting defined in the config

$ fresque test -s 195.168.1.26 -p 6890

This will test your config file, but with the specified redis hostname and port.

You can also test an other config file
You can also test another config file

$ fresque test -c /my/other/config/file.ini

Expand All @@ -313,30 +322,41 @@ A test result will looks like

##Known issues

###`stop` command doesn't stop workers
###`stop` command doesn't behave as expected

#### Case 1: it says 'no such process' when stopping a worker

This happens when the saved list of workers is not synchronized with the real state of the workers on the system.
A worker was stopped, and fresque wasn't notified.

The main and usual cause is that you didn't stop the workers with the `stop` command, like when you reboot the machine, without stopping the workers beforehand.

**Solution:** Stop all your workers using `stop --all`, then use `reset` to clear fresque cache, and start your workers again.

#### Case 2: it says 'no permission'

This happens when you try to stop your workers, but it says **There were no active workers to kill …**. And a look with `stats` will tell you that there are. This occurs when the `--user` doesn't have sufficient permissions to kill the worker process, or the worker PID was 'changed'. In some weird case, the worker fork a child to execute the job, but doesn't switch back to the parent job after. The child become the parent worker, but doesn't notify Resque, and the worker list is outdated.
You don't have permission to manipulate the worker process.

The only way to kill these 'stray' worker are to find their pid and kill them manually.
**Solution:** You have to start and stop the worker with the same user. If you used `start --user=www-data` to start your worker, use `stop --user=www-data` to stop it. To avoid that type of problem, always use the same user for all your workers (usually the apache user), and set it by default in the fresque.ini.

ps aux | grep resque.php
#### Case 3: the stopped worker sometime still appear in the worker list for a few seconds

It will print a list a process. Find the PID of the stray workers and kill them
You just used `stop`, and it stopped a worker. You immediately use `stop` again, and the worker is still here.

sudo kill YOUR_PID
**Solution:** Do nothing. It's perfectly normal, stopping a worker can take some time (it'll wait for the job to finish, or wait for the next polling to stop).

##Notes

###Consult your logs

Logs tell you all you need to know about the issue of a job, and the current status of your php-resque workers. It tells you when a job is enqueued, when a job is about to being performed, and its issue (success/fail). It also display all php related errors that may occurs.
Logs tell you all you need to know about the issue of a job, and the current status of your php-resque workers. It tells you when a job is enqueued, when a job is about to being performed, and its final state (success/fail). It also display all php related errors that may occurs.
Check them frequently, as fresque doesn't capture those errors.


##Background

Fresque is a derivated works from my other plugin, [cake-resque](https://github.com/kamisama/Cake-Resque), a command line tool to manage php-resque, but inside cakephp console.
Very convenient, but limited to only cakephp framework. I wanted to release a tool that can work in any php environment.
Very convenient, but limited to only cakephp framework. I wanted to release a tool that can work anywhere.

##Credits

Expand Down
3 changes: 2 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,6 @@
"email": "kami@kamisama.me",
"issues": "https://github.com/kamisama/Fresque/issues",
"source": "https://github.com/kamisama/Fresque"
}
},
"bin": ["fresque"]
}

0 comments on commit d461ed4

Please sign in to comment.