Changeset View
Changeset View
Standalone View
Standalone View
src/docs/user/field/worker_queue.diviner
@title Managing the Worker Queue | @title Managing the Worker Queue | ||||
@group fieldmanual | @group fieldmanual | ||||
Advanced guide to managing the background worker task queue. | Advanced guide to managing the background worker task queue. | ||||
Overview | Overview | ||||
======== | ======== | ||||
Phabricator uses daemonized worker processes to execute some tasks (like | Phorge uses daemonized worker processes to execute some tasks (like | ||||
importing repositories and sending mail) in the background. | importing repositories and sending mail) in the background. | ||||
In most cases, this queue will automatically execute tasks in an appropriate | In most cases, this queue will automatically execute tasks in an appropriate | ||||
order. However, in some cases you may want to exercise greater control over | order. However, in some cases you may want to exercise greater control over | ||||
which tasks execute, when, and at what priority. | which tasks execute, when, and at what priority. | ||||
Reference: Priority Levels | Reference: Priority Levels | ||||
========================== | ========================== | ||||
Tasks queued by Phabricator use these default priority levels: | Tasks queued by Phorge use these default priority levels: | ||||
| Priority | Name | Tasks | | | Priority | Name | Tasks | | ||||
|---|---|---| | |---|---|---| | ||||
| 1000 | `ALERTS` | Time-sensitive notifications and email. | | | 1000 | `ALERTS` | Time-sensitive notifications and email. | | ||||
| 2000 | `DEFAULT` | Normal publishing and processing. | | | 2000 | `DEFAULT` | Normal publishing and processing. | | ||||
| 2500 | `COMMIT` | Import of commits in existing repositories. | | | 2500 | `COMMIT` | Import of commits in existing repositories. | | ||||
| 3000 | `BULK` | Edits applied via "Bulk Edit" interface. | | | 3000 | `BULK` | Edits applied via "Bulk Edit" interface. | | ||||
| 3500 | `INDEX` | Search engine index updates. | | | 3500 | `INDEX` | Search engine index updates. | | ||||
| 4000 | `IMPORT` | Import of commits in new repositories. | | | 4000 | `IMPORT` | Import of commits in new repositories. | | ||||
Tasks with smaller priority numbers execute before tasks with larger priority | Tasks with smaller priority numbers execute before tasks with larger priority | ||||
numbers (for example, a task with priority 1000 will execute before a task | numbers (for example, a task with priority 1000 will execute before a task | ||||
with priority 2000). | with priority 2000). | ||||
Any positive integer is a valid priority level, and if you adjust the priority | Any positive integer is a valid priority level, and if you adjust the priority | ||||
of tasks with `bin/worker priority` you may select any level even if | of tasks with `bin/worker priority` you may select any level even if | ||||
Phabricator would never naturally queue tasks at that level. For example, you | Phorge would never naturally queue tasks at that level. For example, you | ||||
may adjust tasks to priority `5678`, which will make them execute after all | may adjust tasks to priority `5678`, which will make them execute after all | ||||
other types of natural tasks. | other types of natural tasks. | ||||
Although tasks usually execute in priority order, task execution order is not | Although tasks usually execute in priority order, task execution order is not | ||||
strictly a function of priority, and task priority does not guarantee execution | strictly a function of priority, and task priority does not guarantee execution | ||||
order. | order. | ||||
Large Repository Imports | Large Repository Imports | ||||
======================== | ======================== | ||||
The most common case where you may want to make an adjustment to the default | The most common case where you may want to make an adjustment to the default | ||||
behavior of the worker queue is when importing a very large repository like | behavior of the worker queue is when importing a very large repository like | ||||
the Linux kernel. | the Linux kernel. | ||||
Although Phabricator will automatically process imports of new repositories at | Although Phorge will automatically process imports of new repositories at | ||||
a lower priority level than all other non-import tasks, you may still run into | a lower priority level than all other non-import tasks, you may still run into | ||||
issues like these: | issues like these: | ||||
- You may also want to import one or more //other// new repositories, and | - You may also want to import one or more //other// new repositories, and | ||||
would prefer they import at a higher priority. | would prefer they import at a higher priority. | ||||
- You may find overall repository performance is impacted by the large | - You may find overall repository performance is impacted by the large | ||||
repository import. | repository import. | ||||
You can manually change the priority of tasks with `bin/worker priority`. For | You can manually change the priority of tasks with `bin/worker priority`. For | ||||
example, if your copy of the Linux repository is `R123` and you'd like it to | example, if your copy of the Linux repository is `R123` and you'd like it to | ||||
import at a lower priority than all other tasks (including other imports of | import at a lower priority than all other tasks (including other imports of | ||||
new repositories), you can run a command like this: | new repositories), you can run a command like this: | ||||
``` | ``` | ||||
phabricator/ $ ./bin/worker priority --priority 5000 --container R123 | phorge/ $ ./bin/worker priority --priority 5000 --container R123 | ||||
``` | ``` | ||||
This means: set all tasks associated with container `R123` (in this example, | This means: set all tasks associated with container `R123` (in this example, | ||||
the Linux repository) to priority 5000 (which is lower than any natural | the Linux repository) to priority 5000 (which is lower than any natural | ||||
priority). | priority). | ||||
You can delay tasks until later with `bin/worker delay`, which allows you to | You can delay tasks until later with `bin/worker delay`, which allows you to | ||||
schedule tasks to execute at night or over the weekend. For example, to | schedule tasks to execute at night or over the weekend. For example, to | ||||
pause an import for 6 hours, run a command like this: | pause an import for 6 hours, run a command like this: | ||||
``` | ``` | ||||
phabricator/ $ ./bin/worker delay --until "6 hours" --container R123 | phorge/ $ ./bin/worker delay --until "6 hours" --container R123 | ||||
``` | ``` | ||||
The selected tasks will not execute until 6 hours from the time this command | The selected tasks will not execute until 6 hours from the time this command | ||||
is issued. You can also provide an explicit date, or "now" to let tasks begin | is issued. You can also provide an explicit date, or "now" to let tasks begin | ||||
execution immediately. | execution immediately. |
Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under Apache 2.0 or other open source licenses. · CC BY-SA 4.0 · Apache 2.0