Introduction
Empress Framework provides a robust system for executing background jobs, allowing developers to offload resource-intensive processes from the main application thread. This guide will explore Empress’ background job system from a technical perspective, providing a deep-dive into the code that drives this powerful feature.
Background Job System Overview
Empress utilizes the schedule package and a simple long-running infinite while loop to execute jobs in the background. This functionality is implemented through the frappe.enqueue and frappe.enqueue_doc methods. These methods allow you to enqueue Python methods or Document methods to be executed in the background, respectively.
Here is how you can use the frappe.enqueue method to enqueue a Python method:
def long_running_job(param1, param2):
# expensive tasks
pass
# directly pass the function
frappe.enqueue(long_running_job, queue='short', param1='A', param2='B')
# or pass the full module path as string
frappe.enqueue('app.module.folder.long_running_job', queue='short', param1='A', param2='B')
Enqueue Method Arguments
When enqueuing a job, there are a number of arguments you can pass to the enqueue method to control its execution:
frappe.enqueue(
method, # python function or a module path as string
queue="default", # one of short, default, long
timeout=None, # pass timeout manually
is_async=True, # if this is True, method is run in worker
now=False, # if this is True, method is run directly (not in a worker)
job_name=None, # specify a job name
enqueue_after_commit=False, # enqueue the job after the database commit is done at the end of the request
at_front=False, # put the job at the front of the queue
**kwargs, # kwargs are passed to the method as arguments
)
Enqueue Document Methods
In addition to Python methods, Empress also supports the enqueuing of Document methods. This is particularly useful for operations that are tied to specific Document instances. Here’s how to enqueue a Document method:
frappe.enqueue_doc(
doctype,
name,
"do_something", # name of the controller method
queue="long",
timeout=4000,
param="value"
)
Queue Configuration
Empress provides three default queues: short, default, and long, each with its own default timeout. However, developers can also create custom queues by configuring them in the common_site_config.json file:
{
...
"workers": {
"myqueue": {
"timeout": 5000, # queue timeout
"background_workers": 4, # number of workers for this queue
}
}
}
Worker Configuration
Empress sets up three worker types for consuming from each queue by default. These worker processes can be replicated to handle higher workloads. However, this is not a necessary convention, and you have the flexibility to modify your worker configuration according to your needs.
bench worker --queue short
bench worker --queue default
bench worker --queue long
Scheduler Events
Scheduler Events allow tasks to be run periodically in the background. These can be set up using the scheduler_events hook in the app/hooks.py file.
scheduler_events = {
"hourly": [
# will run hourly
"app.scheduled_tasks.update_database_usage"
],
}
After changing any scheduled events in hooks.py, you need to run bench migrate for changes to take effect.
Conclusion
Empress’ background job system is a powerful feature that allows developers to manage resource-intensive tasks efficiently, boosting the performance and responsiveness of the main application. With a deep understanding of this feature, developers can create more efficient, scalable, and robust applications using Empress Framework.