Introduction
Welcome to our comprehensive guide on Empress Migration and Patch System Tools. In the lifecycle of a software development project, database schema changes and patching existing data are common scenarios. Empress provides a robust and efficient set of tools to handle these scenarios, and in this guide, we will explore these features in detail, focusing on their integration, utilization, and modification.
Migrating to New Schema
When there are schema changes in your app, you can migrate your existing site’s database to the new schema using the following command:
bench --site [sitename] migrate
Schema Changes and DocType
In Empress, you can edit a DocType to add, remove, or change fields. On saving a DocType, a JSON file containing the DocType data is added to the source tree of your app. When you add an app to a site, the DocTypes are installed (database tables are created) using this JSON file.
To make schema changes, you must enable Developer Mode.
On running a migrate, all apps migrate to their current version. First, we run the “before_migrate” hook to sync user permissions. Then the actual patches are run, which migrate the various apps. After this, we sync the following components:
- Database Schema
- Background Jobs
- Fixtures
- Dashboards, Desktop Icons and Web Pages
- Updates Translations
- Rebuild Search Index for all routes
For DocTypes, we compare the MD5 hash of each DocType JSON with the hashes we have stored in the DocType database table. If the hashes don’t match, we reload the particular DocType. This technique is called checksum comparison.
After all these steps, we run an “after_migrate” hook to finish the migration process.
Note: Up Until
v13, DocTypes were synced based on the modified timestamps in the JSON file. This method was error-prone so we moved to the more robust hash comparison method.
When you remove or rename fields in the DocType, the corresponding database columns are not removed from the database table, but they will not be visible in the form view. This is done to avoid any potential data loss situations and to allow you to write related data migrations (patches) which might need values from old fields.
Note: Empress doesn’t support reverse schema migrations.
Data Migrations
On introducing data related changes, you might want to run one-off scripts to change existing data to match expectations as per new code. We call these scripts patches in Empress.
Writing a Patch
To write a patch, you must write an execute method in a python script and add it to patches.txt of your app. The directory structure followed in Empress is as below:
frappe
└── patches
└── v12_0
└── my_awesome_patch.py
The patch can then be added to patches.txt by its dotted path.
frappe.patches.v12_0.my_awesome_patch
Schema during Patch
The DocType meta available in the execute function will be as per the old JSON. This is so that you can write migration code assuming you still have the old fields. After the patch is run, the new schema is applied to the DocType.
If you want to have the new schema during your patch execution, use the reload_doc method.
import frappe
def execute():
frappe.reload_doc(module_name, "doctype", doctype_name)
# your patch code here
One-Off Python Statements
You can also add one-off python statements in patches.txt using the syntax,
frappe.patches.v12_0.my_awesome_patch
execute:frappe.delete_doc('Page', 'applications', ignore_missing=True)
Patch Execution Order
Patches run in the order they are defined. All lines in patches.txt have to be unique. If a patch has been run before, it won’t run again. If you want to run a patch again, add a comment that will make the line appear as new.
For Example,
frappe.patches.v12_0.my_awesome_patch #2019-09-08
Conclusion
In software development and customization, Empress Migration and Patch System Tools play a critical role. They allow for efficient and controlled handling of changes in database schema and data, thus ensuring the integrity of the application and the data it manages. By understanding these tools, developers can effectively manage and control the evolution of their applications, contribute to the stability of the system, and provide a seamless user experience.