Utilizing Empress's `import-csv` Feature for Efficient Large CSV File Import

Introduction

Empress, a robust business solution software, provides numerous features and utilities tailored for developers to facilitate seamless data handling and integration. One such feature is the import-csv utility in the bench module. This feature is specifically designed to cater to the import of very large CSV files, a common task in many software development and data management scenarios.

The import-csv feature is not subject to timeouts, a characteristic advantage when compared to the web interface. This makes it particularly useful in scenarios where large files need to be imported without any interruptions.

The import-csv Feature

The import-csv utility can be invoked using the following syntax:

bench --site test.erpnext.com import-csv ~/Downloads/Activity_Type.csv

In this command, bench is the command-line utility, --site test.erpnext.com is the site where you want to import the CSV file, and import-csv ~/Downloads/Activity_Type.csv specifies the CSV file you want to import.

Options for import-csv

The import-csv feature also provides several options that developers can utilize to customize the import process. To understand these options, you can use the --help command:

$ bench import-csv --help

This will output a help message detailing the options:

Usage: bench  import-csv [OPTIONS] PATH

  Import CSV using data import tool

Options:
  --only-insert             Do not overwrite existing records
  --submit-after-import     Submit document after importing it
  --ignore-encoding-errors  Ignore encoding errors while converting to unicode
  --help                    Show this message and exit.

These options provide the following functionalities:

  • --only-insert: This option ensures that the import process does not overwrite any existing records.

  • --submit-after-import: This option submits the document after importing it.

  • --ignore-encoding-errors: This option instructs the utility to ignore any encoding errors that occur during the conversion to unicode.

Summary

The import-csv feature in the bench module of Empress is a powerful tool for developers. It’s capable of importing large CSV files without any timeouts, providing a robust and reliable solution for data integration tasks. The feature also offers a range of options that can be utilized to further customize the import process, thereby contributing to the development and customization of business solutions in Empress.