Introduction
Smart Import Export is a user-friendly module that allows users to export and import channel data by CSV, XML and JSON formats without writing any queries. Smart Import Export module is a powerful tool to export and import the data of default field types as well as third party field types.
Compatibility
EE Compatible Version: EE3, EE4, EE5, EE6.
Some of the features of Smart Import Export are listed below:
- Easy to Install and use.
- Exports and Imports channel entries without writing queries. (channel wise)
- It supports MSM sites.
- It handles large entries while exporting and importing.
- It exports and imports the entries by CSV, XML and JSON formats.
- It allows filtering entries by their Status for Export.
- It exports and imports all the primary data such as Channel title, URL title, Entry date, Author, Status, and Categories, etc.
- It allows the user to choose the custom fields while exporting and importing.
- It allows user to Map data values to a channel’s fields while importing.
- It supports Relationship fields to export/import and facilitates users to choose the base of relationship entries to export. i.e., Title, URL title, Entry ID.
- It supports Grid and Fluid fields to export and import.
- It supports Relationship fields inside Grid and Fluid fields for export and import.
- It supports Grid fields inside the Fluid field. Supports each relationship field that is inside the GRID. (That GRID is inside Fluid).
- It supports Images export and import by its URLs.
- It supports all default field types for export.
- It supports below third party field types:
- Assets
- Wygwam
- Smart Members Field
- Tags
Note: We are not providing support for EE3.
Installation
Follow below steps to install Smart Import Export version:
EE3, EE4, EE5 and EE6:
- Download and Extract ZIP file of the module. (We recommend to use latest version.)
- Copy “SmartImportExport > system > user > addons > smart_import_export” folder to “Your Site > system > user > addons” Folder.
- Copy “SmartImportExport > themes > user > smart_import_export” folder to “Your Site > themes > user” Folder.
- Login to backend panel with Super admin credentials.
- Go to “Developer > Add-Ons” and Install Smart Import Export module.
Note: If you purchased the addon from Devot:ee then you can get the license key from the licenses tab in your profile.
Export General Settings
General Settings:
- Encode or Decode Exported content?:You can set this parameter to encode or decode exported content. Encode / Decode will be perform for encode into UTF-8 or decode from UTF-8.
- Convert all timestamp dates:You can pass valid date format to convert timestamp to valid date formats. You can refer this URL for all PHP date formats. (ex: Y-m-d H:i:s). Leave blank to keep exporting in timestamp format.
- Encode HTML tags:If enable this field, it will convert HTML tags to HTML entities (ex: <p> => <p>)
- Download file anyway if there is no entries found to export?:If Enabled this field, It will download even a blank file. If Disabled, It will show standard EE error.
CSV Settings:
- Separator for single dimension array:If you have assets or channel images for example. All images will be separated by your separator. ex: (abc.png | def.png)
- Convert type for Multi dimension array:You can select convert type for Multi dimension array to fit in single column/line. We provide four options for this.
- JSON
- Serialize
- JSON + Base64 Encode
- Serialize + Base64 Encode
XML Settings:
- Root Tag name:You can provide the tag name for main root entity to wrap all data in it.
- Element Tag name:You can provide the tag name for individual set of given data.
Create new Export
To create a new export, go to backend Smart Import Export settings. Click on “Create new Export” link.
In the process of creating a new export, You need to select the channel you want to export.
After selecting the channel, Full settings stack will show.
Select which default and custom fields you want to export and fill general settings.
General Settings are described as below:
- Export Name:
Name of your current export settings. This field is just a label of your export. - Access export URL without Login ?:
In case if you want to export these settings outside of EE without any login, Mark this as “Yes.” - Export Type:
Mark this Export as “Private” will not allow any other guest or site user to download this Export. You and only you will be allowed to download this export. In case, if you mark this Export as “Public”, anyone can access and be able to download this export. - Export Procedure:
There are two ways to export channel data.- Normal:
Click and download export on a single click. In this method, no AJAX will fire. If you have plenty of entries to export and your server memory_limit and max_input_vars is low in comparison to total entries to export in a single shot, it will give you memory_limit error or 500 internal server error. This process is recommended if you have a low number of entries to export. - AJAX:
Loop base AJAX export. This method will calculate the total number of entries and divide ajax calls by batches you have entered in settings. Default batches are 50. So, suppose you have 10,000 entries to export, and you chose AJAX export with batches of 100 entries per ajax call. It will throw 100 calls of AJAX on your download button (one by one). After completing all the ajax calls, it will give you a URL to download the generated file. This method is highly recommended if you have plenty of entries to export in a single file. No matter if you have hundreds or thousands of entries, with this method, it’s too easy to generate an export file.
- Normal:
- At last, you have to select the format in which you want the exported data. We have 2 types of export formats. (CSV and XML).
Create new Import
To create a new import, go to backend Smart Import Export settings. Click on “Create new Import” link.
In the process of creating a new import, you need to select the file type (CSV/XML) for import. You need to select a channel to import the data into it.
If you select CSV in file type then CSV related fields will be shown.
- File Name or URL of File:
You have to provide the file name with the system path which you want to import. - Delimiter (Optional):
You can provide the delimiter eg. comma (,). This character is used to separate the fields of the file. - Encloser (Optional):
You can provide the enclosure which is used to enclose each field of the file. - Fields of first row as titles:
if this field is checked means the system considers the first row as title.
If you select XML in file type then XML related fields will be shown.
- File Name or URL of File:
You have to provide the file name with system path which you want to import. - XML Path:
XPath is used to navigate through elements and attributes in an XML file. Generally, XML file is created with “root” and “elements” tags so XML path is considered as “/root/elements”.
Then, save the form by clicking on the “Save Action” button, then full settings stack will be shown.
Default Fields Section:
This section shows the all default fields of the channel.
You have to map default fields by selecting the appropriate options from the select box. For eg, for the “Title” field, you have to select the title option of the select box.
Group Fields Section:
This section shows all the group fields of the channel which are considered as custom fields.
You have to map the group fields using the appropriate options from the select box.
For eg. “about_image” is a grid field which contains the “Image”, “Caption” and “Alignment” fields. You have to map this grid field according to below information
- “about_image ( grid )”” is mapped with “about_image” option.
- “about_image ( grid ) >> Image ( file )” is mapped with “about_image->image” option. For file type fields, user can select the option of directory by “Upload Dir.” field where they want to upload the files during the import. For this thing, user have to select the “Yes” option in “Want to fetch?” field.
- “about_image ( grid ) >> Caption ( text )” is mapped with “about_image->caption” option.
- “about_image ( grid ) >> Alignment? ( select )” is mapped with “about_image->align” option.
Individual Fields Section:
This section shows all the Individual fields of the channel which are considered as custom fields.
You have to map the Individual fields using the appropriate options from the select box.
For eg. “simple asset field” (Field type: assets) is the third party field which is mapped by the “simple_assets_field”.
Check for duplicate entries:
- Use this field to check for the duplicate entries:
This field is checked for duplicate value based on the selected option for the entries to import. - Duplicate Action:
This field is used to define the action when any entry already existed in the channel.- Update:
Duplicate entry is updated with the data of the export file during the import process. - Delete:
Duplicate entry is deleted first, then system insert with the new entry. - Insert New Row:
System considers the new entry event if the system finds the duplicate entry in system.
- Update:
- Delete existing entries :
If this field selects “Yes” then it will delete entries from the selected channel that are not processed (updated or created) by this import.
Categories Section:
This section shows import setting of all the categories of the selected channel.
- Category Default Value:
You can assign the default category value to every entry. - Category Group:
You can assign the category group to the entries by mapping this field with the related column of the import file. - Category Delimiter:
You need to provide the Delimiter value for the categories values of the entries which is defined in the import file. Generally, it is the Comma(“,”). - Parent Child Category Delimiter:
If your import data contains the parent child categories then you should provide the Delimiter value between parent and child category name. For example. “Flower / Artificial Flower, Flower / Original Flower”. In this example, “Flower” is the parent category of the “Artificial Flower” and “Original Flower” child categories. This “/” is the delimiter used to specify the parent child relationship between the categories. - Create category if it does not exist: If you enable this setting then non exist categories will be created.
- Force exact categories: If you enabled this setting then the provided categories name will be found in the exact level of category. For example: “Original Flower”. We know this the child category of the “Flower” but if we enabled “Force exact categories” then the “Original Flower” category will be found in the parent (First) of level category. So, we should pass the “Flower / Original Flower” when this setting is enabled.
Import Settings:
- Import Name:
Name of your current import settings. This field is just a label of your import. - Import Comment:
Description about the current import setting. - Batches:
Select how many entries you want to import at a time. (All entries will import in batches. Your import process will complete once all batches imported.)
Custom Sidebar
We provide one “customSidebar” function in the MCP file. You can add more menus in the sidebar of the addon using this function.
/* To get dynamic menu in the addon */ function customSidebar(){ $sidebar = ee('CP/Sidebar')->make(); $this->navSettings = $sidebar->addHeader('Test Link', ''); }
Run Export
You can now run export from front end templates. All you need to pass is single tag:
{exp:smart_import_export:run}
This parameter has view modifier functions. We suggest you use it in a blank template only.
Parameters
Following Parameters can be use Run tag:
id
This parameter will inform the tag to download the given export ID. This is an alternative parameter of token.
Example:
id="1"
token
This parameter will inform the tag to download given export Token. You can get a token of a given export by clicking on the popup URL on backend table settings. This is an alternative parameter of ID.
Example:
token="ae17e17a80c87776426efd6e164df3a8"
start_date
This parameter is created to override default “Start date” that is specified in export form. You can set a valid timestamp of export entry start date. This parameter will force you to download entries whose entry_date is greater then given timestamp.
Note: To use this parameter, You must need to set “Enable Date wise filter?” to “Yes” in export form.
Example:
start_date="1527061200"
end_date
This parameter is created to override default “End date” that is specified in export form. You can set a valid timestamp of the export entry end date. This parameter will force you to download entries whose entry_date is less then given timestamp.
Note: To use this parameter, you must set “Enable Date wise filter?” to “Yes” in export form
Example:
end_date="1527061200"
entry_id
You can specify which entries should export or which entries should not. This parameter will override all default entries export to limit only specific entries to export.
Example:
entry_id="20|55|48|800|700" entry_id="not 23|36"
type
You can specify if you want to download export on 1 click or AJAX. Passing this parameter will load its own view and you cannot use your view on it. (You can modify it if you want.)
Example:
type="ajax"
Example:
{exp:smart_import_export:run token="{segment_2}" type="ajax" entry_id="8|9|10" start_date="1527050000" end_date="1527061200"}
Import Cron
You can execute the specific import using the cron.
The cron URL of the specific import can be obtained from the “Import List” page.
You have to click on the “Link” icon button for the specific import.
One popup box shows the import URL and you can also copy it using the “Copy to clipboard” button.
Example of the cron URL: https://www.example.com/?ACT=72&token=TOKEN
Here, The value of the ACT is the action ID and it is different for your setup because it is dynamically created at the time of installing the add-on. TOKEN is the unique identifier of the import.
This cron method is suitable for single import via a single URL.
If you want to execute multiple imports using a single URL then you can achieve this thing using the import template tag.
For this thing, you have to create one template file. For example, you can create the template group “home” and template file “import” and then add below template tags inside it.
{exp:smart_import_export:run_import token=”TOKEN_1″}
{exp:smart_import_export:run_import token=”TOKEN_2″}
Here, the value of the token can be get from the “Import List” page.
To run these imports, you have to visit this “https://www.example.com/home/import” URL.
Changelog
- V0.0.1
- V2.0.0
- V3.0.1
- V3.0.2
- V3.0.7
- V3.0.8
- V3.0.9
- V3.1.0
- V3.1.1
- V3.1.2
- V3.1.3
- V3.1.4
- V3.1.5
- V3.1.6
- V3.1.7
- V3.1.8
- V3.1.9
- V3.1.10
- V3.1.11
- V3.1.12
- V3.1.13
- V3.2.0
- V3.2.1
- V3.2.2
- V3.2.3
V0.0.1
- Initial Version
V2.0.0
- Updated the license verification process and setup so please update the license in your addon.
V3.0.1
- Added the mapping of the status and author for import.
V3.0.2
- Solved issue of category data from the XML file for import.
V3.0.7
- Provide the ability to mapping the column for the CSV file when the CSV file does not include the title row.
- Provide the option to delete existing entries of the selected channel which are not included in the import
- Automatic generates url_titls from the title when doesn’t provide any value to url_title.
- If url_title is provided as a raw string, Smart Import Export converts it ExpressionEngine friendly url_title.
- Solved the issue when sometimes selected the option not rendering when user going to edit the import configuration.
- When a user doesn’t provide any appropriate status value then channel default status will be provided.
- Predefined status_id (integer value) and status(status name) both are acceptable in the status field.
V3.0.8
- Added the feature of the cron for the import. Get more detail from here
V3.0.9
- Solved issue of not working relationship field in group field and XML type of file.
V3.1.0
- Solved the CP warnings in the import page.
- An added feature of providing three options for importing the grid fields.
1. Delete all existing rows and add new
2. Keep existing rows and append new
3. Update the rows if the specific column matches
V3.1.1
- Solved the mapping and data fetching issue in the fluid field for the XML file.
V3.1.2
- Solved warning during to update an add-on.
- Increase the width of the dropdown in the import screen to display more information.
- Solved warning during the “Upload Dir.” not selected but assets field is selected.
V3.1.3
- Solved issue of not saving JSON format in the export setting page.
- File related setting field is added for the fluid field on the import mapping page.
- Solved issue of the category not imported for the JSON file.
V3.1.4
- Provided the option to create the category if it does not exist during the import.
- Provided option to define the parent-child category delimiter to specify the relationship in the import setting.
V3.1.5
- Solved the issue of non-encoded characters in the export by providing the setting of “Disable ob_start and ob_clean functions for export” in “Export General Settings.”
- Solved MSM import issue while import process performed via import URL.
V3.1.6
- Introduce new data-type as “Third party xml” to import XML feed from other sources and set it into the entry.
- Now user can set author to the imported entries from the system’s super admin role/group.
- Import batch limit increased to 200 (added 50, 75, 100, 150, 200)
V3.1.7
- Solved unselect dropdown option during edit import configuration
V3.1.8
- Allowed special character in channel entry Title field while importing channel entries
V3.1.9
- Support added for Smart SEO, SEEO, SEO Lite, Transcribe, Structure, Maps, Polls, Wygwam, low variables, Low events
V3.1.10
- Solved issue for the CSV Delimiter and Encloser issue
V3.1.11
- Added file_grid field type support
- Solved some bugs for third party add-ons
- solved some issues for the category import
V3.1.12
- Added new option in CSV import while there is no encloser in the data
- Solved a bug while user select a field other than title URL title field for the duplicate check in Third party XML datatype import
V3.1.13
- Bug fixing.
V3.2.0
- JSON import functionality changes and new setting parameter added in JSON import.
V3.2.1
- Asset import with XML issue solved when xml tag have more than one URL.
- Removed “se–” and add “item” tag during export to set the array of the column.
V3.2.2
- Fixed Fluid field and JSON import issues.
- Fixed Grid field issue.
V3.2.3
- Fixed date filter issue.