Product documentation
Advanced Search - Search Data Update Triggers

The search data in the index can be updated via the 'Update search data' batch job. As this job updates all documents in the index, the job can take longer and longer to run as the amount of documents in the index increases, of which many are likely being updated even though there are no actual changes to it.

To avoid unneccessary updating and increase the speed with which actual changes are reflected in the index, it is possible to use Data change tracking. By using Database tracking together with a queue and batch job, actual changes in the data are tracked and only the affected records are then updated in the index.

This topic will describe how to set up Data change tracking for Product Search.

 

Data change tracking

Set up tables to track

Navigate to System administration > Setup > Advanced search > Search parameters

A 'Product' module record should be available and used for Product Search. If it does not exist yet, please create a new record (and make sure to select the product module if multiple are availble). Add all tables that should be tracked on the tab 'Data change tracking' via the 'Add' button. With the 'Remove' button a table can be removed from the setup.

Once the setup of tables is complete, please make sure to activate the 'Tracking enabled' option below the list of tables. Please note that the setup for the tables can only be edited while the tracking is disabled, so to make any changes afterwards, the tracing will need to be disabled first, then the changes can be made, after which the tracking should be enabled again.

To help with setup, the 'Load from field search' button has been added. Using this button will load all tables that are used within the Field search setup in the Search configuration(s) that are in the system.  

Note that the setup for the tables is global for global tables (such as EcoResProduct with the base data of a product), but company specific for company specific information (such as the Item group, where the values can differ between companies for the same product). When adding company specific tables, the table will have to be added with the company relation for every company that you want to track the changes in.     

Please carefully determine for which (product related) tables changes in the database should be tracked. Any changes made to a product that are stored in a (related) table that is not being tracked, will not trigger an update to the index and the change will not be reflected in Search. This can be used to your advantage (by not doing updates for data that is perhaps a little less relevant), but if product changes are not showing up in Search when you do want/expect them, please first double check if the correct product (related) tables are activated in the setup for all relevant companies.  

 

Set up data queue batch job

Navigate to Product information management > Inquiries &reports > Advanced search > Data update queue

The Data update queue form that opens will in most cases be empty. If there are records displayed here, most likely due to pressing the 'fetch tracked changes' button in the ribbon (see below), then these tracked changes are still in the queue and have not been processed yet (i.e. updated in Azure). If the records do not dissappear after some time, please double check if a recurring batch job (see Run data update queue below) has been set up to process the queue (after which the records get deleted) via the Batch jobs overview.

Run data update queue

Click on the 'Run data update queue' button in the ribbon. This will open up the pane where the batch job can be set up. For the update process to work seamlessly it is very important that the 'Fetch tracked changes' option on the batch job is enabled. If this setting is disabled, the tracked changes will not be fetched, not be processed and thus not be updated in the index. Also make sure to set the recurrence for the batch job, where we advise to set it with a relatively short recurrance pattern (such as every couple of minutes). Once the job processes a tracked change, it deletes those records.

Please note that if Data change tracking is activated and no recurring batch job has been set up to process the queue, a warning mentioning this will appear regularly.

Fetch tracked changes

This button in the ribbon can be used to manually fetch the changes that have been tracked and that have not been processed yet. This can be used while setting up the process for example (before the batch job runs) to double check if changes are being tracked as expected, after which you could run a (single) data update queue (with the fetching either activated or disabled) to process and delete the records. Note that this manual process should only be used for troubleshooting and during setup etc. and that for the updates to work smoothly in general the batch job should be used with a short recurrence and the 'Fetch tracked changes' option enabled.