In the first minor release of SearchWP since version 4 (which was a complete rewrite!) SearchWP 4.1 is now available to all active license holders. 🎉
This (minor) release is full of exciting changes, improvements, and bug fixes that all customers will significantly benefit from. The biggest changes to discuss are:
- Handling of Comments
- Refactoring of the Indexer
- Search algorithm improvements
- More bits and pieces!
These items are a big deal and customers should know everything about them, so let’s get into the details!
Updates to Comment handling
Note: Comments will need to be re-added to any applicable Engines
Your Index needs to be rebuilt after doing so!
Comment handling in SearchWP has always been mostly straightforward. Entire comment threads were treated as an attribute of the corresponding post type. While this helped to keep things simple, it did not scale very well.
For example: because Comments were treated as a single attribute of an entry, any time a Comment was added/removed/edited the entire entry would be reindexed. This kept things simple, straightforward, and stable, and it worked! The underside of that approach however meant that entire entries (and sometimes significant Comment threads) were reindexed quite often. Further, if an entry was popular and receiving many Comments in a short amount of time, it meant that SearchWP’s indexer was doing quite a bit of work to keep everything up to date.
This has changed in SearchWP 4.1. Comments are now treated as a standalone entity, and as a result of that: if (and only if) you have added Comments to any applicable post type in any SearchWP Engine you will need to edit your Engine(s) to re-include Comments after updating to SearchWP 4.1. It is recommended that you also rebuild your index after saving your Engines by clicking the Rebuild Index button on the Engines tab of the SearchWP settings screen.
Note: Comments will continue to transfer their calculated relevance weight to the appropriate parent post, as was the case in SearchWP 4.0. The benefit however arises when Comments change over time. With Comments as their own Source edit events will now apply only to individual Comments instead of the entire Comment thread let alone that and the associated
WP_Post. This is a huge performance win for sites with a lot of Comment traffic!
There are other benefits to Comments being their own Source. For example, you can specify relevance per Attribute (e.g. First Name, Last Name, etc.) as you can with other Sources and you can also take into consideration Comment metadata as with Post Type Sources. All of these things are good things and it’s exciting to make this change available in SearchWP 4.1!
Significant Indexer improvements
SearchWP’s Indexer was completely rewritten in version 4.0 and version 4.1 brings with it a significant complexity reduction that will not only make the indexing process more stable, it will have performance benefits as well.
In addition to the improvements made to the underlying Indexer, the handling of various circumstances in which the Alternate Indexer is a factor have been improved as well. That means better interface feedback and request handling, making interaction with the Alternate Indexer a more pleasant experience when it is necessary.
Another change to note is that SearchWP’s custom database tables will now include primary keys when previously there were none. This is for new installations only as adding keys to existing tables could be a significant undertaking.
SearchWP 4.0 omitted primary keys on multiple tables because they were not necessary for SearchWP’s operation so omission was in the name of performance. This did not take into consideration some lower level server configurations and backup/migration routines that fully relied on primary keys to operate. With the objective of being adaptive, SearchWP 4.1 adds primary keys to tables that previously did not have them for new installations only. If you have an existing installation and would like to add the primary keys for any reason, please open a support ticket using the Support tab of the SearchWP settings screen for assistance.
Search results are even better
SearchWP 4.1 brings with it a number of updates and changes to the search algorithm itself that will improve results across the board. Optimizations have been made when it comes to weight transfer such that the accuracy of the transfer itself is now more accurate.
AND logic received some attention and it too will bring more optimized results in some cases that previously were not strict enough.
An update has been made to SearchWP’s ability to work with regular expression pattern matches that improve the way SearchWP works with search strings. Common variations of search strings are now handled in a way that will bring in relevant results without reducing the accuracy of the results themselves.
Improvements have been made in a few cases where HTML-formed content was not always handled as expected, and highlighting has been improved in some cases as well.
Additional bits and pieces!
There’s a lot to cover for SearchWP 4.1 and while a full changelog is included below, here are a few more high level items that are worth pointing out:
- New WP-CLI commands to reindex and show diagnostics data
- Add custom ignored search strings with wildcard support to help reduce spam
- Automatic maintenance of Statistics logs
- All Advanced Custom Fields called out, not just “Repeatables” as in 4.0
- Synonyms Actions (sorting, clearing)
- Document processing/handling performance improvements
- Interface improvements to make configuration even faster
There’s even more than that! Here’s the full changelog for SearchWP 4.1:
- [IMPORTANT] Comments are now a separate Source (if you are using Comments for any Post Type Source you will need to edit your Engine and rebuild your index, this is not done automatically and should be planned for when updating)
- [New] Refactored indexer for stability, reliability, and performance optimization
searchwp\index\update_entryaction when an Entry is updated in the Index
searchwp\debug\front_end_summaryfilter to control whether the HTML comment summary is output when debugging is enabled
wp searchwp reindexWP-CLI command
wp searchwp diagnosticsWP-CLI command
- [New] Primary keys added to database tables for new installs where previously there were none
- [New] Adds
\SWP_Queryto reference underlying query object
searchwp\source\comment\parent_attribution\stricthook to control whether SearchWP is strict about parent attribution which can introduce overhead in some cases
- [New] SearchWP will now generate more expected variations of regex pattern matches
searchwp\tokens\generate_parttern_match_variationsfilter to control whether additional pattern match variations are generated during indexing
- [New] Ability to enter custom ignored queries for Statistics (with wildcard
- [New] Ability to automatically trim Statistics logs
- [New] Notice when WordPress available memory can be improved
- [New] Export/import now optionally includes Settings, Stopwords, and Synonyms
- [New] All ACF fields are displayed in the Custom Fields dropdown when applicable, not only ACF "Repeatables"
- [New] Synonyms actions: Sort ASC, Sort DESC, Clear
- [Change] Refines default data indexed for Taxonomy Terms to be: taxonomy name, term name, term slug, and term description
- [Change] Abstracts Statistics Dashboard Widget from
- [Improvement] Indexer auto-scaling when server load is high
- [Improvement] Handling of HTML-formed content during indexing
- [Improvement] Integration of partial matches and keyword stemming
- [Improvement] Indexer delta trigger specificity in some cases
- [Improvement] Document processing handling, footprint
- [Improvement] Highlighting when a suggested search has been returned
- [Fix] Prevent duplicate results during weight transfer in some cases
ANDlogic restriction, performance in some cases
- [Fix] Persistent dismissal of missing integration notice
- [Fix] Synonyms management in some cases
- [Update] Dependencies
- [Update] Translation source
- [Update] Interface updates and refinements
Hopefully you’re as excited about SearchWP 4.1 as we are! As always, if you need any help updating please submit a ticket using the Support tab of the SearchWP settings screen or emailing firstname.lastname@example.org (including your license key) and you’ll be up and running in no time!