SearchWP News

Category: Announcements

In continuing the trend of making information about SearchWP 4 available, there are a couple of major things to outline in this post:

  1. Custom content support
  2. Proper Multisite global search

These two changes have been a very popular feature request leading up to the release of SearchWP 4, it’s very exciting to announce their availability!

Custom content support

SearchWP 3.x integrated only with content powered by WordPress’ WP_Post objects. This accounted for Posts, Pages, Media, and all registered Custom Post Types. This limitation was baked into the DNA of SearchWP itself, and the only way to work around it was to implement some sort of code solution to work around it.

While satisfying the majority of use cases, there are many circumstances in which site data is stored in a custom database table and made available through other means when editing your WordPress site. Unfortunately this content was inaccessible to SearchWP for a variety of reasons.

To circumvent the problem, some customers would mirror content to a ‘hidden’ Custom Post Type which SearchWP could work with. Other customers would dynamically pull in the content by using any number of SearchWP hooks. While these implementations got the job done in most cases, it was definitely a workaround.

A major goal of SearchWP 4 was to remove this limitation, and that’s been done! SearchWP 4 can be set up to index anything with a local database table. This includes Posts, Pages, Media, and all registered Custom Post Types. It also includes Users!

Not only that, but anything with a custom database table can be indexed and made searchable by SearchWP 4 as well.

This change has been made possible by the rewriting of the indexing process and search algorithm, and it will allow for integration with countless content setups within your WordPress installation.

How does it work?

SearchWP 4 has a concept of Sources, and each Source represents a content type that can be indexed/searched. In building out a Source a database table relationship is established, along with all attributes for that Source e.g. Title, Content, Slug, and anything else facilitated by the Source.

More information will be made available in the documentation for SearchWP 4, but the overarching rule is that if your content is stored in a database table and it has a column that maintains a unique ID for each entry, SearchWP can work with it and all of its attributes!

Proper Multisite global search

Another often requested feature that was not supported prior to SearchWP 4 is a global Multisite search that can search and return entries from any number of sites within a WordPress network installation.

SearchWP 4 can search across Multisite installations! When performing searches you will be able to specify which site(s) for which you would like results. SearchWP will handle retrieving those sites and returning them in such a way that minimizes the workload on your part. You’ll be able to mix and match searches across single or multiple sites in any way you’d like.

This is a big change that many customers will welcome, and it’s thrilling to have the opportunity to make it available. A change like this was fundamentally related to the decision to make SearchWP 4 a complete rewrite, and one of the reasons that such an undertaking was worth the effort that has been put into it.

Powerful new features

These two new features coming in SearchWP 4 will make it an extremely powerful solution for on-site WordPress search. Additional information and documentation outlining the specifics of these changes will be made available leading up to the release of SearchWP 4. Stay tuned!

Version 4.0 of SearchWP is getting closer and closer! ๐ŸŽ‰ In preparation of the release there are some exciting updates to share. In this post we’ll cover the initial performance benchmarks that have been run to date.

SearchWP’s primary operations are indexing and searching so those are the two benchmarks discussed in this post.

Index build times (in seconds, lower is better)

The index, indexer, and indexing process have all been completely rebuilt in SearchWP 4.0. Based on the new data models in use the overall indexing speed has been greatly improved.

Note: this data is based on an initial index build. SearchWP’s indexer applies very small delta updates to content changes over time.

Take for example this benchmark with a site that has about 1,000 Posts and Pages that have content ranging from a few hundred words to a few thousand:

For the same site with the same Engine configuration, the indexer in SearchWP 4.0 is running on average in about 36% of the time it takes to run in SearchWP 3.x (about 2.7 times faster)!

The indexer running via WP CLI is even more impressive, taking only 20% of the time it takes for SearchWP 3.x to build its version of the index. That’s about 5x faster! ๐ŸŽ‰

Another benchmark to consider is that of a larger site with 10,000 entries (again time is in seconds):

Improving the speed of the indexer was a large focus when working on SearchWP 4.0, with the hope being that a faster index build process the faster your website is displaying the best possible search results.

Search query times (in seconds, lower is better)

Indexing is a significant part of what SearchWP does, but the most important part is returning the best search results quickly. SearchWP 3.x did a good job that, but there was however room for improvement both in query execution time and query complexity itself.

Let’s examine some benchmark results when performing various searches on our site with 10,000 entries:

Note: SearchWP’s AND logic handling plays a part in decreased query time for increased term count.

Examining this results set we can observe a few things:

  1. SearchWP 4.0 performs search queries faster than 3.x
  2. Search query times are in many cases at least 2x faster!
  3. Observable speed benefit increases as the number of search terms increases

Please note that these benchmarks are completely dependent on source content and search terms. While the source content and search terms were identical for both test series, actual differences on customer sites will likely vary.

There’s even more in 4.0!

While searching and indexing are arguably the most important features of SearchWP (and both have been improved significantly in version 4.0) there is even more to cover as we approach the release date!

Stay tuned for a few more overview posts outlining the latest and greatest coming to SearchWP 4.0 ๐Ÿ‘

SearchWP was first released in August of 2013. It’s come a long way since then! Not only has SearchWP itself as a product matured and grown, I’ve learned a ton since writing that first version.

Without waxing poetic about things of the past, I’m very excited to be able to share some preliminary information about SearchWP 4.0 and why this release changes a number of things that will require your attention.

This is the first post of many that will outline what’s new/changing in SearchWP (it’s a lot!)

When first building SearchWP, the scope was limited in such a way that would have it work only with WP_Post objects, because it was during a time where the WordPress community at large was (mostly) against custom database tables and heavily in favor of “the WordPress way” being adoption and usage of existing APIs. It also made building the first version much easier.

The WordPress world (myself included) has changed direction on that, and we’re all going to be better for it.

That said: SearchWP 4.0 will NO LONGER be limited to WP_Posts! ๐ŸŽ‰ Note Users in this preview screenshot ๐Ÿ˜

Screenshot of SearchWP 4.0

With that, though, comes the biggest announcement:

SearchWP 4.0 is a (full and complete) rewrite!

To date, SearchWP has stood alongside the idea that backwards compatibility should never be broken. It’s one of the things WordPress has done really well, as have a number of fantastic WordPress products that the community holds near and dear to its heart. That is changing in SearchWP 4.0.

Please note: SearchWP 3.x will continue to be supported well after SearchWP 4.0 becomes available, allowing ample time to upgrade when customers see fit.

In order for SearchWP to support more content types than WP_Post (and without having to write/maintain Extensions that would mirror additional content types as WP_Post entries) it meant that (quite literally) every existing model would need to change. The database schema would need to be updated, the assumptions made by the code would need to change. Terminology would need to change. The indexer would need to change.

Rewriting a code base is often frowned upon for a number of legitimate reasons, but if SearchWP is going to mature and get even better, a line had to be drawn in the sand.

Committing to this rewrite means that as of version 4.0 SearchWP is adopting SemVer. SemVer is an intentional versioning approach that better communicates what’s happening with a project. To date it hasn’t been super popular in the WordPress world, but that tide is changing as well.

If you aren’t familiar with SemVer, it means that major releases of SearchWP will include breaking changes. SearchWP 4.0 is the first of these releases. Because so much had to change in order to support what SearchWP 4.0 has to offer, the code has been completely rewritten, without regard to backwards compatibility. On purpose.

There will be a partial upgrade process (e.g. existing engine configuration) but SearchWP’s index will need to be rebuilt when upgrading from SearchWP 3.x. It will also be possible to upgrade to SearchWP 4.0 without losing your existing settings/index in case you need to switch back for any reason. There will also be the option to clean up (remove) all traces of SearchWP 3.x once you are ready to finalize your upgrade.

In subsequent posts more detail will be given regarding the upgrade process and details about SearchWP 3.x support/updates.

Technical debt be gone

With SearchWP being nearly seven years old, it had its fair share of technical debt. Moving to SemVer allowed all of that debt to be removed which has resulted in a much leaner and cleaner (and faster!) code base that can be iterated upon in a much more stable way than was possible with SearchWP 3.x.

Further, the WordPress ecosystem has embraced modern PHP which means that SearchWP 4.0 is going to require PHP7 at a minimum. This is good for everyone all around. ๐Ÿ‘

Technical debt wasn’t limited only to the database schema and models used, it overflowed into front end code and just about everything SearchWP did. Being able to remove all of that and embrace how much both the back and front end has modernized in the past seven years is a big win.

It also allows for new (and better!) features to be included. Take for example the updated Rules implementation in the screenshot above. SearchWP 3.x had some exclude/limiter Rules, but they were limited in and of themselves and a bit awkward to work with. SearchWP 4.0 rethinks both the implementation and UI resulting in something much more powerful:

SearchWP 4.0 Rules Preview

Much about the main interface in SearchWP 4.0 looks similar, but time was given to evaluating the existing workflow and optimizations have been made to ensure that setting up SearchWP is as easy and straightforward as possible.

Managing which attributes are considered for each engine source (e.g. Posts, Pages, Users, etc) has also been updated and streamlined:

SearchWP 4.0 Engine Source Attributes manager

There is so much more to talk about (including enhancements to the indexer and search algorithm itself ๐Ÿค“) but hopefully this brief overview begins to outline how great SearchWP 4.0 is going to be. Stay tuned for additional updates outlining what to expect and when!

SearchWP’s Metrics Extension has been updated to version 1.0.9 and is a recommended update for all Metrics users. Here is the full changelog:

  • [New] Added additional parameters hits_min, hits_max when retrieving popular search queries over time
  • [Fix] Relocates hooks to make them more accessible to plugins
  • [Fix] False positive when preventing duplicate click tracking

A new Extension has been released: Custom Results Order! This extension was designed and built to help you manually curate results generated by SearchWP.

SearchWP Custom Results Order in action!

Why is it necessary?

If SearchWP was originally built to return the most relevant results, why is this Extension necessary at all?

SearchWP does return the most relevant results it can find, but those results are directly tied to SearchWP’s algorithm used to find those results in the first place. As with all algorithms, there are edge cases that go against the norm, and finding search results is no different.

You are the most knowledgeable of your site content, and you know what your visitors are looking for. Using SearchWP’s Metrics you can gain insight not only into what your visitors are searching for, but also which results they’re clicking.

There will be times that visitors are clicking results that don’t rank highest in the results set. You could manually edit the engine configuration to compensate, or stuff some keywords in your content, or even create a ‘hidden’ Custom Field to stuff keywords to get the overall rank higher.

Those solutions can work, but Custom Results Order removes the burden of having to set up and manage some sort of undocumented system. It provides you with a single screen containing all of these customizations you have in place.

Make your search even better

I hope Custom Results Order helps you improve your on-site search to an even higher degree. It’s already in use on this site, making sure that customers can find the documentation they’re looking for!

Version 1.0.3 of Metrics has been released. It is primarily a performance release and is recommended to all Metrics users. Full changelog:

  • [Improvement] Improves performance when generating Insights
  • [New] Settings button now has its own capability
  • [New] New filter searchwp_metrics_capability_settings to modify the Settings button capability

Enjoy!

Version 1.1.4 of Term Archive Priority has been made available to all active license holders. Full changelog:

  • [New] New filter searchwp_term_archive_term_args
  • [Improvement] Better support for Live Ajax Search searches
  • [Update] Updated updater

SearchWP’s Related extension has been updated to version 1.1. There is now a Widget available should you choose to take advantage of that, but please know that the Widget is for placement only, the template loader is still in full effect and responsible for display of the related entries.

Full changelog:

  • [New] Adds Widget
  • [New] New hooks to control meta box placement searchwp_related_meta_box_context and searchwp_related_meta_box_priority

Version 2.1.14 of Term Highlight has been made available to all active license holders. Full changelog:

  • [Fix] Fixes an issue where highlighting was applied when no search term was provided
  • [Fix] Restricts global excerpt generation according to applicable meta keys only
  • [Improvement] Better highlighting when terms are flanked with punctuation
Fix keyword search on your site. No coding required!

Now you can utilize all of the content that's gone unrecognized by native WordPress keyword search instantly with SearchWP

Get SearchWP