Can SearchWP work on a huge site?

SearchWP was designed from the get-go to work best on small-to-midsize sites. It will work best on a site with anywhere from a few posts to a few thousand. If your website has tens of thousands of posts (e.g. Posts, Pages, Custom Post Types) or even hundreds of thousands I would not recommend using SearchWP.

SearchWP was built on a test database consisting of about 20,000 posts in total (that includes Posts, Pages, and a few Custom Post Types). Performance was acceptable at that scale, but I would consider it the upper end of SearchWP’s sweet spot. These were fairly standard content types with some Custom Fields for each, and searches ran very acceptably. If those 20,000 entries were something more complex (e.g. an ecommerce product with a ton of metadata) and the search engine was configured to support a number of taxonomies and post meta, the search queries would have likely taken much longer.

The other thing to consider with a lot of content is the total time it will take to build the initial index. A site with tens of thousands of entries will likely take multiple days to build the first index. This is not ideal. That time increases as the overall volume of content increases. There are of course other factors to consider such as the specifications of the server, but the total number of posts can be a good gauge out of the box.

Alternative solutions to SearchWP for large websites

As WordPress continues to become the platform for extremely large sites, more and more will need an effective search solution. Without getting too technical: the database technology WordPress was built on was not specifically designed for rapid indexing and search. SearchWP adapts to that and works really well on small-to-midsize websites, but for large websites you will want something more fine-tuned.

Algolia

Algolia has taken the search world by storm! It’s an amazing product that with some development effort can be a truly fantastic search solution for sites of any size.

Swiftype

I would recommend checking out a hosted ElasticSearch service that basically replicates the turnkey approach offered by SearchWP, but will allow you to scale up to something much more powerful. Swiftype is a great company that touts excellent integration with WordPress.

ElasticPress

If you want to get your hands a bit more dirty and set up your own ElasticSearch server (or have access to one that has been set up for you) I would definitely recommend checking out ElasticPress from the folks at 10up. ElasticPress integrates your WordPress site with ElasticSearch and can bring all of the benefits of that along with it.