“SearchWP Failed to Index X Posts”

From time to time SearchWP is unable to index specific posts. When this happens the following admin notice will be shown:

Using that link you can check to see which specific post(s) cannot be indexed by SearchWP. Clicking the Reintroduce button will reintroduce that post to the indexing process, but please note that it does not change anything about the indexer (i.e. it will not resolve the underlying issue automatically).

The purpose of this link is to reintroduce a problematic post once a change has been made to get around the issue.

What causes index failure?

The cause of these failures is usually specific to the post in some way, but here are a few things to check out:

Is it an Attachment?

SearchWP’s document processing is a powerful feature, but it is limited to the capabilities of PHP, MySQL, and sometimes your server. SearchWP may be having trouble working with the file itself (especially if it is a large (2MB+) file).

If you are seeing a lot of your documents being flagged as index failures, check your error log for mentions of ‘timeout’ or ‘allowed memory size’ which can indicate that the server isn’t able to work with the document in it’s current state.

Attachments in the list of index failures includes a note to manually populate PDF content. SearchWP allows for manual content population if automatic parsing consistently fails.

Huge Custom Fields

Having Custom Fields that are huge in length can sometimes cause an issue, especially if they’re encoded in some way. Check out the postmeta for the failed post(s) for records that are sizable but not valuable to search. You can exclude these Custom Fields with the searchwp_excluded_custom_fields hook.

Large number of Comments

If a post being flagged for review has a large number of comments it might be causing the indexer to fail. If comments are not necessary you can prevent the indexer from parsing them with the searchwp_index_comments hook.

Other issue

If none of the above applies, the indexer may simply be running too aggressively on your server. Navigate to the Advanced tab of the SearchWP settings screen and tick the box to reduce the aggressiveness of the indexer.