A search application I was working on had an annoying performance problem.
Most searches completed quickly, but some requests occasionally took more than 10 seconds. The cluster wasn't overloaded. The index looked healthy. CPU utilization wasn't unusually high.
Yet users were still experiencing long waits.
After profiling the request, the culprit wasn't the search query itself. It was the way the search engine executed different parts of that request.
Step 1: Profile before changing anything
The first rule of performance tuning is simple:
Never optimize until you know exactly where the time is being spent.
In Solr, there are several ways to do this:
- debugQuery=true
- the request timing section in the response
- slow query logs
- request tracing through your search pipeline
Instead of looking only at the total request time, break the request into its components:
- query execution
- filter queries
- facet computation
- highlighting
- response transformation
In our case, document retrieval completed relatively quickly. Facet computation did not. Some searches were calculating facets over approximately 120,000 matching SKUs, and those requests occasionally exceeded 10 seconds.
Step 2: Expensive filters are still expensive
One common misconception is that filter queries are essentially free because they don't participate in scoring. That's only true for simple cached filters.
Function queries, joins, geospatial filters, collapse queries, and complex range functions can consume substantial CPU time. Although they don't affect ranking, they still have to be evaluated.
Fortunately, Solr allows us to indicate that some filters are more expensive than others.
Before: fq={!cache=false}expensive_function(...)
After: fq={!cache=false cost=200}expensive_function(...)
Assigning a higher cost tells Solr that this filter should be executed after cheaper filters have already reduced the candidate document set. Instead of evaluating an expensive function across millions of documents, Solr evaluates it against a much smaller subset.
Nothing about the ranking changed. Nothing about the hardware changed. Only the execution order changed.
Step 3: Protect the cluster from expensive facet requests
The larger bottleneck was faceting.
One request was producing roughly 120,000 matching SKUs, and computing facet counts across that result set became the dominant source of latency.
The immediate mitigation was adding timeAllowed:
Before: facet=true
After: facet=true timeAllowed=1000
This limits how long Solr will spend processing a request. Instead of allowing a handful of expensive searches to monopolize CPU resources, the server can return partial facet results once the time limit is reached.
For our application, that reduced worst-case response times from over 10 seconds to under 2 seconds, while maintaining an acceptable user experience.
Step 4: The architectural solution
Although timeAllowed dramatically improved latency, it isn't the ideal long-term solution. It protects the cluster. It doesn't eliminate the work.
If the same expensive facet requests occur repeatedly — as they often do in ecommerce category pages — it is usually better to stop calculating them on every request.
Instead, precompute the facet counts periodically. For example:
- "Living Room → Sofas"
- "Dining Tables"
- "Sale Items"
- "Outdoor Furniture"
A scheduled background job can calculate these facet counts every few minutes or after indexing completes and store them in Redis or another fast cache. At query time, the application simply retrieves the precomputed counts instead of asking the search engine to calculate them again.
Dynamic, long-tail searches can still use live faceting.
The result is a system that is both faster and more predictable under load.
Does this apply to Elasticsearch and OpenSearch?
The specific parameters differ, but the design principles are the same.
Solr exposes cost for filter execution order and timeAllowed for limiting request execution. Elasticsearch and OpenSearch optimize filter execution internally, so there is no direct equivalent to cost. However, they provide mechanisms such as request timeout, terminate_after, search task cancellation (search.cancel_after_time_interval in OpenSearch), and circuit breakers to prevent expensive searches or aggregations from consuming excessive cluster resources.
Likewise, the recommendation to precompute or cache frequently requested aggregations applies equally to all three search engines.
The engine may change. The performance principles do not.
Lessons learned
This investigation reinforced three lessons that apply to almost every enterprise search platform:
- Profile first. Guessing is rarely productive.
- Execution strategy matters as much as query complexity.
- If users ask the same expensive question thousands of times a day, don't compute the answer thousands of times.
Search optimization isn't always about new hardware or better ranking algorithms. Sometimes the biggest improvements come from understanding how your search engine spends its time — and making sure it spends that time wisely.