By tuning parameters such as lo_water and hi_water,
you can create thread pools that handle variable loads while striking a good balance
of CPU and memory usage.
A dynamic thread pool (i.e. a pool in which worker threads are created and destroyed
as needed) can help a server manage variable loads.
Problem is, the pool creates new threads—which contribute to
overhead—at the worst time possible: when the server is getting hit
by a heavy load. This approach is preferable to not creating the threads,
but comes at a cost nonetheless.
In some cases, you can avoid the issue by setting the pool to a fixed size.
To take this approach, you need to:
-
Determine, whether through testing or system analysis, the number of threads that
the thread pool requires.
-
Set maximum to the number of required threads.
-
Set both lo_water and hi_water to
the same number as maximum.
Of course, setting the pool to a fixed size isn't always appropriate. For instance,
you may need to tune your parameters to handle the following dynamic
scenarios:
-
A high initial load that drops down to a lower, steady load — In this case,
you likely need to set hi_water to be significantly lower than
maximum. This configuration allows many threads to be created
at the high load point, but will recover the memory overhead of the per-thread data
as the load decreases.
-
An ongoing series of burst loads — In this case, you would set
hi_water to maximum. This configuration assumes
that, once you've hit the load required to create a new thread, you're likely
to hit that load again. It thus avoids the overhead of thread creation
each time the load occurs.
Compared to an approach that creates and destroys threads, this configuration commits
more memory, but reduces CPU overhead and latency for future requests to the pool.
A further concern is the thrashing that can occur when load bursts cause threads
to be repeatedly created and then destroyed. To prevent this issue, increase the
difference between lo_water and hi_water.