Limiting I/O usage

Before continuing further with this recipe, read the Choosing the proper directory configuration recipe of this chapter to see what directories are available and how to configure them.

Let's assume that we want to limit the I/O usage for our use case that uses solr.MMapDirectoryFactory. So, in the solrconfig.xml file, we will have the following configuration present:

<directoryFactory name="DirectoryFactory" class="solr.MMapDirectoryFactory">
</directoryFactory>

Now, let's introduce the following limits:

To do this, we change our previous configuration to the following:

<directoryFactory name="DirectoryFactory" class="solr.MMapDirectoryFactory">
 <double name="maxWriteMBPerSecFlush">20</double>
 <double name="maxWriteMBPerSecMerge">10</double>
 <double name="maxWriteMBPerSecRead">50</double>
</directoryFactory>

After altering the configuration, all we need to do is restart Solr and the limits will be taken into consideration.

The logic behind setting the limits is very simple. All directories that extend the Solr CachingDirectoryFactory class allow us to set the maxWriteMBPerSecFlush, maxWriteMBPerSecMerge and maxWriteMBPerSecRead properties. The mentioned directory implementations are all the directory implementations that were mentioned in the Choosing the proper directory configuration recipe of this chapter.

The maxWriteMBPerSecFlush property allows us to tell Solr how many megabytes per second can be written by Solr during segment flush (so, during the write operation that is not triggered by segment merging). The maxWriteMBPerSecMerge property allows us to specify how many megabytes per second can be written by Solr during segment merge. Finally, the maxWriteMBPerSecRead property specifies the amount of megabytes allowed to be read per second. One thing to remember is that the values are approximated, not exact.

Limiting I/O usage can be very handy, especially in deployments where I/O usage is at its maximum. During query peak hours, when we want to solve server queries as fast as we can, we need to minimize the indexing and merging impact. With proper configuration that is adjusted to our needs, we can just limit the I/O usage and still serve queries with the latency we want.