Caching How Twitter uses memcached

Redis Part 2 - Application Cache Benchmark: Redis, NVMe SSDs and memcached

Which form of application cache leads to more performance in e-commerce: the use of Redis, memcached or NVMe SSDs? We carried out a benchmark for this and measured the throughputs of write, read and delete operations per second.

Why these Redis benchmarks?

In our daily work with online shops, we have repeatedly found that the use of Redis has led to an increase in performance. The throughputs of the write, read and delete operations were improved by Redis, which was proven by previous benchmarks by Redis developer Colin Mollenhour. But technology continues to develop and so does our hardware: We use NVMe SSDs as data storage on our clusters. They achieve significantly higher data throughputs than SATA SSDs or HDDs and therefore lead to better performance in e-commerce. The data storage is used by the shop systems Magento and Shopware by default as a cache backend. Redis and memcached, on the other hand, store the data structures in the main memory. With this benchmark, we wanted to find out which form of data storage in application caches ultimately leads to better performance for online shops: Redis (memory), memcached or NVMe SSDs (file system).

To do this, we measure the throughput of delete, write and read operations per second and compare them.

In the second benchmark, which we will present in the next blog post "Redis Part 3", we concentrated on the sessions. We wanted to find out which session memory is more efficient when reading, writing and changing data: Redis, MySQL or NVMe SSDs.

Test environment

The tests were carried out on one of our clusters with our current hardware (at the time of the test), such as Intel Xeon Gold 6254. In the test environment, we selected 4 CPU cores to represent the computing power of one of our average clusters. We set the RAM amount to the very high value of 32 GB so that all cache files could be stored in the Linux file system cache. In most cases this can be done with much less RAM.

componenthardware
CPUIntel Xeon Gold 6254 @ 3.10GHz (4 activated CPU cores)
SSDSamsung SSD PM1725b 3.2TB, PCIe 3.0 x8 NVMe SSD
R.A.M.32 GB DDR4 2933 MHz

Benchmark: Application caches under Magento 1

Since Magento 1 has a complex cache system, we decided to use the Benchmark example under Magento 1 perform. This can be used to demonstrate the effects of cache backends on complex data structures. The benchmark for application caches was under the current version (at the time of the test) Magento 1.9.4.1 carried out.

Shopware and Magento online shops store the data in the data storage device (with us NVMe SSDs) by default. Redis, on the other hand, stores the data in memory. In order for Redis to be supported by Magento 1, certain plugins must be installed. Magento 2 and Shopware already support Redis natively.

The benchmarks established by Colin Mollenhour in 2012 served as the starting point for our test. We simulated this test on our hardware and the current Magento 1 shop system and tested various backends.

Backends for the benchmark

Zend_Cache_Backend_File

The backend Zend_Cache_Backend_File is used with Magento 1 if no other special backend is used via the local.xml or a module. A special configuration is not necessary here.

For our tests, we performed the benchmark on the fast NVMe SSDs we used.

Cm_Cache_Backend_Redis

This is a module from Collin Mollenhour, which allows the Magento 1 cache to be stored in Redis. For Magento 2, an implementation based on the same code is already integrated in the shop system. Shopware also allows you to store the cache in Redis without additional modules.

memcached (Zend_Cache_Backend_TwoLevels) + Zend_Cache_Backend_File

Magento 1 allows the cache to be split into two parts according to fast and slow backend. This allows part of the cache to be placed in a fast backend that does not support tags. The remaining part of the cache, which needs the tags, is placed in another, slower backend. In this case, it is possible to use the key-value database memcached as a fast backend without additional modules. For this test we used memcached as the fast backend and the file system as the slow backend.

memcached (Zend_Cache_Backend_TwoLevels) + Cm_Cache_Backend_Redis

As a fourth variant, we tested how memcached and Redis behave in combination. We have therefore activated the TwoLevels backend again, but in this case entered Redis as the slow cache.

After configuring the cache benchmark, elements were then created for the benchmark:

The actual benchmark was then carried out and evaluated.

We use a cache size of 200,000 keys in the test, which in our experience corresponds to an average Magento shop.

Results

Diagram 1: Delete operations per second for the selected backends. (The more operations, the better the throughput.)

Diagram 2: Read operations per second for the respective backend. (The more operations, the better the throughput.)

Diagram 3: Write operations per second, per backend. (The more operations, the better the throughput of the write operations.)

In the case of a write operation, a single element is stored in the cache; in the case of a read operation, it is read. A delete operation removes all elements from the cache which are provided with a certain cache.

In contrast to the other backends, the support of complex data structures in Redis makes it possible to use Tags to find. For this purpose, the cache backend creates a set for each day in which all associated keys are saved. There is also a set that includes all Tags contains. In this way it is extremely efficient to remove all keys belonging to a tag. This method is not available with file system-based backends or with memcached, so that the number of delete operations on these backends is significantly lower (see Diagram 1). Due to the inefficient structure of the standard Magento application cache (default setting), the throughput of delete operations is extremely low.

This affects the speed. The _Cm_Cache_BackendRedis module achieves a significantly higher speed than the other backends, as delete operations can be carried out quickly using any tags due to the structured data. In the case of file system-based caches, on the other hand, it is much more laborious to remove all elements of the cache that have a specific tag, since the corresponding files cannot be found directly via a tag.

in the Diagram 2 the number of read operations can be read off. It is highest with filesystem-based cache backends. This has to do with the very efficient file system management of Linux, which automatically keeps files in memory if there is free space. In our test environment we made a very high RAM of 32 GB available so that there is always enough storage space for our test. This allows files to be read extremely efficiently. In a real world situation with limited RAM this is not always the case. In such a case, Redis has the advantage of ensuring that the application cache is in RAM at all times.

Diagram 3 shows the number of write operations per second. The differences between the individual backends are the smallest in this throughput test. However, with its simple data structures, Redis offers the advantage of quickly storing data in RAM.

Conclusion: is Redis the best option as an application cache?

At first glance it can be seen that Redis is less performing in the area of ​​read operations due to the lower throughput. However, Redis has a higher throughput in the area of ​​write operations and a significantly higher throughput in the area of ​​delete operations.

The great advantage of Redis can be seen when comparing the number of operations per second with each other. The relevance of the individual types of operation must be taken into account. The relevance of delete operations is much higher than that of the other operations. This is due to the fact that elements from the cache lose their validity every time a product is purchased in the online shop and must therefore be deleted. Then the previously deleted cache elements must be rewritten, i.e. write operations. Here, too, Redis shows a higher performance. Because the throughput of write operations is higher with the Redis backend than with the other backends.

As an application cache in the area of ​​read operations, NVMe SSDs have the highest number per second. NVMe SSDs offer a lower throughput for erase and write operations. However, these operations have more influence on a shop with a high conversion than the read operations. In this respect, the delete and write operations are more relevant in e-commerce. For online shop operators, the values ​​of the delete and write operations are therefore particularly important, in which, according to our measurement results, Redis offers clear advantages. For this reason we recommend the implementation of Redis for online shops. Especially for medium and large online shops.

For the key value database memcached as the backend, the results in both two-level deployment scenarios show that memcached does not have a better overall result than the other backends. Only memcached offers a higher throughput for the read operations. Since memcached has a higher level of complexity in setup and operation as an application cache, we cannot recommend memcached after our tests, as this setup effort does not result in any great advantage for online shops.

For Showpare, Magento 1 and Magento 2 online shops, we recommend using Redis for efficient storage of application caches in the main memory after our test. For smaller shops, the NVMe SSDs already lead to a sufficiently good performance. The combination of fast SSDs for loading PHP files and static content as well as Redis as an application cache represents the best environment for high-performance online shops.

Tabular recommendation for the different shop systems and shop sizes

Shop systemDelivery status of the manufacturerOur recommendation for application caches
Shopware - small shopFile systemRedis or NVMe SSDs
Shopware - medium to large shopFile systemRedis
Magento 1 - small shopFile systemRedis or NVMe SSDs
Magento 1 - medium to large shopFile systemRedis
Magento 2 - small shopFile systemRedis or NVMe SSDs
Magento 2 - medium to large shopFile systemRedis

More posts in this blog series

Cover picture: raspirator / iStock.com

Do you have any questions, requests, criticism, suggestions or just want to give us your opinion on our blog? Here you have the opportunity to contact us directly.

Send e-mail