> Dragonfly expects you to have the latest hardware[6] and the latest kernel[7] on all your servers. I find this outrageous because not all servers run 5.10
I don’t understand why this is “outrageous”? There’s performance features and better hardware utilisation in newer kernels? If you want the performance offered by the applications built on top of these, then arguably you should be prepared to run the hardware and software required to achieve this.
> Dragonfly expects you to have the latest hardware[6] and the latest kernel[7] on all your servers. I find this outrageous because not all servers run 5.10 and a majority of them are still using long-running branches on 4.x.
I don't see anything wrong with that. Either your kv store is performing OK or can be scaled using the usual methods and it's not holding you back... or you actually critically depend on the fastest kv performance you can get. In the second case, you'll be likely happy to dedicate some time to standing up a couple of machines with updated kernels.
If you're running a really old version of Linux, you're probably giving up on many-many performance optimizations.
> I find this outrageous because not all servers run 5.10 and a majority of them are still using long-running branches on 4.x
I don't think it's outrageous to ask people to upgrade 5.10. This was released over 1.5 year ago and there's no reason why your corporate overload shouldn't consider migrating to this, or at least have someone working on qualifying their workloads.
KeyDB also has another key advantage over Redis: if your data set exceeds RAM, it will spill to disk (SSD) and maintain good performance.
Hi, I am the author of Dragonfly. Following this post, I performed the loadtest on instance m5.4xlarge, recorded myself and posted the video here: http://assets.dragonflydb.io/videos/video2609676488.mp4
the command I used to load was: "memtier_benchmark --ratio 1:0 -t 28 -c 20 -n 100000 --hide-histogram --distinct-client-seed -d 256"
I did the recording to clear out any doubts whether my benchmark results are real.
Sorry, you say you ran the benchmarks, but where in this post do you link to your results?
KeyDB author here. Lack of stable kernel support was a major reason we didn't do io_uring yet. The goal for KeyDB is good performance out of the box not in specific benchmarks on specific hardware/kernel versions. As io_uring becomes mainstream we will start to support it. In 6.2.3 we have async commands which permit reads to happen outside of the global lock. In some cases it can take extra time for writes from other clients to be visible which changes the consistency model so it is off by default but dramatically improves performance when enabled.
Doing a fresh rewrite would have enabled a nicer threading model, but it would make it harder to maintain compatibility. Plus at the time it was more to suggest to Redis multithreading was both desirable and possible which were disputed back then.
This article interests me as it has been a bucket list todo to get a good sense of some of the key/value offerings, thank you for posting them.
Quick ask: I don’t see “some” of the other offering out there like MemCached… what was the criteria used to select these? I don’t see any source of how the test where run, specs of the systems, how the DB where set up, etc. Would be very valuable to have in order to attempt to re-validate these test on our own platform. I also came back and saw some of your updates to the outcomes, While the Get/SET numbers are provided, I don’t find a reference to time, resource utilization, or time which would IMO make the data more valuable in understating how the numbers where achieved. * I might be a factor of the benchmarks themselves, but would add value IMO to your fantastic post.
Redis and sqlite are two projects I can't really believe exist. They both are absolutely magical.
One important thing I would look at is "is there independent managed hosting options", in case at some point you have better things to do than managing database and having to care if it's the right hardware and kernel version. I know at least redis shines on this, with managed offering on all major cloud provider.
I'm trying to get my workplace to accept Redis for simple caching but they're not open to the idea. Instead they're all in Hazelcast but real stingy about who gets to use it and for how much memory.
I just want to cache some objects and avoid hammer the DB on every request :´(
How about including Pedis in this? It is a Redis partial reimplementation, written around Seastar: https://github.com/fastio/1store
Also if you can add amazon memory db to yor list of benchmarks, would also be interested to know more.
Redis can do 1M or even 2M requests per second on a decent box. What payload are you using? Can we see the script you are using to generate the traffic and measure the performance?
I have run the benchmarks myself for each store.
I feel like I'm missing something here. Which benchmarks did you run, and what were the results?
Why do you consider single threaded as a reason to have less throughput?
Are you looking for throughput or speed?
Also, what are your needs?
Have YOU tried benchmarking any of them locally or in the cloud?
Anyone knows where can I find a good helm chart for KeyDB?
>Redis, which I'd like to call the "original" key/value store
My old friend Memcached just came by and want to say Hello. He is still alive and kicking. Netflix has been treating him well.
> I was NOT able to reproduce the 25X speed that Dragonfly claims
The Dragonfly benchmark which showed 25x faster was run on c6gn.16xlarge which has 64 vCPU. The m5.4xlarge you used "only" has 16 vCPU. Since Dragonfly is achieving its performance by using multiple threads it's not weird that you weren't able to replicate its performance gain.
In my opinion, when it comes to these types of multi-threaded benchmarks it's much better to separate the "baseline, one-process performance" from "how it scales with number of processes". E.g. if you first pin Dragonfly to only run on a single core you can find the baseline performance compared to Redis, and then you can run different benchmarks of Dragonfly with increasing number of cores to see how it scales. This should give you much better understanding of how the system behaves. (The release of Dragonfly actually prompted me to write a blog post on this: https://t.holmium.no/dia/apples-vs-oranges/)