People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware.
Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to websocket connections on any connected client, and archival to PostGres for historical processing. Total hardware required was 1 m4.large + 1 r5.large AWS instances, for a bit under $200/month, and the boxes would regularly run at about 50% CPU.
What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app?
I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page.
50req/sec for a Html file is super low which shows that a $4 month server cant do much actually. So yes this is enough for a blog, but a lot of websites are not blogs
One of my proudest moment in my career is when I lowered our app processing time from ~8hrs to 17 minutes. When I deployed my first update, it reduced it to 2 hours. The sysadmin immediately contacted me that there was something unusual. I confirmed the results but he was skeptical.
Then with my second update, he told me that the app must be broken or that the script must be dying. There is no way it could complete this fast.
What was the issue? We processed terabytes of data. Each and every single line processed created a new connection to the database and left it hanging. A try catch was added when the connections failed and restarted the process. Removing the connection from the for loop and properly handling it reduced the time drastically.
And... why would you loop through millions of records when you can use batches? Also this was a phperlbashton* script. I turned it into a single PHP script and called it a day.
As a consequence, backup time was reduced to 2 hours as opposed to 12 hours (no one was allowed on the website until the back up was done).
Modern machines are incredibly fast.
* PHP/Perl/Bash/Python
Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware.
https://en.wikipedia.org/wiki/C10k_problem
"By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over 2 million connections (WhatsApp, 24 cores, using Erlang on FreeBSD),[6][7] 10–12 million connections (MigratoryData, 12 cores, using Java on Linux).[5][8]"
Although I do understand the boxes listed above have more resources then the VPS you are using. I am also not criticizing your write up, or results, bench-marking is in general interesting to do. I just wanted to provide some additional information.
I understand your excitement for being able to handle a decent amount of requests on such a small server, but just like many other websites that get on the frontpage of HN, your site is taking multiple seconds to load for me, depending on when I refresh.
As you said in your post, adding caching to your site increased your throughput by ~20% (or +10/req/sec). What you and other sites seem to lack is a more distributed caching, a la CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only really work well for a static site, however as mentioned in your post that's essentially what you're serving.
While I'm all for having a free-as-in-freedom hosting solution and keeping things lean, the internet is a fickle beast, and nothing looks worse for a company who posts on HN when their technology-oriented site can't handle a few thousand requests per minute. (Or in this case, when a blog claims to handle 4.2M requests a day -- 2.9k req/min)
All these "X requests per unit time" posts are starting to make me want to break out some of my experimental code... I have some services that can process several million events per second. This includes: compressing the event batch, persisting to disk, validation of business logic, execution of all view updates (state tracked server-side), aggregation and distribution of client update events, etc. These implementations are easily capable of saturating NVMe flash.
If you want to see where the theoretical limits lie, check out some of the fringe work around the LMAX Disruptor and .NET/C#:
https://medium.com/@ocoanet/improving-net-disruptor-performa...
You will find the upper bound of serialized processing to be somewhere around 500 million events per second.
Personally, I have not pushed much beyond 7 million per second, but I also use reference types, non-ideal allocation strategies, etc.
For making this a web-friendly thing: The trick I have found is to establish a websocket with your clients, and then pipe all of their events down with DOM updates coming up the other way. These 2 streams are entirely decoupled by way of the ringbuffer and a novel update/event strategy. This is how you can chew through insane numbers of events per unit time. All client events get thrown into a gigantic bucket which gets dumped into the CPU furnace in perfectly-sized chunks. The latency added by this approach is measured in hundreds of microseconds to maybe a millisecond. The more complex the client interactions (i.e. more events per unit time), the better this works. Blazor was the original inspiration for this. I may share my implementation at some point in the near future.
This is a good, simple way to show how much can be done with modest resources.
Sometimes we see people fetishizing bigger and faster, then gatekeeping when people want to do the same work with modest means, whether it a four quid a month hosting service or a first generation Raspberry Pi. Not everyone has the money or desire for bigger & faster, and it's nice to see that here.
Absolutely. Even a terrible Wordpress instance can be beautifully (and transparently!) cached behind either nginx or varnish with ease, in which case you’re just serving static html pages and can probably handle any traffic you are likely to ever get.
I'm hosting my static blog site on a physical Raspberry Pi 3B+ powered by Solar [1].
That blog post got hugged by HN but it didn't even raise the CPU above 10% on a single core.
And a Raspberry Pi 3B+ is dog slow. And severely limited by bandwidth, unlike the Raspberry Pi 4B+. (But it uses less power so that's why I use a 3B+).
However I have another point to make. Professional rack-mount servers from HP and Dell can be had second hand for dirt cheap and you get a ton of CPU (20+ cores) and an ocean of RAM for next to nothing.
For many applications, an old Gen8 or similar Dell server will perform more than adequately. Even more so if you have a little bit more to spend on Gen9.
They are so cheap that you can like buy four to eight, sprinkle them across two different datacenters and even if one breaks, you won't be in any hurry.
[1]: https://louwrentius.com/this-blog-is-now-running-on-solar-po...
That estimate can be verified with load testing systems like Artillery. My theory is that things would break far sooner than estimated along the following lines:
- Too many WSGI connections if the timeouts aren’t tweaked
- Too many database connections, especially without caching and tuning
- on the Apache side if MaxRequestWorkers isn’t set there will be memory issues with 1GB RAM
- the disk could easily hit IOPS limits, especially if there is a noisy neighbor
It’s not likely all or any of these things will hit IRL, but that all depends on traffic and usage patterns. It matters not, if you were getting 4.2 M requests each day you’d be in the Alexa Top 1000 and could probably shell out for the $8 server :)
All these examples show even more why software engineer(not developer) is a discipline where 10x salary difference can be seen between the best and the worst. For the 10x you are paying, you are getting a (n^2 - logN) times performance gain, especially when you are dealing with problems with large amount of data.
However, relying on people themselves is often not the best stable solution. I am wondering if all these N^2 mistakes people made can be prevented by innovative means like language features, framework improvements, tooling and etc. And I'm talking about prevention, not the post mortem perf measure and fix kind
In contrast, the Australian 2016 National Census crashed and burned since the system could not cope with over 250 requests a second.
Parliamentary enquiry:(PDF) https://www.aph.gov.au/DocumentStore.ashx?id=0a7f6bd5-8716-4...
https://www.zdnet.com/article/census-2016-among-worst-it-deb...
https://www.theguardian.com/australia-news/2016/aug/10/compu...
There is a difference between being able to handle 4.2M requests a day, and handling 4.2M requests per day.
Visitors don't come neatly one after the other. You might only have 1M requests a day but get random spikes with 100 requests at the same time.
#1 on HN and still up. That speaks for itself.
The comments in this thread surprise me quite a lot. I suppose it shouldn't, but it does. This post + the response calls to the surface how badly basic system operations knowledge is needed in the industry and how much of it is missing from the toolkit of most developers.
I'm not sure there was ever an argument saying otherwise. The ease of processing X million requests is heavily dependant on what those requests actually do. Trivial use cases shouldn't be a surprise to have high throughput.
And here I am building a simple API using Lumen (Laravel's stripped down, hopefully faster cousin) getting response times that are just abysmal.
The raw queries themselves are fast enough, but for some reason running them in a framework, transforming them in to a Resource and dumping it as json takes so long that I'm scared to find out what this super popular framework is even doing under the hood.
Once I learn enough Python I'd like to compare its performance to something like FastAPI. But even that probably won't come near what these recent posts are describing.
(Disclaimer - it's just a side project and I haven't really looked in to making it faster)
Working on other companies' mobile apps, about half the performance problems I've discovered have been down to some accidental crazy, like initialising something you only need once, in a loop, in 3 different places (because the code has become so unnecessarily complicated that no one really knows what it's doing). The rest are due to some piece of code accidentally blocking the UI thread.
A well written mobile app doesn't really have any need to be sluggish at all, including smooth animations and fast scrolling lists, it was doable 10 years ago, it's doable now. (*I don't know about games).
But unlike on the server side, the accepted wisdom in most places I've worked at is that the answer to the performance problems is: a new framework.
(I feel like this is a lie that developers tell the business side, and maybe themselves. It avoids having to explain that software is hard, sometimes you don't get it right the first time, and if you don't spend time and effort tending to it, it can turn into an ungodly and expensive mess - and that's got nothing to do with the hardware or the framework)
Well the front page of HN won't get you 4.2M views today, but it's a pretty good real world test!
Re: benchmarking, sometimes the bottleneck is the machine or server that issues the requests, not the receiver that you are testing. To figure out your actual capacity, you sometimes need multiple request servers or a more powerful request server. This was the case for a project I did a few years ago. Not a critique of the blog post, just remembering something out loud
His site, https://peepopoll.com/, took about 10s to load for me. It’s also good to chart other metrics like response times while you benchmark. Requests per second isn’t the same as a low response time
> These benchmarks show that a very cheap server can easily handle 50 requests a minute
I think you mean a second, but yeah, old tech is fast.
I find it funny when I read “raw html” emphatically, as if it was akin to writing assembly.
Combined with a CDN and you can do a lot more, I like to think in terms of origin requests rather than raw req/s. The problem I have is getting people to really understand how caching works at the cdn and browser layers and design their frontends and backend responses around that. There is also lot you can do with edge compute to clean the incoming requests before the CDN evaluates them to increase cache hits. Even if you are trying to give a "real time" view of data, caching it for even a second and allowing stale data to be served while it is updating can reduce origin requests significantly. I've seen people hammer sites hundreds of times a second looking for changes that only happen once every fifteen seconds or once per minute - the best thing you can do for yourself is handle all those requests at the CDN level (eventually you'll do log analysis and see the activity and can take other measures, but in the mean time don't let all of those requests go to the origin). Your CDN is probably giving you better rates for network egress then Amazon or Google anyway - the later are more focused on incentivizing you to use their ecosystem exclusively by penalizing you for sending data "outside". Cheap VPS hosts discourage you exceeding your bandwidth allocation because they are overselling their capacity and heavy usage upsets that - so again you want to shift as much as you can to your CDN.
It’s the database part that gets expensive for web applications. Serving up static web pages is absolutely trivial for modern servers.
The database is also the part that doesn’t easily scale, unless you pick a highly scalable database from the outset, and those have their own complexity and tradeoffs as well.
That’s why I believe every project should start with a bulletproof model of how the database will work first, then fill in the other details from there.
It’s not always as easy as picking Postgres and calling it a day, unfortunately.
50 rps not that much, though of course easily sufficient for many situations. This is also Django which certainly isn't the fastest choice. I played around with it a long time ago and liked it quite a bit, but you don't choose Django for performance but for the other benefits.
I'm really more surprised that static serving is so slow at 180 rps. This should be able to easily saturate the network, statically serving files is very, very fast. From what I see in the blog I doubt that the files are very large, so there is probably some other bottleneck or I'm missing something here.
If you host it on GitHub Pages, GitLab Pages or Vercel it's even ÂŁ4 less :o
I run a FOSS Pi-Hole esque public DoH resolver in the most expensive way possible (over Cloudflare Workers) and it costs $2 for 4M requests. Granted the CPU slice is limited (50ms) but the IO slice (30s) is plenty for our workloads (stub dns-resolution).
The reason this is cheaper in a sense is because Workers deploys globally and needs zero devops. Per our estimates, this setup (for our workload) gets expensive once the request range goes beyond 1.5 billion a month after which deploying to colos worldwide becomes cheaper even with associated cost of devops.
When my stuff has been shared on HN and Reddit, I’ve seen peaks of around 500 rps (according to google analytics), so if you can hit 1k rps you’re almost certainly ready to weather any kind of viral sharing that a blog might experience. Several krps on cheap VPS hardware is easy with a good compiled language backend (Haskell’s Warp is a good one). If you use a node/python/ruby/other interpreted backend, you will need aggressive caching through a reverse proxy.
I hosted draftsim.com on a $3/month hosting plan for a few years.
We served 500GB of data the first month.
I imagine that the hosting company lost money on us (but they never called to complain).
this story remind me, the dot com bubble. dotcom companies bought servers from Sun Microsystem. they needs to handle the large traffics that "PC" server can't handle.
anyone remember Cobalt server?
https://en.wikipedia.org/wiki/Cobalt_Networks#/media/File:Co...
You have clearly stated in your footnote that if anything goes wrong then the given numbers can go down. To be honest, that's what happens all the time. Developers do something wrong and number of requests just drop down to the floor.
Only static websites are the one which handle large amount of requests at low cost. Web hosting providers don't make money out of those clients, so they run shared plans.
Impressed that it is front page of HN and it isn't "This page cannot be displayed" with that kind of premise. Props.
My Java HTTP app. server manages that on a Raspberry 2 at 2 watts serving this reddit clone: http://talk.binarytask.com
Really we need to compare apples to apples (how many watt)!
Most of you have an external IP address, open port 80 and put it to good use before they put you behind a shared IP!
Nginx can handle ~250M requests a day out of the box, and ~600M by tuning a few parameters. *
My website https://www.v2ph.com
$6 VPS can handle 500,000 requests daily
On this server, I have PHP-fpm workers, nginx and MariaDB
The average CPU usage about 30%, load average is about 0.5
Most of the times the more interesting question is not really how to make a server that can handle 4.2M requests a day, but how to make something so useful that it gets more than 100 pageviews a day.
"If we can handle 50 requests a second that means we can handle 4.2 million requests a day."
Not really. Real world traffic won't be uniform over one entire day. 50 QPS would be more accurate.
I was skeptical that it would fail under real situations (the calculation says it's 50 requests/second). But it looks like reaching the top of HN didn't crash it.
The new metrics for server performance is "unique buzzwords per paragraph". If you don't write a 4 page blog post with at least 4-5 ubpp then it is shit.
I used to be on the top 500 of Alexa with a shared hosting server running PHP. Never had any issue. Mind you, I wasn't doing anything complicated, but still.
My 1cpu 2gb ram server: - nginx - slack bot behind Django - OpenVPN - Minecraft server (20 players) - tunneling (reverse proxy for local dev)
That's a good spec for 4 bucks. With cloud hosting you might be able to push the cost down a bit with less CPU resource and memory.
Can it?
> Service Unavailable
> The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
160 Request /s for a No DB Call, Simple File Serving on a Single Core System.
Why am I the only one not impressed by this.
There are a few comments in here that predictably suggest that simple static sites can handle large request rates easily.
Sure, that's true - but to try to progress the conversation: how would you measure the complexity of serving web requests, in order to perform more advanced cost comparisons?
(bandwidth wouldn't be quite right.. or at least not sufficient - maybe something like I/O, memory and compute resource used?)
Hey! Someone else from Jersey! Nice one
So 50req/sec. I'd hope it could handle a lot more than that!
apache + wsgi isn't even close to being the most performant webapp server software, either. Bet he'd get 5x the performance out of nginx + lua on the same virtual hardware.
I wonder what the 99th - 95th percentile response times look like…
Put Wordpress on it, and do a new battery of TTFB tests ;)
But can it run k8s /s
That's less than 50qps. I can't count that low.
It feels to me like most websites out there could run on way less hardware, if only people would embrace a few things.
#1 Minimalism. You don't need 400 KB of JS to display some mostly text content to your users with some interactivity sprinkled in.
You don't need to reinvent office software, or very rich text editors in browsers, stop using the web as a universal delivery platform/mechanism, because that's not what it was meant for. When browsers will ship integrated dependencies so that even CDNs don't need to be hit (like versions of jQuery, Bootstrap and numerous JS frameworks as well as WASM code like Blazor which contains a .NET runtime), then you'll be able to do that, but arguably that will never happen.
Use the web as a platform for displaying primarily text content with the occasional images, forms and a little bit of interactivity sprinkled in. Most sites out there simply aren't and shouldn't be like this (that said, when you have exceptional reasons for throwing aside that suggestion, do so): https://geargenerator.com
#2 Static content. You don't need to use Wordpress, Drupal, Joomla or many of the other CMSes out there, since they can get really heavyweight with numerous plugins and are not only a security challenge, but are also problematic from a performance perspective.
Consider using static site generators instead. When reading an article of yours, the DB shouldn't even be hit, since most of the article contents are unlikely to change often, so you should be able to pre-render each of the article versions as a set of static HTML and use the common JS/CSS that you already have for the rest of the articles. Furthermore, it's easy to just jump into CMSes and introduce ungodly amounts of complexity, all of which cause your back end to process bunches of code for each request. Static files don't have that drawback.
#3 Caching. Know when and what to cache, and how. Images, JS files, CSS files and even entire HTML pages should be cache friendly. Know which ones aren't, make exceptions for those and cache everything else.
Not only is it not necessary to hit the DB for many of the pages in your site at all, but also sometimes you shouldn't even hit the back end either. The most popular pages of your site should just live in a cache somewhere, be it within your web servers or a separate solution, so that they can be returned instantly. HTML is good for this, use it.
Furthermore, know what cache policies to use. Sometimes even the cache resources shouldn't be redownloaded, if the user already has these resources loaded from a different page. Use bundle splitting responsibly, extract common functionality in easily cacheable bundles and set the appropriate headers.
And yet, i've seen a surprising amount of ignorance in regards to caching, static site generation and even how large webpages have gotten: https://idlewords.com/talks/website_obesity.htm
I don't claim to know it all, but working towards the goal of efficiently using pages should definitely be viewed as an important one: be it because you want to pay less for your infrastructure, or care about the environment, or even just want to manage fewer nodes.
Instead, nowadays far too many orgs just try to be the first to market and ignore the engineering based approach to ensuring that the solutions are not only functional but also sustainable. That saddens me.
tl;dr: a machine with 2 GB of ram that is doing pulling a blog post out of a DB and passing it through Django can do ~50tps
i'm running my SaaS for 10 years already, and still using the same $5 plan a month
HTTPS and certificates? i have no clue how to setup that, i use dns from cloudflare and they have it all automatic for free
if your employees are asking you to pay ton of money for your services, hire someone else
You can also handle 50 requests per second on a 66MHz 486DX2 with 16MB of RAM and a 10Mbit/s network card. Not with modern "I have infinite resources" software, but we used to handle more than that traffic regularly in the early 90s.
ROTFL Wikipedia 2010 handle 7mln request per seconds suqids in USA Europe in Japan. Some random dud who do benchmark with 0.00001 seconds TCP on localhost "HEllo World" only ycombinator lamerers can believe this without asking questions.
Pepe ftw
I might get downvoted for not getting on the 2000s style of development bandwagon, but do you really need a web server to serve static text?
> Not taking into account any issues that may occur around CPU/RAM/Disk IO due to sustained levels of traffic as well as bandwidth issues
congrats?
How did this make it to the number two spot on HackerNews?
Dont mean to be the negative Joe but you dont need a webserver if you're serving a static-able website.
You don't need ÂŁ4. As long as you can structure your site as a set of static files with interactivity done client side, even if some of the files change every minute or two, you can serve everything for $0 with Cloudflare in front of any free host. I've served 1M pageviews a day for $0 with Cloudflare + App Engine free tier and there's no reason it wouldn't scale to 100M or beyond.
This one's even cheaper:
My ÂŁ0 a month server can handle 4.2M requests a day [1]
[1] https://ahamlett.com/blog/post/My-%C2%A30-a-month-server-can...
When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Rica, and once a user logged in, they were socketed to a server that handled the gaming action in the Isle of Man. Graphics and sounds were still sent from the Costa Rica server. I didn't have a gaming license in the IoM, though - that was around $400k to acquire legally. So I found a former IoM MP who was a lawyer, who wrote a letter to the IoM gov't stating that we didn't perform random calculations on their server, thus weren't a gambling site under IoM law. Technically that meant that no dice roll or card shuffle leading to a gambling win or loss took place on that server. So the IoM server handled the socketed user interactions, chat, hand rotation and tournament stuff. Also the bitcoin daemon and deposits/withdrawals. But to avoid casino licensing, I then set up a VPS in Switzerland that did only one thing: Return random numbers or shuffled decks of cards, with its own RNG. It was a quick backend cUrl call that would return a fresh deck or a dice roll, for any given random interaction on the casino servers. The IoM server would call the Swiss server every time a hand was dealt or a wheel was spun; the user was still looking at a site served from a cheap web host in Costa Rica. And thus... yeah, I guess I handled millions of requests a day over a $6 webserver, if you want to count it that way.