Bursty or not, if the aggregated number of requests per day is in millions not hundreds, AWS API Gateway & Lambda is significantly more expensive than the more traditional options.
Not if you have to serve 1 million requests in one second (with low latency) and then don't get any requests for the rest of the day. On the contrary, traditional options will probably end up costing a lot more since you'll need to have them ready in advance to serve the requests instantly and so you could be paying for the entire day of capacity or at least several hours.
Low latency for 1m requests in one second? Yeah, Lambda won’t help you with that. Their cold starts will stop you dead. Particularly if you’re in a VPC and they have to allocate servers with the appropriate ENI. Not to mention, you’ll be hitting concurrency limits.
Sure, if you ping your lambdas frequently to keep them from having cold starts, and you’ve worked with Amazon to increase your concurrency limits 100 fold, you might be able to handle 1M requests in a second. But then you’re being charged a lot more than just for those 1M requests.
Add a couple of seconds when spinning up an ENI. New nodes in a VPC are around 3 seconds (and our last EBC with Amazon indicated that it won't get lower than that for awhile). It's worth noting that this impacts ECU scaling for Aurora Serverless as well.
Why would you need to put a Lambda in a VPC? For access to a database, other EC2 bound resources, or overall network security reasons (such as outbound network monitoring).
You may be interested in the latest news re: allocating ENIs in VPCs for Lambda[1]. They've finally got the cold-start situation a little more under control in that scenario.
Here's the meat, to save you a click:
> AWS says that the one-time setup of the shared network interface can take 90 seconds to complete. But because the network interface gets created when Lambda functions are first created or VPC settings updated, the connectivity-related latency during cold start approaches zero. The same goes for function scaling, which no longer requires new network interfaces for each new execution environment.
All Lambda does is spin up a bunch of containers to handle that. Why is that any different than traditional options? You would just horizontally auto scale in the exact same way. You don't HAVE to pre-provision anything.
Plus, receiving 1 million requests in one second, you'll have cold starts for every new container that gets created for you, so achieving consistent low latency would be impossible since you don't control that.
No.
It depends on the IoT application. The one I'm involved with (smart metering) produces data at regular 15 minute periods 24/7.
This is a very predictable and constant traffic load and not at all suitable for serverless/lambda.
In fact, its the complete opposite.
Many IoT applications are very predictable (anything with regular sensor-readings for example or with repeatable patterns such as people returning from work and turning things on).
People 'returning from work' and 'turning things on' are exactly the kind of situation where serverless/lambda shines.
See also: traffic jams and people switching on their electric kettles en-masse during intermission of major soccer matches. The first disrupts whole economies and the second has given rise to 'fast pumped storage' at the electrical grid level.
Just because you don't see the accumulated effects does not mean they aren't there.
What you need to consider [is] that there are a lot of applications like yours hosted by an IoT SaaS. (I work on one.)
So the patterns are (obviously) predictable, but the 'dynamic range' of the load is extremely wide, ranging from stampedes on the cardinal points of the clock face (0/15/30/45 min) to crickets the rest of the time. Some form of elasticity is required unless you are willing to pay for idle servers sized for the episodic peaks.
Thats not really true as generally IoT nodes will manage the traffic they generate to avoid synchronisation with other nodes.
For example, to avoid the 00/15/30/45 issue you offset yourself randomly over a few minutes within that range to avoid overloading mesh & cellular radio networks.
This has a smoothing effect on the traffic as a whole (as seen from the server) which reduces the "dynamic-range" as you're terming it.
What you propose is one way of doing so. Again, I remind you that an IoT SaaS has multiple tenants and these tenants can't be expected to collectively coordinate. And in certain domains, it is not possible to dictate timing requirements to the client tenant and thus enforce the (SaaS) providers traffic requirements.
I don't know the specifics of your architecture, but if your (multiple) tenant applications are sharing code on the node-side, then it follows that they can self-organise in a similar manner.
Specially considering the staggering effect due to predisposition to picking a cardinal direction - 0/15/30/45 minute - in the clock. This is a real issue specially for SaaS offerings in the IoT space.
