> The idea is to only allocate a running instance while there is at least one active user of that instance.
That first user is going to have a bit of a wait while you turn on their server. Maybe you keep some empties warmed up that just need a reconfig and restart so it’s fast.
Personally, I’d only do any type of multi-tenant if the cost of some micro instance behind an auto scaler was negligible to the value of the contract anyway.
I wrote a system at work that does this for VS Code instances - on a commodity server and not much optimisation effort it goes from a click to the UI starting to appear in about 10s (mostly thank you to Firecracker and Alpine) . There's a loading screen that's instant though, and probes for when the VM is ready.
I think that would work fine for a lot of other apps, at least where you're looking to start a lengthier session.
I'm copying a 4GB root filesystem (on ext4!) per connection, so that's a couple of seconds, more like 10s on my laptop. The boot time for the kernel is about 2s to userspace initialisation. Then the userspace is running node.js to bring OpenVSCode up (2-3s?), starting a couple of other processes, and then the front-end is polling only every 1s.
It's all on 1 server, just a Go https proxy launching firecracker on demand, no k8s.
If I were optimising, I'd first make sure the root filesystem copy ops happed on a CoW filesystem, then change the readiness polling for a "long" poll that's under the server's control. And next buy a faster server :) But I'm using a relatively heavyweight userspace app, so I'm curious if you can see there might be other gains to be had.
Our application is using VS Code as an environment for programming exercises. So even 30s would have been fine when people are using it after that for 1-2 hours.
If I could get it down to 2s I'd certainly enjoy testing it more!
Ahh fair. I was thinking along the lines of storing the 'base' VM as a memory snapshot and CoWing it per connection to save some initialization overhead perhaps...
That first user is going to have a bit of a wait while you turn on their server. Maybe you keep some empties warmed up that just need a reconfig and restart so it’s fast.
Personally, I’d only do any type of multi-tenant if the cost of some micro instance behind an auto scaler was negligible to the value of the contract anyway.