Pulished: May 5, 2022
I’m happy to announce another big development to our server-side application architecture. It follows on neatly from our last article on the migration to object storage which was a prerequisite for this work.
Analysing our customer’s audio files is quite intensive, requiring quite a lot of CPU and RAM resources to generate a report within a reasonable period of time.
With our current number of users, uploads are quite irregular so if we were to keep large machines running they would be idling for the vast majority of the time. As we grow, it’s also possible that multiple users will upload while someone else’s report is still processing — in this case we don’t want the subsequent user’s reports to be delayed.
With our new architecture, more worker machines get added when more work needs doing and then they are terminated (switched off) when not in use. We use Kubernetes as our container orchestrator running on Google Cloud using their Autopilot feature. Traditionally you would have to fix the size of your Kubernetes cluster upfront but Autopilot gives us the flexibility and simplicity we need as it deals with adding more node machines to the cluster as we increase the number of pod replicas in our specification.
The direct benefits are:
There are, however, some downsides:
Our initial benchmarking of the new system (from a 0 workers start) is showing that reports are generated in about 6% of the audio file length. For example an hour long podcast show would take less than 4 minutes from upload to report. We can tweak the resources for speed and efficiency as we gain more insights.
I hope this dive into our technical architecture was somewhat interesting to you. I think we have a strong foundation now to cope with all the processing we will hopefully have to deal with in the future.
Audio Audit is an automatic benchmarking and proofing tool which checks the quality of your podcast MP3 files, giving you peace of mind before you publish.
It checks things like loudness, silences, restarted sentences, encoding, swearing and metadata.Learn more ⇢
Creating an account only takes a couple of minutes. You’ll soon be able to start uploading your own audio files and improving your shows.