How do people use the HTTP interface in production? I’m curious because I found it necessary to use a queue in production in order to handle many requests and smoothly scale up workers, yet every containerization approach I’ve seen uses this synchronous HTTP based approach. Are there any standard containerization methods that rely on queues instead ?

If you’re pickling that much data you should definitely consider using a more appropriate data format. Maybe a database or HDF5?