Introduce advanced scheduling during bulk deployments
To prevent resource exhaustion when deploying many bulk application instances at once, a more elaborate, user-controlled, scheduling mechanism needs to be introduced.
In the initial iteration, it should be enough to simply give the user performing the bulk deployment an option to select the parallelism level using a dropdown list. The max value present in the dropdown list can either be statically hardcoded (the value can be discussed further) or, alternatively, configurable as an environment variable. Which option to support is up to the implementation, but the end-goal is the same, to have a max value acting as an upper limit to the number of deployable instances in bulk at a given moment.
During the bulk deployment process itself, only the specified number of instances will be deployed in parallel. If there are more instances to be deployed, they will need to wait until the current batch is finished before moving forward with the next one. It would be best if the existing readiness check performed by NMaaS can be reused so that the next batch is started only after all the applications from the previous batch have been confirmed to be in a healthy, running state. Preferably, a visual indicator would be available on the Portal to track the deployment progress.
As discussed previously, this upper parallelism limit should only apply to the scope of the bulk deployment at hand and not to other bulk deployments or individual, end-user initiated deployments that might happen at the same time.