Forecasting cloud resources before the bill arrives

• An enterprise platform team paid for cloud capacity sized for the worst case, all year, because its usage forecasts were coarse and reactive.
• Deep Netts models now forecast resource demand from the team's own usage history, inside the team's JVM-based tooling.
• The forecasts feed autoscaling and capacity planning directly – no separate ML pipeline.
• The case reports 15–25% lower cloud costs, with better stability during load spikes.

Nobody wants a service to slow down because it ran out of capacity. Without a forecast they trust, teams protect themselves the simple way: they size capacity for the busiest day they can imagine and keep it there.

That margin is paid for every hour, also on quiet nights, weekends and slow months. As usage grows, the safety margin grows with it – and so does the bill for capacity that mostly sits idle.

The training data is the team's own monitoring history: how much capacity was used, day by day. Usage like this is not random – it has a shape a model can learn:

The forecast is one step in the tooling the team already runs on the JVM:

1. Usage history

   Daily demand from the team's monitoring.

   ↓→
2. Deep Netts

   Forecast model

   A Deep Netts network predicts demand for the coming days and months.

   ↓→

3. Capacity plan

   Forecast plus headroom becomes the capacity to keep.

   ↓→

4. Autoscaling

   The plan feeds autoscaling and capacity planning directly.

Because the model runs in the same Java tooling as the rest of the pipeline, there is no data export, no separate Python service and no second stack to keep in sync.

A forecasting network answers one simple question again and again: given the recent past, what comes next? The animation shows it like this:

• Input: demand over the last 28 days, plus the day of the week.
• Output: demand for the next day.
• Rolling forward: the window slides on by one day and the prediction becomes part of the next input – day by day into the coming months.
• A range, not just a line: the further ahead, the less certain the forecast, so the range around it widens.

A forecast like this is a regression – the network outputs a number, not a class. With Deep Netts it is built the same way as any other feed-forward network, with a linear output and mean squared error as the loss:

int NUM_INPUTS = 28 + 7;   // last 28 days + day of the week (one-hot)

FeedForwardNetwork forecaster = FeedForwardNetwork.builder()
        .addInputLayer(NUM_INPUTS)
        .addFullyConnectedLayer(16, ActivationType.TANH)
        .addFullyConnectedLayer(8, ActivationType.TANH)
        .addOutputLayer(1, ActivationType.LINEAR)   // demand for the next day
        .lossFunction(LossType.MEAN_SQUARED_ERROR)
        .randomSeed(42)
        .build();

forecaster.train(history);                         // windows cut from past usage

float nextDay = forecaster.predict(lastWindow)[0];

A forecast only saves money when it changes what is bought. In the animation each month's capacity is set from the highest forecast day of that month plus 10% headroom, in whole blocks of capacity:

In the animation's made-up year, forecast-based sizing pays for about a fifth less capacity than worst-case sizing – inside the reported 15–25%.

A model learns from the history it is given. A spike caused by something that never happened before – a sudden campaign, an outage elsewhere – is not in that history, so no forecast predicts it.

• That is what the headroom is for: in the animation a November spike lands outside the forecast range, but inside the capacity kept.
• Known events (a planned launch, a sale) can be added to the plan by hand or given to the model as an extra input.
• Retraining on fresh history keeps the forecast in line with how usage really changes.

One stack

The team's capacity and autoscaling tooling runs on the JVM. A forecast that runs there too is just another library – the same build, deployment and monitoring as the rest.

No separate ML pipeline

No exporting data to a Python environment, no model server to call, no second set of dependencies to keep up to date. The forecast is read where it is used.

Provisioning

Reserving (and paying for) compute capacity in the cloud.

Over-provisioning

Keeping more capacity than needed, as a safety margin.

Autoscaling

Adding or removing capacity automatically as load changes.

Forecast

A prediction of future demand from past demand.

Window

The stretch of recent days the model reads to make one prediction.

Headroom

Extra capacity kept above the forecast, for what the forecast misses.

Regression

A model that predicts a number (here: demand), not a category.

Mean squared error

The error measure used to train a regression model.

• The customer's case (the customer is not named) – the problem, the approach and the 15–25% cost reduction.
• The data, the 28-day window, the 10% headroom and the November spike are the animation's illustrative example.
• The code sketch follows the regression examples in the Deep Netts cookbook.