Improve autoscaler's observed average calculation. (#1194)
Improve autoscaler's observed average calculation. (#1194)

* Take average per pod before aggregation.

The autoscaler takes all probes over a certain window, either panic or stable respectively, to calculate the observed concurrency in the system. If you simply take all probes and calculate the average across all of them, newly joined pods get an unfairly low share in the average calculation. That means that even though the system has successfully scaled up to more pods, the "old" and overloaded pods' measurements are given an unnaturally high weight vs. the newly joined pods.

This aims to solve this issue by not aggregating over all probes but aggregating an average per pod first. That way, all oberves pods in the system get an equal share in the observed concurrency calculation and the autoscaler scales more precisely.

* Cleanup autoscaler code.

* Some more cleanup.

* Refactor into constructor.

knative/serving,knative/serving