Filter 429s from being sent to kibana
Filter 429s from being sent to kibana

As part of setting up ip rate limiting, we're going to be returning
429s for any request that gets limited.

One of the reasons that provoked ip rate limiting, was the amount of
logs being generated when we were being crawled, which was causing us to
blow through out daily storage limit on logstash.

By not sending the logs for requests that are limited, we can avoid
that. The logs will still exists in cloudwatch, and if we have an issue
we can search for them there.

We can also set up a metric in cloudwatch to measure 429s, and then
ship it to hosted graphite and/or add some alerting to it, to give us
visibility on if we're returning a lot of 429s.

alphagov/digitalmarketplace-aws,alphagov/digitalmarketplace-aws,alphagov/digitalmarketplace-aws