Add support for DeterministicOrder failover policy with MSB awareness in memcache
Add support for DeterministicOrder failover policy with MSB awareness in memcache

Summary:
As part of the efforts to support failover with MSB awareness in memcache, we decided to enable Deterministic failover in Memcache since that policy can be MSB aware if properly enabled.

At first sight, the code changes at Mcrouter level seem to be simple, however there are multiple factors to consider:

* Json/config files need to be changed so that DeterminsticOrder failover policy is used.

* Added a new flag named "ignore_normal_reply_index", which can be specified in the section where failover policy is specified. For example:

        "failover_policy": {
          "type": "DeterministicOrderPolicy",
          "max_tries": 2,
          "max_error_tries": 2,
          "ignore_normal_reply_index": true .   --> HERE!
        }

If that flag is set to true, "normal_reply_index" will not be set/used. That variable makes sense for TAO only. In the case of Memcache, we need to skip over "normal_reply_index" for correctness.

* Failover PoolRoute needs to be expanded whenever memcache is used. This is because DeterministicOrder failover expects a list of children of PoolRoute, whereas current config in Memcache would provide two children which turn out to be PoolRoutes: Normal and failover. We can expand the failover PoolRoute via smc.

For example:
```
"failover": "gut.oregon",
```
to
```
"failover": "Pool|smc:twmemcache.gut.oregon"
```

* In order to get more confidence in the recent code changes, we have enabled the McRouter simulation framework with mock memcache server instances, tested and debugged failover flow with multiple keys, and verifed that MSB domain of failover reply is ALWAYS different from the one in normal reply when using DeterministicOrder policy. Code changes on this area will be submitted in another diff.

Reviewed By: kkondaka

Differential Revision: D23443619

fbshipit-source-id: 1d66407187e63d640b85390fb66e3790dd4fa38e

facebook/mcrouter,facebook/mcrouter,facebook/mcrouter,facebook/mcrouter