Print out some more information in bisect stats, including times for each step, std error difference in means, and a little explanation about how to interpret the confidence value.
Print out some more information in bisect stats, including times for each step, std error difference in means, and a little explanation about how to interpret the confidence value.

Sample output:

Tested commits:
         Depot                         Commit SHA                     Mean       Std. Error       State             Step Time        
        chromium        243576fda09119fb91c151395d69c713f0ede6c3     250.28        +-0.12          Bad           8 mins, 38 secs     
        chromium        824d389d6bd478947f86213604375b257f4b6bab     251.05        +-0.14          Bad            3 mins, 3 secs     
        chromium        35986df002b4c8590bb7eb1b403a89ba2bf130e2     249.16        +-0.36          Bad            2 mins, 3 secs     
         webkit         c666302fccb3438c605a4cef94bbafdc8fedf134     246.05        +-0.10          Bad           1 mins, 59 secs     

         webkit         36f5e1addaac952cfaf729f6219ce51b1818a132     246.09        +-0.17      Suspected CL      1 mins, 46 secs     

         webkit         10d43ee4f9f6051a7c45fc40a93ba0ba8fa6e535     242.56        +-0.15          Good          1 mins, 49 secs     
         webkit         df208043ef9bfba8c111317dd37bbe592ca0bc36     242.50        +-0.13          Good          2 mins, 48 secs     
         webkit         7f545c03bb3b5110e8898804622d219384e29a12     244.85        +-0.15          Good          4 mins, 12 secs     
         webkit         f036a9311dab3622b88966c678f54cf405355796     242.45        +-0.16          Good          3 mins, 49 secs     
        chromium        d6366c211a8eb5b6f369b2fd3038b6b6bc3d5e7e     242.52        +-0.16          Good           3 mins, 3 secs     
        chromium        32f91e45045bcebb026178847d44ed6f956db7ba     241.30        +-0.13          Good          4 mins, 20 secs     

Approximate size of regression: 2.45%,  0.14% std. err
Confidence in Bisection Results: 25%

Experimental - If confidence is less than 100%, there are most likely some other strong candidates for this regression. You can try increasing the repeat_count, or looking for a sub-metric that shows the regression more clearly.


BUG=298279
NOTRY=true

Review URL: https://codereview.chromium.org/25812002

git-svn-id: de016e52bd170d2d4f2344f9bf92d50478b649e0@227026 0039d316-1c4b-4281-b951-d872f2087c98

Pluto-tv/chromium-crosswalk,ltilve/chromium,krieger-od/nwjs_chromium.src,PeterWangIntel/chromium-crosswalk,axinging/chromium-crosswalk,jaruba/chromium.src,Pluto-tv/chromium-crosswalk,Chilledheart/chromium,dednal/chromium.src,jaruba/chromium.src,hgl888/chromium-crosswalk,Chilledheart/chromium,PeterWangIntel/chromium-crosswalk,littlstar/chromium.src,jaruba/chromium.src,M4sse/chromium.src,axinging/chromium-crosswalk,PeterWangIntel/chromium-crosswalk,anirudhSK/chromium,anirudhSK/chromium,littlstar/chromium.src,ltilve/chromium,M4sse/chromium.src,TheTypoMaster/chromium-crosswalk,dushu1203/chromium.src,dushu1203/chromium.src,Chilledheart/chromium,patrickm/chromium.src,bright-sparks/chromium-spacewalk,hgl888/chromium-crosswalk,Pluto-tv/chromium-crosswalk,krieger-od/nwjs_chromium.src,hgl888/chromium-crosswalk-efl,axinging/chromium-crosswalk,Jonekee/chromium.src,chuan9/chromium-crosswalk,Fireblend/chromium-crosswalk,PeterWangIntel/chromium-crosswalk,PeterWangIntel/chromium-crosswalk,dednal/chromium.src,M4sse/chromium.src,jaruba/chromium.src,ltilve/chromium,anirudhSK/chromium,Just-D/chromium-1,Jonekee/chromium.src,dushu1203/chromium.src,axinging/chromium-crosswalk,patrickm/chromium.src,crosswalk-project/chromium-crosswalk-efl,crosswalk-project/chromium-crosswalk-efl,Chilledheart/chromium,fujunwei/chromium-crosswalk,Fireblend/chromium-crosswalk,ChromiumWebApps/chromium,dushu1203/chromium.src,Fireblend/chromium-crosswalk,Fireblend/chromium-crosswalk,Pluto-tv/chromium-crosswalk,chuan9/chromium-crosswalk,crosswalk-project/chromium-crosswalk-efl,chuan9/chromium-crosswalk,bright-sparks/chromium-spacewalk,dushu1203/chromium.src,hgl888/chromium-crosswalk,dednal/chromium.src,krieger-od/nwjs_chromium.src,mohamed--abdel-maksoud/chromium.src,dednal/chromium.src,Just-D/chromium-1,Just-D/chromium-1,dushu1203/chromium.src,crosswalk-project/chromium-crosswalk-efl,Just-D/chromium-1,PeterWangIntel/chromium-crosswalk,dednal/chromium.src,ondra-novak/chromium.src,TheTypoMaster/chromium-crosswalk,hgl888/chromium-crosswalk,axinging/chromium-crosswalk,markYoungH/chromium.src,patrickm/chromium.src,ltilve/chromium,mohamed--abdel-maksoud/chromium.src,ChromiumWebApps/chromium,littlstar/chromium.src,mohamed--abdel-maksoud/chromium.src,M4sse/chromium.src,hgl888/chromium-crosswalk-efl,PeterWangIntel/chromium-crosswalk,krieger-od/nwjs_chromium.src,chuan9/chromium-crosswalk,M4sse/chromium.src,markYoungH/chromium.src,anirudhSK/chromium,markYoungH/chromium.src,krieger-od/nwjs_chromium.src,hgl888/chromium-crosswalk-efl,ondra-novak/chromium.src,TheTypoMaster/chromium-crosswalk,ltilve/chromium,Pluto-tv/chromium-crosswalk,ltilve/chromium,bright-sparks/chromium-spacewalk,crosswalk-project/chromium-crosswalk-efl,Jonekee/chromium.src,crosswalk-project/chromium-crosswalk-efl,hgl888/chromium-crosswalk-efl,markYoungH/chromium.src,ondra-novak/chromium.src,littlstar/chromium.src,axinging/chromium-crosswalk,hgl888/chromium-crosswalk-efl,anirudhSK/chromium,jaruba/chromium.src,jaruba/chromium.src,krieger-od/nwjs_chromium.src,axinging/chromium-crosswalk,fujunwei/chromium-crosswalk,ondra-novak/chromium.src,Chilledheart/chromium,crosswalk-project/chromium-crosswalk-efl,mohamed--abdel-maksoud/chromium.src,hgl888/chromium-crosswalk,krieger-od/nwjs_chromium.src,crosswalk-project/chromium-crosswalk-efl,chuan9/chromium-crosswalk,markYoungH/chromium.src,anirudhSK/chromium,ondra-novak/chromium.src,markYoungH/chromium.src,anirudhSK/chromium,Just-D/chromium-1,ChromiumWebApps/chromium,Fireblend/chromium-crosswalk,Fireblend/chromium-crosswalk,ondra-novak/chromium.src,ltilve/chromium,littlstar/chromium.src,M4sse/chromium.src,fujunwei/chromium-crosswalk,krieger-od/nwjs_chromium.src,TheTypoMaster/chromium-crosswalk,mohamed--abdel-maksoud/chromium.src,ondra-novak/chromium.src,hgl888/chromium-crosswalk,Jonekee/chromium.src,mohamed--abdel-maksoud/chromium.src,jaruba/chromium.src,Jonekee/chromium.src,bright-sparks/chromium-spacewalk,axinging/chromium-crosswalk,anirudhSK/chromium,dednal/chromium.src,anirudhSK/chromium,fujunwei/chromium-crosswalk,ChromiumWebApps/chromium,Pluto-tv/chromium-crosswalk,mohamed--abdel-maksoud/chromium.src,littlstar/chromium.src,patrickm/chromium.src,Jonekee/chromium.src,krieger-od/nwjs_chromium.src,chuan9/chromium-crosswalk,mohamed--abdel-maksoud/chromium.src,Pluto-tv/chromium-crosswalk,anirudhSK/chromium,ChromiumWebApps/chromium,PeterWangIntel/chromium-crosswalk,Jonekee/chromium.src,M4sse/chromium.src,chuan9/chromium-crosswalk,bright-sparks/chromium-spacewalk,dushu1203/chromium.src,Fireblend/chromium-crosswalk,markYoungH/chromium.src,TheTypoMaster/chromium-crosswalk,M4sse/chromium.src,chuan9/chromium-crosswalk,krieger-od/nwjs_chromium.src,Jonekee/chromium.src,axinging/chromium-crosswalk,anirudhSK/chromium,mohamed--abdel-maksoud/chromium.src,axinging/chromium-crosswalk,TheTypoMaster/chromium-crosswalk,dednal/chromium.src,hgl888/chromium-crosswalk,bright-sparks/chromium-spacewalk,TheTypoMaster/chromium-crosswalk,M4sse/chromium.src,ChromiumWebApps/chromium,crosswalk-project/chromium-crosswalk-efl,markYoungH/chromium.src,chuan9/chromium-crosswalk,PeterWangIntel/chromium-crosswalk,dednal/chromium.src,ChromiumWebApps/chromium,dednal/chromium.src,dednal/chromium.src,Chilledheart/chromium,Just-D/chromium-1,anirudhSK/chromium,Fireblend/chromium-crosswalk,fujunwei/chromium-crosswalk,markYoungH/chromium.src,hgl888/chromium-crosswalk-efl,Jonekee/chromium.src,ondra-novak/chromium.src,M4sse/chromium.src,hgl888/chromium-crosswalk,ChromiumWebApps/chromium,Chilledheart/chromium,bright-sparks/chromium-spacewalk,mohamed--abdel-maksoud/chromium.src,hgl888/chromium-crosswalk-efl,dushu1203/chromium.src,Just-D/chromium-1,patrickm/chromium.src,jaruba/chromium.src,hgl888/chromium-crosswalk,patrickm/chromium.src,axinging/chromium-crosswalk,mohamed--abdel-maksoud/chromium.src,Pluto-tv/chromium-crosswalk,dushu1203/chromium.src,krieger-od/nwjs_chromium.src,fujunwei/chromium-crosswalk,dushu1203/chromium.src,markYoungH/chromium.src,dednal/chromium.src,ChromiumWebApps/chromium,fujunwei/chromium-crosswalk,hgl888/chromium-crosswalk-efl,Jonekee/chromium.src,M4sse/chromium.src,markYoungH/chromium.src,patrickm/chromium.src,Just-D/chromium-1,Fireblend/chromium-crosswalk,Chilledheart/chromium,patrickm/chromium.src,jaruba/chromium.src,bright-sparks/chromium-spacewalk,fujunwei/chromium-crosswalk,littlstar/chromium.src,bright-sparks/chromium-spacewalk,fujunwei/chromium-crosswalk,ltilve/chromium,Just-D/chromium-1,ChromiumWebApps/chromium,Jonekee/chromium.src,hgl888/chromium-crosswalk-efl,TheTypoMaster/chromium-crosswalk,littlstar/chromium.src,patrickm/chromium.src,Pluto-tv/chromium-crosswalk,jaruba/chromium.src,ChromiumWebApps/chromium,ondra-novak/chromium.src,hgl888/chromium-crosswalk-efl,Chilledheart/chromium,dushu1203/chromium.src,jaruba/chromium.src,ChromiumWebApps/chromium,TheTypoMaster/chromium-crosswalk,ltilve/chromium