Is there a way to detect search engine bias? Search engines are increasingly being relied on as information gatekeepers, yet the criteria used by search engines to rank results is opaque to users. How can users be sure their results aren't biased or tampered with in some way to benefit some interest at the expense of search result quality?

Governments routinely demand that search providers remove or lower the ranking of websites deemed politically undesirable. Businesses may pay providers to boost certain results over others to increase their revenues. Firewalls may meddle with results before they're transmitted back to users.

Even seemingly innocuous changes to ranking algorithms that might not on the surface appear to be biased, could actually be deviously designed to harm websites that share some common attribute (unrelated to actual quality).

Is it possible to detect search engine bias, by say monitoring results over a period of time and evaluating whether some "hidden variable" (perhaps a political affiliation) is a driving factor in the change in website rankings?

A sneaky provider may gradually over time lower the ranking of targeted websites (and perhaps random websites as well to distract users). What are the limits on how much bias a provider can introduce without detection? Or is it possible to always conceal such interference by deviously selecting weighted ranking criteria that incidentally produce the intended result (by way of "data snooping").

Does any of this change if the ranking criteria is made public? Do we need to open-source the criteria search engines use?

This reminds me of the result that detecting whether or not a complex financial instrument such as a CDO has been tampered with by the seller is equivalent to solving the densest-subgraph problem:

http://www.cs.princeton.edu/~rongge/derivative.pdf

Thanks!
This is obviously a very open ended question, but so as to stay on topic, here is one CS theory approach to the idea of "fairness" and how to enforce it.

"Fairness Through Awareness" Dwork, Hardt, Pitassi, Reingold, Zemel
http://arxiv.org/abs/1104.3913