Using Indicator Functions as Transfer Functions for Neural Networks Does there exist any theory (other than Cybenko's proof of the Universal Approximation Theorem with sigmoids) advocating the use of indicator functions as transfer functions for machine learning with neural networks?

After having read matus's beautiful answer in this thread explaining (among other things) Cybenko's proof, I wonder: if it weakens the approximation to use sigmoid transfer functions instead of indicator functions, what are the theoretical reasons for not using indicator functions?

As suggested here, perhaps it's because indicator functions have negative implications for generalization. 

However, indicator functions are computationally far cheaper to implement than sigmoid functions, and also more closely resemble biological neural networks (ie the brain). Therefore, does there exist any other theory advocating the use of indicator functions as transfer functions for machine learning with neural networks?
Sigmoid functions have clear probabilistic interpretation, so one can derive optimal learning algorithms for them within the bayesian approach. Moreover, they are differentiable, so the gradient descent can be directly applicable.
However, in general, neural networks are "universal approximators" in almost the same sense as, for example, polynomials. They can capture a limited set of regularities. Different activation functions (like different basis functions in functional approximation) will capture different regularities and will have different biases in approximating other regularities.
Thus, what activation functions are better depends on what underlying regularities are presented in data. Indicator functions can surely be better for some tasks. However, no generally applicable (task-independent) theory advocating the use of indicator functions is possible.