Neural Networks: Incorporate feature-specific knowledge Has any research been done about / are people interested in incorporating prior knowledge of good high-level features into a deep neural net?

I know this somewhat defeats the purpose of feature learning but consider the following applied scenario: image classification task, 20 classes,  'small' training set (10k), expert human knowledge of high-level features. 

Ideal would be to compensate for lack of training cases by incorporating knowledge of these features into the architecture (or via other means). How?﻿
This 2013 paper integrates deep learning models with structured hierarchical Bayesian models, for the net to learn novel concepts from very few training examples. It shows encouraging results.

Another approach is to import a general-purpose top-performing neural net, and replace the top fully connected layers with initialised ones (including an output layer that corresponds to the number of classes in your specific task). OverFeat is an example, for image classification - it was trained on ImageNet.