Check out this work from Rich Caruana & collaborators on model compression: http://www.cs.cornell.edu/~caruana/compression.kdd06.pdf

which was a precursor to the model distallation work from Geoff Hinton: https://arxiv.org/abs/1503.02531