We present a technique of using Deep Neural Networks (DNNs) for offline character recognition of Telugu characters. We construct DNNs by stacking Auto-encoders that are trained in a greedy layer-wise fashion in an unsupervised manner. We then perform supervised fine-tuning to train the entire network. We provide results on Consonant and Vowel Modifier Datasets using two and three hidden layer DNNs. We also construct an ensemble classifier to increase the classification performance further. We observe 94.25% accuracy for the two hidden layer network on Consonant data and 94.1% on Vowel Modifier Dataset which increases to 95.4% for Consonant and 94.8% for Vowel Modifier Dataset after combining classifiers to form an ensemble classifier of 4 different two hidden layer networks. © 2012 Springer-Verlag.