Regression based machine translation (RegMT) model provides a learning framework for machine translation, separating learning models for training, training instance selection, feature representation, and decoding. Transductive learning approach employs training instance selection algorithms that not only make RegMT computationally more scalable but also improve the performance of standard statistical machine translation (SMT) systems. Sparse regression models for SMT are introduced and the obtained results demonstrate that sparse regression models perform better than other learning models in predicting target features, estimating word alignments, creating phrase tables, and generating translation outputs. We develop good evaluation techniques for measuring the performance of the RegMT model and the quality of the translations. We demonstrate that sparse L1 regularized regression performs better than L2 regularized regression in the German-English translation task and in the Spanish-English translation task when using small sized training sets. Graph based decoding can provide an alternative to phrase-based decoding in translation domains having low vocabulary.