In this paper, a new heuristic algorithm for the sparse adaptive equalization problem, termed as stochastic gradient pursuit, is proposed. A decision-feedback equalization structure is used in order to effectively mitigate the effect of long mul- tipath channels. Diverging from the commonly used approach of sparse channel identification, we exploit the sparsity of the inverse problem under the compressive sensing perspective. Also, an extension to the case where the sparsity order parameter is unknown, is developed. Simulation results verify that the pro- posed schemes exhibit faster convergence and improved tracking capabilities compared to conventional and other sparse aware equalization schemes, offering at the same time a reduced compu- tational complexity.