Raman spectroscopy is a powerful diagnostic tool in biomedical science, whereby different disease groups can be classified based on subtle differences in the cell or tissue spectra. A key component in the classification of Raman spectra is the application of multi-variate statistical models. However, Raman scattering is a weak process, resulting in a trade-off between acquisition times and signal-to-noise ratios, which has limited its more widespread adoption as a clinical tool. Typically denoising is applied to the Raman spectrum from a biological sample to improve the signal-to noise ratio before application of statistical modeling. A popular method for performing this is Savitsky–Golay filtering. Such an algorithm is difficult to tailor so that it can strike a balance between denoising and excessive smoothing of spectral peaks, the characteristics of which are critically important for classification purposes. In this paper, we demonstrate how Convolutional Neural Networks may be enhanced with a non-standard loss function in order to improve the overall signal to-noise ratio of spectra while limiting corruption of the spectral peaks. Simulated Raman spectra and experimental data are used to train and evaluate the performance of the algorithm in terms of the signal to noise ratio and peak fidelity. The proposed method is demonstrated to effectively smooth noise while preserving spectral features in low intensity spectra which is advantageous when compared with Savitzky–Golay filtering. For low intensity spectra the proposed algorithm was shown to improve the signal to noise ratios by up to 100% in terms of both local and overall signal to noise ratios, indicating that this method would be most suitable for low light or high throughput applications.