The recovery of spectral reflectances from camera responses is usually composed of several distinct operations. We propose a new approach that connects edge-preserving denoising, deblurring and spectral reconstruction. Each of the steps is based on actual physical properties of the camera system that can be obtained using established methods, and the filter eliminates the need for manual sharpening of the images. Results on both real images and synthetic data show significant improvements over previous methods for spectral reconstruction.