Visual place recognition (VPR) in changing environments is an urgent challenge for long-term autonomous navigation. One recent ConvNet landmark-based approach exploits region landmarks coupled with ConvNet features to match images, and the approach has shown promising results under significant environmental and viewpoint changes. In this paper, we propose a robust ConvNet landmark-based system for VPR in changing outdoor roadway environments by extension of this approach from the following two aspects. First, our method utilizes more discriminative landmarks obtained by a novel refinement method called SemLandmarks, which leverages roadway scene semantic information to screen landmarks directly detected by an existing object proposal method. Second, our method improves the accuracy of image matching by introducing consistent spatial constraints based on the use of geometry-preserving landmark pairs. Experimental results demonstrate that our method significantly improves the state of the art in VPR in terms of recognition accuracy on three challenging benchmark data sets with various environmental and viewpoint changes.