Computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) all have the potential to directly visualize local and distant relapse of colorectal cancer (CRC). Nevertheless, the role of diagnostic imaging for routine follow-up of CRC patients remains controversial. Although MRI and PET have advantages over CT in the detection of local recurrence, until now only a few surveillance programs recommend the use of annual CT for routine follow-up. The objective of this review is to elucidate the current status of diagnostic imaging for the detection of recurrent rectal cancer based on the recent literature and our own experience. Furthermore, an insight into contemporary surveillance programs and an outlook concerning a novel technical approach to moving-table MRI at 1.5 Tesla for staging purposes are given.