Informatics interventions generally take place in rapidly changing settings where many variables are outside the control of the evaluator. Assessment must be timely so that feedback can instigate modification of the intervention. Adapting a methodology from international health and epidemiology, we have developed and refined a Rapid Assessment Process (RAP) for informatics while conducting a study of clinical decision support (CDS) in community hospitals. Using RAP, we have not only been able to provide implementers with actionable feedback, but we have also discovered that users and informaticians conceptualize CDS in vastly different ways. Further understanding of this difference will be needed if we are to improve CDS acceptance by users.