Accurate pointing is an obstacle to computer access for individuals who experience motor impairments. One of the main barriers to assisting individuals with pointing problems is a lack of frequent and low-cost assessment of pointing ability. We are working to build technology to automatically assess pointing problems during every day (or real-world) computer use. To this end, we have gathered and studied real-world pointing use from individuals with motor impairments and older adults. We have used this data to develop novel techniques to analyze pointing performance. In this article, we present learned statistical models that distinguish between pointing actions from diverse populations using real-world pointing samples. We describe how our models could be used to support individuals with different abilities sharing a computer, or one individual who experiences temporary pointing problems. Our investigation contributes to a better understanding of real-world pointing. We hope that these techniques will be used to develop systems that can automatically adapt to users’ current needs in real-world computing environments.