A surprising feature of animal locomotion is that organisms typically produce substantial forces in directions other than what is necessary to move the animal through its environment, such as perpendicular to, or counter to, the direction of travel. The effect of these forces has been difficult to observe because they are often mutually opposing and therefore cancel out. Indeed, it is likely that these forces do not contribute directly to movement but may serve an equally important role: to simplify and enhance the control of locomotion. To test this hypothesis, we examined a well-suited model system, the glass knifefish Eigenmannia virescens, which produces mutually opposing forces during a hovering behavior that is analogous to a hummingbird feeding from a moving flower. Our results and analyses, which include kinematic data from the fish, a mathematical model of its swimming dynamics, and experiments with a biomimetic robot, demonstrate that the production and differential control of mutually opposing forces is a strategy that generates passive stabilization while simultaneously enhancing maneuverability. Mutually opposing forces during locomotion are widespread across animal taxa, and these results indicate that such forces can eliminate the tradeoff between stability and maneuverability, thereby simplifying neural control.