We study theoretically the Berry curvature of the magnon induced by the hybridization with the acoustic phonons via the spin-orbit and dipolar interactions. We first discuss the magnon-phonon hybridization via the dipolar interaction, and show that the dispersions have gapless points in momentum space, some of which form a loop. Next, when both spin-orbit and dipolar interactions are considered, we show anisotropic texture of the Berry curvature and its divergence with and without gap closing. Realistic evaluation of the consequent anomalous velocity is given for yttrium iron garnet.