We present a novel method of machining optical fiber surfaces with a CO_2 laser for use in Fiber-based Fabry-Perot Cavities (FFPCs). Previously FFPCs were prone to large birefringence and limited to relatively short cavity lengths (≤ 200 μm). These characteristics hinder their use in some applications such as cavity quantum electrodynamics with trapped ions. We optimized the laser machining process to produce large, uniform surface structures. This enables the cavities to achieve high finesse even for long cavity lengths. By rotating the fibers around their axis during the laser machining process the asymmetry resulting from the laser’s transverse mode profile is eliminated. Consequently we are able to fabricate fiber mirrors with a high degree of rotational symmetry, leading to remarkably low birefringence. Through measurements of the cavity finesse over a range of cavity lengths and the polarization dependence of the cavity linewidth, we confirmed the quality of the produced fiber mirrors for use in low-birefringence FFPCs.