We introduce a new class of qubit codes that we call Evenbly codes, building on a previous proposal of hyperinvariant tensor networks. Its tensor network description consists of local, non-perfect tensors describing CSS codes interspersed with Hadamard gates, placed on a hyperbolic geometry with even , yielding an infinitely large class of subsystem codes. We construct an example for a manifold and describe strategies of logical gauge fixing that lead to different rates and distances , which we calculate analytically, finding distances which range from to in the ungauged case. Investigating threshold performance under erasure, depolarizing, and pure Pauli noise channels, we find that the code exhibits a depolarizing noise threshold of about in the code-capacity model and for pure Pauli and erasure channels under suitable gauges. We also test a constant-rate version with , finding excellent error resilience (about ) under the erasure channel. Recovery rates for these and other settings are studied both under an optimal decoder as well as a more efficient but non-optimal greedy decoder. We also consider generalizations beyond the CSS tensor construction, compute error rates and thresholds for other hyperbolic geometries, and discuss the relationship to holographic bulk/boundary dualities. Our work indicates that Evenbly codes may show promise for practical quantum computing applications.