This paper presents a search for high redshift galaxies from the Euclid Early Release Observations program "Magnifying Lens." The 1.5 deg area covered by the twin Abell lensing cluster fields is comparable in size to the few other deep near-infrared surveys such as COSMOS, and so provides an opportunity to significantly increase known samples of rare UV-bright galaxies at (). Beyond their still uncertain role in reionisation, these UV-bright galaxies are ideal laboratories from which to study galaxy formation and constrain the bright-end of the UV luminosity function. Of the 501994 sources detected from a combined , , and NISP detection image, 168 do not have any appreciable VIS/ flux. These objects span a range in spectral colours, separated into two classes: 139 extremely red sources; and 29 Lyman-break galaxy candidates. Best-fit redshifts and spectral templates suggest the former is composed of both dusty star-forming galaxies and quiescent systems. The latter is composed of more homogeneous Lyman break galaxies at . In both cases, contamination by L- and T-type dwarfs cannot be ruled out with Euclid images alone. Additional contamination from instrumental persistence is investigated using a novel time series analysis. This work lays the foundation for future searches within the Euclid Deep Fields, where thousands more Lyman break systems and extremely red sources will be identified.