We present spectroscopic confirmation of nine moderate-redshift galaxy groups and poor clusters selected from the ROSAT Deep Cluster Survey. The groups span the redshift range z ~ 0.23-0.59 and have between 4 and 20 confirmed members. The velocity dispersions of these groups range from ~125 to 650 km s-1. Similar to X-ray groups at low redshift, these systems contain a significant number of early-type galaxies. Therefore, the trend for X-ray-luminous groups to have high early-type fractions is already in place by at least z ~ 0.5. In four of the nine groups, the X-ray emission is clearly peaked on the most luminous early-type galaxy in the group. However, in several cases the central galaxy is composed of multiple luminous nuclei, suggesting that the brightest group galaxy may still be undergoing major mergers. In at least three (and possibly five) of the groups in our sample, a dominant early-type galaxy is not found at the center of the group potential. This suggests that many of our groups are not dynamically evolved despite their high X-ray luminosities. While similar systems have been identified at low redshift, the X-ray luminosities of the intermediate-redshift examples are 1-3 orders of magnitude higher than those of their low-redshift counterparts. We suggest that this may be evidence for group downsizing: while massive groups are still in the process of collapsing and virializing at intermediate redshifts, only low-mass groups are in the process of forming at the present day.