We present arcsecond resolution Chandra X-ray and ground-based optical Hα imaging of a sample of 10 edge-on star-forming disk galaxies (seven starburst and three "normal" spiral galaxies), a sample that covers the full range of star formation intensity found in disk galaxies. The X-ray observations make use of the unprecedented spatial resolution of the Chandra X-ray observatory to more robustly than before remove X-ray emission from point sources and hence obtain the X-ray properties of the diffuse thermal emission alone. We have combined the X-ray observations with existing, comparable-resolution, ground-based Hα and R-band imaging and present a mini-atlas of images on a common spatial and surface brightness scale to aid cross-comparison. In general, the morphology of the extraplanar diffuse X-ray emission is very similar to the extraplanar Hα filaments and arcs, on both small and large scales (scales of tens of parsecs and kiloparsecs, respectively). The most spectacular cases of this are found in NGC 1482 (for which we provide the first published X-ray observation) and NGC 3079. We provide a variety of quantitative measures of how the spectral hardness and surface brightness of the diffuse X-ray emission varies with increasing height z above the plane of each galaxy. Of the eight galaxies in which diffuse X-ray emitting halos are found (the starbursts and the normal spiral NGC 891), significant spatial variation in the spectral properties of the extraplanar emission (|z| ≥ 2 kpc) is only found in two cases: NGC 3628 and NGC 4631. In general, the vertical distribution of the halo-region X-ray surface brightness is best described as an exponential, with the observed scale heights of the sample galaxies lying in the range Heff ∼ 2-4 kpc. The presence of extraplanar X-ray emission is always associated with the presence of extraplanar optical line emission of similar vertical extent. No X-ray emission was detected from the halos of the two low-mass normal spiral galaxies NGC 6503 and NGC 4244. Active galactic nuclei, where present, appear to play no role in powering or shaping the outflows from the starburst galaxies in this sample. The Chandra ACIS X-ray spectra of extraplanar emission from all these galaxies can be fitted with a common two-temperature spectral model with an enhanced α-to-iron element ratio. This is consistent with the origin of the X-ray emitting gas being either metal-enriched merged SN ejecta or shock-heated ambient halo or disk material with moderate levels of metal depletion onto dust. Our favored model is that SN feedback in the disks of star-forming galaxies create, via blow-out and venting of hot gas from the disk, tenuous exponential atmospheres of density scale height Hg ∼ 4-8 kpc. The soft thermal X-ray emission observed in the halos of the starburst galaxies is either this preexisting halo medium, which has been swept up and shock-heated by the starburst-driven wind, or wind material compressed near the walls of the outflow by reverse shocks within the wind. In either case, the X-ray emission provides us with a powerful probe of the properties of gaseous halos around star-forming disk galaxies.