The REBCO-coated conductor is one of the promising technologies for Maglev application in terms of its high operating temperature. Since REBCO coils are directly cooled using cryocoolers, liquid helium is unnecessary. However, the coil structure must be optimized for efficient conduction cooling. We propose a novel REBCO coil structure that has high thermal conductivity and eliminates the delamination due to epoxy impregnation. A REBCO coil is wound with the coated conductor and a polytetrafluoroethylene (PTFE) tape for insulation. The coil winding is then impregnated with epoxy resin. At the same time as the impregnation, heat transfer members (e.g., copper plates) are bonded to the top and bottom surfaces of the coil winding. This paper contains three topics. First one is the thermal resistance measurements of the impregnation materials. It was observed that thermal stress increases the thermal resistances of paraffin wax and cyanoacrylate resin. It is believed that impregnation materials with high adhesiveness are appropriate for cryocooler-cooled coils. Second is demonstration of the novel coil structure. The third topic is the experimental production of a REBCO coil that has a congruent racetrack shape like that of the Maglev onboard magnet. It was confirmed that the novel coil structure eliminates degradation of the full-scale racetrack REBCO coil.