To enhance automobile fuel efficiency, it is crucial to develop lightweight and compact components with improved strength. This research specifically addresses the increasing demand for higher fatigue strength in components that undergo surface hardening processes. The study focuses on the impact of refining the prior austenite (prior γ) grain size on the fatigue strength of low carbon steel sheets that have been processed through vacuum carburizing followed by quenching. To investigate this relationship, specimens were prepared under various quenching temperatures, and their plane bending fatigue strength was evaluated. The results indicated a clear trend: finer prior γ grain sizes generally correlate with improved fatigue strength. However, it was observed that when the maximum grain size fell below 47 μm, the enhancement in fatigue strength became minimal. Further analysis of the fracture surfaces revealed a significant shift in the initiation point of fatigue failure, transitioning from intergranular to transgranular fracture. This shift is crucial as it contributes to the overall fatigue behavior of the materials. The findings of this study underscore the importance of achieving a sufficiently fine prior γ grain size in vacuum carburized and quenched materials. Such refinement is essential for attaining stable and high fatigue strength, as it facilitates transgranular fracture initiation, ultimately leading to enhanced durability and reliability of automotive components.