In this report, at first, the effect of the roughness of the testing surface of specimen on the Rockwell hardness was investigated, and then the effect of the roughness of the reverse surface was investigated, after the experiment on many specimens which were of different hardness or material. As a result, these effects on the Rockwell hardness number have been made clear, and it has been revealed that the roughness of the testing surface of specimen affects the hardness number, but on the other hand the roughness of the reverse surface does not influence the hardness number. The critical roughness curve which shows the relation between the surface condition ranging from the maximum (critical) roughness for the testing surface to free-from-the-effect of the roughness and the hardness of specimen was given. And after discussion of this result, it was presumed that the so-called "effect of the roughness of the reverse surface of specimen" which is told to be existed in industrial measuring is not due to the roughness itself but to the more direct causes related to the condition of the reverse surface. And also, the effects on three hardnesses, the Shore, Vickers and Rockwell, which are most widely used nowadays, were compared in this report.