Carbonaceous hard coatings as DLC and CNx coatings keep attention for low friction and high wear resistance. Transformed layer which is graphite like layer induced by cyclic friction generated ultralow friction coefficient in dry condition. However the transformed layer in oil lubrication has not been observed yet because this transformed layer could be too thin to measure quantitatively. The spectroscopic reflectometry has been applied to measure the thickness of transformed layer of wear scar of DLC and CNx after friction under oil boundary lubrication and dry nitrogen blowing, respectively. After measurement of thickness and hardness of transformed layer from optical properties of transformed layer, we found that thickness and roughness of transformed layer affected friction so much. Therefore we appied solid thin film lubrication theory to explain the effect of transformed layer on friction for both of DLC and CNx under oil boundary lubrication and dry nitrogen blowing, respectively.