This paper proposes a similarity measurement technique for retrieving similar driving scenes, using driving behavior signals and features of the driving environment. A previous work proposed a similarity-based retrieval system for finding driving data, which retrieved driving scenes by measuring similarity between scenes using driving behavior signals, such as steering angle and vehicle velocity. However, driving scenes can also be characterized by the surrounding driving environment. In this study, we assume that driving scenes consist of three major entities: the driver, the driver's vehicle, and the driving environment. We measure the distance between driving scenes using road features as well as the position and motion of surrounding vehicles (i.e., the surrounding driving environment), in addition to driving behavior signals obtained from the driver and the driver's vehicle. We then conduct a driving scene retrieval experiment to evaluate our similarity measurement method, using driving data collected on an expressway. Experimental results show that the additional use of environmental information significantly improves the precision of retrieval of scenes of driving events compared with a conventional method. According to our results, we also find that different people focus on different elements when comparing driving scenes, which may indicate that different drivers focus on different things when driving.