We developed a nondestructive-inspection method for concrete structures by using frequency sweep sound wave. When irradiating sound waves with one frequency produced by a speaker to a concrete wall with a defect, the region between the defect and the concrete surface resonates. Then by irradiating frequency sweep sound wave we can detect defects with arbitrary natural frequency. The vibration of the surface is measured with laser Doppler vibrometer and internal defects are detected based on the vibration. We inspected concrete specimens containing an artificial defect by the present method. As a result, a disk-shaped defect of 200 mm in diameter and 100 mm in depth could be inspected successfully, where the frequency was continuously changed from 1 kHz to 10 kHz while 10 seconds. In addition, we examined the effect of laser scanning velocity to vibration measurement. The scanning velocity should be under 1 m/s when the measuring frequency is 5 kHz to recognize a dominant natural frequency.