We propose a method for detecting and stabilizing shaky frames of omnidirectional videos captured by omnidirectional camera systems mounted on top a car. We introduce a technique to detect frames where the video has unwanted shakes and jitters. This enables us to avoid stabilizing all frames of the video that can be very time-consuming in long videos. The main difference between two consecutive frames, specially when the video is shaky, is caused by camera rotation. We decouple rotation and translation and compensate only for rotation. We use a fast and robust method to estimate the camera rotation between two consecutive spherical frames. Our method uses only two pairs of corresponding points as seeds in RANSAC that makes the computation faster than other methods. Our experiments show that our results are comparable to those methods that compensate for both rotation and translation but our method is faster and more robust.