In real environments, the detection and motion of an interesting moving object against a complex background in computer vision are very important. However, many undesirable factors that can prevent stable detection arise in a scene. The factors are, for example, occlusion, lighting change and jittery background. In addition to these, an infinite number of randomly moving particle-like patterns can make moving objects more difficult to detect. These particles have ambiguous edges and no definite shape. In a more complicated scene, we assume that such particles cannot be smoothed out by a simple preprocessing such as a low-pass filtering. Thus, they can distribute optical flow of moving objects in time and space. To deal with these issues for matching between frames, it is first assumed that particle-like patterns and moving objects have properties of fluidity and rigidity, respectively. The image brightness change and motion smoothness between frames can constrain the estimated optical flow of a moving object. However, a local large brightness change caused by the above factors violates such constraints. Thus, on the basis of a statistical approach, a robust optical flow estimation method has been developed by adding a locally parallel flow contraint. Estimated flow is used to count the number of multiple moving objects with a clustering method, which is applied starting with a large number of centorids. Centorids are iteratively merged within a distance until converged. Experimental results show that our method outperforms previous methods, thus validating our proposed method.