One of the effective means as a method of supporting a worker is the remote visual information sharing method. In this work we focus on sharing of visual information between the expert person who is an expert of a task in a remote site and the assistant person who assists the task execution. The visual information of the remote site is captured by a head-mounted camera on the assistant person and is shared with the expert person. Normally the shared visual information is tightly linked between the expert person and the assistant person and the link can be the constraint between them and prevents efficient work execution. This paper explains the outline of the asynchronous visual information sharing system proposed in order to solve those problems and the camera motion compensation method for the proposed system. The effectiveness of the proposed compensation method has been confirmed by the conducted experiments.