It has been demonstrated that there are many similarities between Pavlovian conditioning in nonhuman animals and causal judgment by humans, such as conditioned inhibition, overshadowing, and blocking. However, there was a notable difference in empirical studies between animal Pavlovian conditioning and human causal judgment: lack of retrospective inference (i.e., backward blocking) in animals. Although human participants showed symmetrical results for forward and backward blocking procedure in causal judgment, researchers failed to obtain backward blocking in animal conditioning. In the case of forward blocking, a cue is first paired with an unconditioned stimulus (US) (e.g., A+), and the first cue is then presented together with a target cue and the US (e.g., AX+). In the case of backward blocking, the compound cue is learned first (AX+), and then the competing cue alone is paired with the US (A+). In subsequent tests, human participants inferred in both cases that X is not a cause of the outcome, whereas the response of animals to X alone was “blocked” only in the forward blocking procedure but not in the backward one. In this article, we review the existence studies on retrospective inference in humans and animals including our ongoing primate study and explore a possible role of retrieval deficits in memory for retrospective inference in animals.