Deep learning becomes promising techniques in many applications such as machine translation and drug discovery. Recently, it is actively studied to use deep learning for discovering new material such as chemicals and crystals. These works contribute to efficient and effective material discovery by reducing researcher-dependent tasks and time costs. In this article, we review existing deep learning techniques on crystal discovery. We explain recent studies categorized into: classifier/regression, generation, and crystal growth. Also, we describe fundamental knowledge of deep learning for beginners: general deep learning models, data representations of crystals, and difference between drug-like materials and crystals. Furthermore, we explain databases and tools that are useful for applying deep learning on material discovery. Finally, we discuss some challenges for crystal discovery that we need to tackle in the future.