In developing countries, it is difficult to grasp the living condition of people because there is no detailed data on residential status. Especially, it is difficult to grasp the condition of poor people because some of them live in illegally occupied areas. In this research, therefore, the deep learning model to grasp the residence of poor people at the building level from satellite image and household survey data was developed. This model can classify buildings into three levels: poor, middle and rich. Three methods for creating labeled training data were considered and the influence of building area, land use and elevation data on estimation accuracy was also considered. The accuracy of the method with the highest estimation accuracy was 81.8%. The result can be visualized by using GIS and it helps people to understand where many poor or rich people live.
