Thermal effect on machine tools is a well-recognized problem in an environment of increasing demand for product quality. The performance of a thermal error compensation system typically depends on the accuracy and robustness of the thermal error model. This work presents a novel thermal error model utilizing two mathematic schemes: the grey system theory and the adaptive network-based fuzzy inference system (ANFIS). First, the measured temperature and deformation results are analyzed via the grey system theory to obtain the influence ranking of temperature ascent on thermal drift of spindle. Then, using the highly ranked temperature ascents as inputs for the ANFIS and training these data by the hybrid learning rule, a thermal compensation model is constructed. The grey system theory effectively reduces the number of temperature sensors needed on a machine structure for prediction, and the ANFIS has the advantages of good accuracy and robustness. For testing the performance of proposed ANFIS model, a real-cutting operation test was conducted. Comparison results demonstrate that the modeling schemes of the ANFIS coupled with the grey system theory has good predictive ability.