The Hidden Markov Model (HMM) has been used for speech recognition, but there is not effective method for deciding the HMM structure to raise the recognition rate. This paper proposes a new method to search the optimum structure of HMM by genetic algorithms (GA) automatically. In order to carry out the searching operations by GA, the genotype is expressed by the HMM matrix, and the methods for the crossover, the mutation are also discussed. The concept of time normalization is defined for preventing the occurrence of lethal genotypes, and the concept of time slot is used to decide the initial parameters for starting the GA operations. The results of experiments show that the recognition rate is greatly improved from 65% to 91.5% when inputting speech data for recognition to the optimum HMM searched by GA.