The influence of coarse aggregate on the fracture toughness of high strength concrete was investigated. Fracture parameters for concrete were analyzed based on the poly-linear approximation analysis of the tension softening diagram with the data of a load-load point displacement curve. Three-point bending tests on the pre-notched aggregate beam specimens and the pre-notched concrete beam specimens were performed. Toughness modulus (T.M.) was considered to be an appropriate parameter to evaluate the effect of microcracking on the toughness of various concretes. The fracture energy of aggregate was relative to the tensile strength of aggregate. In concrete with the high strength mortar matrix, failure was initiated by aggregate cracking and the fracture energy of concrete was strongly affected by the fracture energy of aggregate. The crack resistance of high strength concrete depended on the fracture properties of aggregates and significantly varied from the types of aggregates.