We present an approach to detecting and localizing defects in random color textures which requires only a few defect free samples for unsupervized training. It is assumed that each image is generated by a superposition of various-size image patches with added variations at each pixel position. These image patches and their corresponding variances are referred to here as textural exemplars or texems. Mixture models are applied to obtain the texems using multiscale analysis to reduce the computational costs. Novelty detection on color texture surfaces is performed by examining the same-source similarity based on the data likelihood in multiscale, followed by logical processes to combine the defect candidates to localize defects. The proposed method is compared against a Gabor filter bank based novelty detection method. Also, we compare different texem generalization schemes for defect detection in terms of accuracy and efficiency.