Low signal to noise ratio and low contrast are the major limitations for segmentation, image analysis and interpretation of medical thermal images. In this work, an attempt has been made to improve and preserve the inter-regions edges by effectively removing the noise without blurring and hence, to extract the breast tissues from infrared images using level sets based on improved edge information. Gaussian filter is a linear, homogenous diffusion process that performs smoothing operation at each location that blurs the edge information resulting in difficulty of detection and localization of edges. To avoid smoothing across boundaries, an anisotropic diffusion based smoothing filter is used. This enables smoothing within the region by preserving sharp region boundaries. The performance improvement of the diffusion filter is verified and validated by extracting the breast tissues. The segmentation of regions of interests (ROIs) is performed by evolving the initial level set function based on this improved edge information. The extracted ROIs are compared with the corresponding four sets of ground truth images. The results show a good agreement of the segmented ROIs with ground truths. Further, the performance of the segmentation method is analyzed across inter person variations by calculating quantitative measures based on overlap and the statistics of regional similarities. It is observed that the segmentation method could able delineate the accurate regions of interest irrespective to the limitations of thermal images such as lack of clear edges. Average accuracy of 98% of regional similarity is obtained between segmented ROIs and ground truth images. Therefore, the enhanced edge detail seems to be useful to improve the performance of segmentation algorithm which could be used during breast cancer screening for early detection of tumor. © 2014 Elsevier Ltd.