Image Comparison
In this section we explore more into how our filters can affect different types of images since it would be unfair to simply rank how efficient or capable a filter is based on a wide range of abstract images. We explore 3 types, first images that have to do with nature and scenery. Images dealing with details on people and capability of identifying features. And finally images dealing with actual texts that would read for example road or commercial signs ect.
Judging Quality
Towards the end we will be ranking these filters based on 3 things: PSNR,SSIM, and intuitively the human eye. A brief explanation of PSNR and SSIM is below:
PSNR
-
Compares the original image to the compressed or reconstructed image and computes the ratio of the peak signal power to the mean square error (MSE) between the two images.
-
Usually used for lossy compression algorithms like JPEG or MPEG.
SSIM
-
A measure of the perceived quality of an image, takes into account the structural similarity between the images comparing luminance, contrast, and structure of the original and distorted images
-
A value between 0 and 1, 1 being identical to the original structure.
​
Simply put PSNR is a measure of the distortion between the original and compressed/reconstructed image, while SSIM is a measure of the perceived similarity between the two images in terms of their structure, contrast, and luminance.
​
Filters on Nature & Scenery
For the first set of images of leaves we have Mean having SSIM:0.50 & PSNR: 38.40, built-in wiener 0.55 & 36.06, our wiener 0.50 & 41.23, DCT 0.48 & 37.69, Wavelet Threshold 0.58 & 41.19, finally the weighted median filter 0.65 & 22.08. Straight away even though the wiener filters may high decent SSIM scores they were clearly over smoothed, especially in the areas where the color shifts from the top half of the image being bright green to the bottom half which contains left brightness due to shade from the sun possibly. But in the end the wiener filters seemed to do the worst. The best filters not only from the score, from simply looking at the images weighted median does the best with the changing of shades while both the wavelet threshold and DCT come in at a close second. All of them doing a good job at preserving details and depth in the image. While the threshold and DCT do have better PSNR scores, the median filter was able to denoise a bit better coming closer to the original.
For our image with more expanding view of nature we have Mean having SSIM:0.60 & PSNR: 36.97, built-in wiener 0.65 & 36.51, our wiener 0.67 & 39.29, DCT 0.62 & 37.31, Wavelet Threshold 0.67 & 38.84, finally the weighted median filter 0.72 & 22.08. Going from the image we had before to this we can see that both wiener filters again have not really done a great job while all other filters seem to have preserved the details like branches and rocks on the shore equally. All PSNR and SSIM scores are very close while but this time it can visually be seen that the DCT and wavelet filter were able to actually remove more noise than median. This can be seen more in the sky area where a lot of the shades are brighter.
Filter on Humans
& their Features
For this image we have moved away from the bright colors and focus more on shades and detail in more expanded settings. We have Mean having SSIM:0.34 & PSNR: 37.28, built-in wiener 0.58 & 37.26, our wiener 0.57 & 40.66, DCT 0.44 & 37.66, Wavelet Threshold 0.47 & 41.21, finally the weighted median filter 0.39 & 20.77. Now visually looking at this it seems as thought the filters did not do well when it comes to depth in the image. The mean filter seems to come out looking the best and preserving details of the tree and grass field, but all the images after being denoised had an abundant amount of noise still left in the image. With all images having around the same SSIM score since they were all able to somewhat preserved structure compared to the original but even with the images with decent PSNR scores, visually it seems the filters did not really do a great job when it comes to the image detail.
Starting off from the Abraham Lincoln Image we have Mean having SSIM:0.33 & PSNR: 40.79, built-in wiener 0.70 & 40.45, our wiener 0.55 & 43.03, DCT 0.39 & 39.49, Wavelet Threshold 0.48 & 45.72, finally the weighted median filter 0.35 & 22.10. Now looking at these pictures both wiener filters seem to remove the most noise out of the picture while still preserving the shape and structure hence why they have a good SSIM score. But the mean & weighted filter carry over more of the detail and sharpness that would would prefer if you were trying to identify the face of Lincoln, but the downside would be that even after denoising there is some left over noise left. The DCT and threshold filter seem to have gone somewhat in between by preserving detail but not really doing a good job of removing enough noise from the image. So simply trying to identify a person from an image seems to have worked best with the mean and weighted filter.
Moving on we have an rgb image of a people close together we have Mean having SSIM:0.80 & PSNR: 37.50, built-in wiener 0.82 & 32.41, our wiener 0.82 & 35.08, DCT 0.82 & 37.86, Wavelet Threshold 0.83 & 42.72, finally the weighted median filter 0.9 & 22.86. For this set of images it seems as though all the SSIM scores are really high since compared to the Abraham Lincoln pictures the structural differences on the faces are much smaller than the Lincoln image. So when judging bigger blotches of color it is less likely to take into consideration changes in smaller detail. Wavelet threshold having the highest PSNR still is not impressive when you look at the actual image and realize how much noise is still on the image. For this batch of images it seems that both wiener filter come up on the top ranking with our own wiener filter removing the most noise while still being capable of preserving detail, even though they don't have the highest SSIM it is still a very good score. Here the weighted median filter does come in as a close second after wiener filters but you can tell it had over smoothed the images.
A side note about using SSIM and PSNR on images with many details like the images we have above. When ran through the filters SSIM fails to actual identify these changes in faces, of course it isn't meant to actually notify anyone that the faces have changed but because the algorithm is more based on structural similarity than PSNR the quality's of both of these images after being denoised are nearly identical. I would assume the error difference between them would be that colors and shading may be off causing some SSIM values to be higher or lower. But the PSNR does a way better job evaluating these pictures which would give the user a better heads up that they are different.
Image Readability and Text
For this section we use an image of a highway sign to try and gaugh how well these filter may be able to handle text in images and readability. For scores we have Mean having SSIM:0.61 & PSNR: 34.60, built-in wiener 0.56 & 31.36, our wiener 0.68 & 34.76, DCT 0.69 & 36.18, Wavelet Threshold 0.68 & 35.84, finally the weighted median filter 0.72 & 20.04. These set are a perfect example of where the quality judgement of SSIM falls short. As you can see the images have fairly good SSIM scores regardless of how readable the actual words in the images may be, but the actual structure and how close in comparison they are to the original is good. Now on the other hand PSNR scores are a bit on the lower side especially for the weighted median for example, showing how the PSNR gives you more insight on how off the image may be. Here it seems that the mean, weighted and DCT seem like the only filters that caught "New York " from the sign despite the other words being over smoothed in the denoising process. Another side note, just like we had before when we changed faces the same can be foreseen when it comes to pictures with text, the SSIM scores would not be much different between two signs with the same color but different words and could possibly be a misjudgment when trying to judge quality. When it comes down to detail, the best thing would be to use our eyes and judge readability and PSNR to actually know if there is a difference or not.