Digital Light & Color

I posted this on the old message board by mistake:

Hi,

I have been scanning some negatives to 48 bit TIFF's to edit in Picture Window Pro and have noticed if I save with LZW compression the size of the file actually increases.

I understand enough about compression algorithms to know this can sometimes occur but I was very surprised to see this happen because if I perform LZW compression of the same file in IfFanView it manages to shrink the file size quite a bit and it still opens fine in PW Pro 5.0.

I could of course just use IrfanView to compress the TIFF's but this strips the EXIF data.

I'd expect the images to compress at least a bit in PW Pro 5.0 because many have plain blue skies.

So some questions:

Is there a problem with the LZW compression in PW Pro 5.0?

Is it likely to be fixed and if so does the compression retain the EXIF data?

The reply cam via email from Kiril:

Dave,

The problem with compressing 48 bit files is that they have quite a bit of noise in them. (The least significant bits are really mostly sensor noise.) And, of course, noise is not compressible. In testing we have zeroed out two or three of the least significant bits and obtained much better compression. If you run the same algorithms on 8 bit data, you will see significant compression.

Kiril

So I'd like to follow up here in the right place :-)

I can understand why noise would produce the effect of increasing the file on compression and compressing a shot from my d-slr as opposed to my scanner does indeed give a reduced file size when using Pw Pro.

However I have tried IrFanView and another program that dooes LZW compression and both achieve significant file size reductions on my scanned files. So given the answer above the implication is that these programs are tossing away high bit information which to my mind means they are not doing lossless compression.

Or are they doing something cleverer than that in that they recognise the noise for what it is and do not apply the compression algorithm thus not inflating the data required to describe the file contents?

I find that hard to believe that a program such as IrFanView that has been around for a long time is tossing away data when doing LZW compression.

I am also pretty sure if I open a file LZW compressed in Pw Pro and save it again as a compressed file that the file size still increases (I will need to check this when at home later) which I would not expect if the noise had been effectively removed by ignoring the highest bits when it was first compressed by IrFanView.

Are you sure your compression algorithm for LZW is working to spec?

Dave

I found out what is going on. IrFanView is saving the files as 24 bit not 48 bit despite me not selecting the option to change the bit depth so it isn't compression that is shrinking the files but the conversion to 24 bits.

Interestingly using ZIP compression I do see a reduction in the 48 bit image file sizes when compressing in Pw Pro. Not a great deal but they do shrink compared to actually increasing with LZW.

So given the original explanation about noise being the likely culprit for the increase in size using LZW does anyone have any tips how to reduce this noise when scanning negatives? I am using an Epson V750 and Silverfast (as it puts the EXIF in and Epson scan does not). I have access to VueScan as well.

Dave