I'm not sure if any of us know technically what it does, but in the end it produces much sharper shadow lines. If you look at the shadows closely, you will see that an angled shadow line comes out in small squares (like pixels). Lower quant reduces the size of these boxes to produce cleaner shadow lines. It's not the difference between "blurry" and sharp, but about straight or broken into little squares.
Mattn was playing with the default -quant setting recently, but I don't know where he left it. Small quant values (such as 2) resulted in map compiling errors for some maps (max lightmap). Quant 2 is tons better than 3 and in my opinion our goal should be to support it, but I'm not sure anyone knows a good way to get there. I was told increasing lightmap size is not a good idea.
IMHO we should use the setting, which produces the best visual quality for the map compilation. Enhanced visual quality on the battlescape in the end results in getting a better atmosphere & also would justify even longer map compiling times.
Great to see that there are experiments regarding this undertaken already...
