Tony, the impedance switch affects the power transformer. Usually connecting different taps on the secondary windings used for the power amp section.
Speakers of the same sensitivity (say 88dB/Watt/meter) would produce the same SPL if the power input to both is the same. Whether they be 4 Ohm or 8 Ohm units. But to get to that same power and SPL level, the 4 Ohm unit will require 3dB (about 30%) less voltage on its input terminals.
So if you want the same power available, you can relax the supply voltage to the amplifier by almost 30% when using the 4 Ohm speaker. This will have a good effect on the power amplifier section, as less voltage sitting across the output stage will heat up the amplifier less.
Semiconductors have only two enemies. Heat and voltage**. So less voltage and less heat is better for the longevity of the amplifier. If you're kind of green-minded, it's better for wasting less energy too.
So if you are never requiring the extra voltage in use with your 6 Ohm speakers, there is no point in having it there. If the amp sounds good with the 4 Ohm setting, use the 4 Ohm setting. By all means use the 4 Ohm setting even in the unlikely event that you find a weird pair of 16 Ohm speakers to connect, IF it sounds good.
If the amp ever clips or just sounds "stressed" on the loudest passages that you play, then use the 8 Ohm setting for a higher supply voltage. It would be the lesser evil in that case, IF it brings about an audible improvement. Higher voltage will not of necessity sound better. There are very many factors involved.
The objective of this switching is to extend the life of the amp and to protect your pocket. While it is very possible to build an amp that is both extremely powerful and perfectly reliable with all reasonable loads, you may be pleased that you don't have to pay for it. Especially if it is a multichannel amp.
**Beer and wine inside the amp are also enemies. Beer and wine inside the owner initially get "transformed" to voltage and heat inside the amp by a complex Math usually referred to as the "Homer Transform" which splits the emotional domain from the physical domain for analysis, much like Fourier Transform splits the time domain from the frequency domain. Imaginary numbers become misunderstood benefits etc.
