Well, this is all very interesting.
Firstly we need to establish whether we are talking about Solid State amps or Tube amps. Both have issues with differing speaker loads. Let's look at Tube amps first. The ideal setup is to match the output impedance of the amp to the input impedance of the speaker cab. This allows for the best transfer of energy from the amp to the speaker (Electronics 101, impedance matching). If you increase or decrease the speaker load (impedance) you will affect the output power of your amp and to a lesser extent, the frequency response. Either way will result in a lower volume. Now tube amps are more than happy to run into low impedance speaker loads, in fact, they will happily drive a totally short circuit. What they don't like is high impedance or open circuit. An open circuit allows massive voltages (20kv at least) to build up in the output transformer (called back emf) that actually punctures the insulation on the transformer windings and smokes it (we all know that smell, right). Generally, tube amps will come with three speaker impedance options, 4, 8, 16 ohms. Your mission is to choose a cab that meets one of these values (or is as close as you can get). However, if you decide to put a 4-ohm cab in an 8-ohm socket on the back of the amp, no worries, you'll just get less power (volume) out of your amp. The same goes for attaching a 16 ohm to a 4 or 8 ohm output socket. You won't do any damage, it just won't sound optimal. But, what you must NEVER do is connect no speaker to the output of a going tube amp...doing that in most cases will smoke your amp (Most manufacturers these days short the output when you remove the speaker plug to get around this very real danger, but old amps didn't do it. (this is the reason you never use low wattage speakers with high wattage amps. When a high wattage amp smokes a low wattage speaker the cabinet will appear as an open circuit. That is certain death to a tube amp)
Transistor amps also want to see the same speaker load as the output impedance of the amp (for the same reason as tube amps, impedance matching gives the best output, but not the most) However, if you reduce the speaker impedance on a transistor you will get an increase in volume (not a decrease like with tubes). The output stage of a transistor amp doesn't use an output transformer, like a tube amp. Instead, the output devices (transistors) are connected directly to the speaker. The lower the speaker impedance, the more current that flows in the output transistors. At some point, we reach a critical node (usually around 2 ohms) where the output transistor will overheat and generally go short circuit. This then connects the DC power supply of your amp directly to your speaker. The speaker shoots forward or backwards to its limit, depending on which of the output transistor shorted out first, and fries itself. If you go the other way and increase the speaker impedance all that will happen is your amp will get quieter and quieter as less current flows in the output stage (Ohms laws - same voltage, bigger impedance = less current. Less current, less volume) You can't hurt a transistor amp by connecting a 16 or 32 or any big number impedance speaker to it. Also, changing impedance, unlike tube amps, has minimal effect on frequency response, your amp just gets quieter or louder depending on which way you go.
So, rules of thumb. Always try to match your cab to your amp for best performance. With a tube amp, lower impedance is better than high impedance for the amp, but still not ideal and affects both volume and frequency response. With transistors, it's the other way around. Don't go low with a transistor amp or you risk smoke, go high if anything. Having said that most modern transistor amps run a system called load line limiting and if you go dangerously low with cab impedance the amp just shuts itself off for protection.
Sorry, that was bit of a rave but I trust it's cleared up a few issues for you. I'm an electronics engineer so have a vague idea about this stuff