I'm also a bedroom only player, and like you went from the XL+ to the Axe3. I'm sure that part of it is my increasing knowledge of how to use the Axe, but I find the 3 to be a huge improvement and very much worth the money. To me, the cab sims on the 3 make all the difference - the stock ML, YA, and OH cabs sound incredible (and the ML Freeman cab pack pretty much answers any mid-high gain issues you might have).
A couple reasons justify it to me. One is that, as someone who used to play live and now works a dayjob, my time for jamming is pretty limited, and sound is a factor as I live in a neighborhood and am married. So having something that has an incredibly high quality headphone jack and is operable from my laptop is really necessary; I tried doing the Two Notes Torpedo + 100watt head thing and it just never worked out in the way I wanted it to. This is something that is easily usable at home, and because it's so easy to dial in, I don't have to spend a lot of time tweaking - I can mostly just sit down with a guitar and have the tones I want ready to go immediately. And if I do have more time to tweak, it's incredibly easy to make very satisfying and distinct presets. So part of it for me is just that I have limited time to play, and I want to spend that time enjoying the instrument. The Axe 3 is the best and makes the 30 minutes to an hour of playtime I get per day a real joy.
The second reason - and this is an economic one - is that the Axe 3 has fully and completely killed my amp GAS, saving me a ton of money on trades and buying/selling amps. Before this, I owned a ton of tube amps and cabs - BE-100, DS40, Butterslax, Mesa mark series, Tone King, and on and on - and I could never find one that did everything I wanted and fit my circumstances. The Axe 3 slays every other option to me. I've literally never considered going back to tube amps since I got this, and that for me is worth its weight in gold.
I say take the jump. You're right that if you buy the FM3 you're always going to be thinking about the Axe 3. It's worth the money, and you won't regret it.