Because you are served decent food & beverage in a far more pleasant environment than the rest of the terminal and the lounge itself is more comparable to a decent restaurant than the rest of the terminal.
I will confess to having not been in the QANTAS F Lounges in Sydney or Melbourne.
However, I have been in BA's LHR F Lounge many times.
It is not an environment that is anything close to a posh restaurant. Certainly, I have never felt even vaguely uncomfortable the times I have been in there wearing shorts, a T-shirt, and thongs (well, sandals, but I think the point is not closed-in shoes).
As far as I'm concerned, the point of the lounge is to relax, grab a drink and maybe some food before you get onto a plane. It's not to be the equivalent of dining at a high-end restaurant. Certainly, if we were judging by the quality of the food available, then at most you're talking about a high-end pub, or maybe a restaurant a step or two above Sizzler.
Most of these F Lounges are pretty big. I'm sure you folks can find a part where other people who still think they're flying in the '50s are sitting. Or maybe we need a bit sectioned off where the Toffs can huff and puff amongst themselves about the lack of decorum in the working classes ?