Thanks - these considerations are extremely helpful.Maybe not cancelled the instant citizenship is conferred, but a visa certainly would not (or shouldn't) be granted or renewed after expiry to an Australian citizen.
Departing on a foreign passport without a in force visa, would mean a chat with ABF when going through Immigration to understand the circumstances, but once citizenship is established etc etc one will be on their way. A person is not prevented from leaving the country for visa irregularities, entry can be refused though.
Now, I am very much inclined to go down the route that you describe which essentially is leaving the next morning on my current non-AU passport, having a friendly chat with the ABF incl. showing them my citizenship certificate, my new work contract etc. I hope that they AFB does understand (but will be unhappy), but does not find a reason to not allow me to travel.
Then I sort out the Australian passport overseas at a higher fee (it's an easy Asian country, so should not be any dramas), and re-enter on an Aussie passport once I have it.
Seems like much less stressful, and cost effective as well (as I can book a flight well in advance).