I didn't have to. Lived o/s for many years, not required to pay tax in Australia. Guess it depends on agreements between particular countries and type of employee (PAYE etc).
It depends on how you intend to declare yourself to the Australian government, and the agreements between the other country and Australia with respect to tax.
I lived in Switzerland for 15 months. I was a resident of Switzerland (not a permanent one, but a resident nonetheless) and of course had to pay Swiss taxes (administered by the canton). I asked our family accountant for advice before I left Australia regarding tax status in Australia.
He said to me that I had a choice of declaring whether I
was an Australian resident for tax purposes, or not. In theory, there is nothing wrong with the latter option and it means I wouldn't be subject to any Australian tax for earnings outside Australia. However, he said that if I were to declare myself a non-resident, I would lose tax-related benefits accorded to Australian residents - OK not a big deal. The other thing, he said, was that if I were to return to Australia and thus re-declare myself as a resident for tax purposes again, the tax office may investigate a possible case of tax evasion. If it was more than five years away, maybe not a big deal, but less than that would be suspicious.
Given that the maximum I was likely going to be in Switzerland would be three years (very optimistic) at the time, I decided not to declare myself a non-resident for tax purposes in Australia. Whether or not I would have been subjected to scrutiny had I gone through with it is moot, though I ended up having to pay additional tax in Australia. (Switzerland and Australia have a double taxation agreement which means you pay the relevant government the difference between the taxes due based on the same income in both jurisdictions). Thanks to some shrewd deductions by our accountant, the amount I had to pay the ATO was rather small, cf. many thousands of dollars. It does go to show that I was not paying so much tax in Switzerland, even though I lived in one of the highest taxing cantons in the confederation!
One thing now is that since HECS/HELP has been ruled by the government as payable irrespective of where you live in the world, there's nothing you can do to escape that bit of taxation (except death, I assume). That said, I'm not sure if Australia would issue an Interpol after you if you fail to declare and pay your HECS/HELP debt each year.
.
