Of course with any used car/car dealerships you will have the occasional lemon. It's just the way it is in the industry. But, you can rest assured that many of the cars imported from Japan and other places go through a rigorous and stringent export process before they come to our shores. But of course you're going to get the occasional one slip through under the radar.
What I suggest, and it seems you've already taken some prior precautions, is to find an importer which you trust, or someone who deals regularly with imported vehicles and get them to have an in depth look into the car you're buying. Personal opinion - I wouldn't go RACV - but take it to an importer or equivalent mechanic.
Importers, as I say, go through such a process to get cars here. If they start to import lemons situations can back fire and import licensing can be revoked. Cars with rust 'must not be imported' cars with prior accident damaged or chassis damage 'must not be imported'.
So don't let hearsay deter you, just take the right precautions and speak to the right people and you should be clear.