If you mod your car it'll probably be illegal and even if it isn't, you can still be defected for it and engineer's can still be reluctant to put their name on it (especially if the owner is clever enough to 'opt' for an RB20). I believe there's actually some legal problem with putting a smaller engine into a car too (at least in the sensible state of Vic).
I have been told by guys who've swapped RB20s into Silvias that it's the worst decision they ever made and when their budgets get back up to strength they usually reverse the swap or sell and buy another. A Silvia for sale with an RB20 is like a girl with the Jack. SR20s are capable of much more power, much more durable, better on fuel if that's on your list and more reliable.
But, you've got an R34, there's a reason Nissan dropped the RB20 when they thought of the RB25; and all your engine mounting points, gearbox mounts, driveline, aircon, other electrics etc. I would assume would mostly match up with an RB25DET. I think you'll find money you spend on the RB25DET will be way less than the cost difference or time consumption if you diy of putting in an RB20. If you just wanna be unique, try a 2JZ (Supra) or something?