I'd say it's the spiggot bush and the reason i'd say this is that I noticed on the weekend that my car made this noise about 10x (when clutch is in selecting a car, noise stops when clutch pedal is released) and I had the exact same noise/feeling about 2yrs ago and it was the spiggot bush that I got replaced (with a new clutch at the same time).
From memory its like a $15 part but requires a day of labour to get the gearbox in/out and not much more time to swap over a new clutch. The noise is extremely irritating/embarressing though, pity it costs so much to fix such a cheap part.