yep, amps get warm quickly and get hot after a while of high usage. Most come built in with heat-synchs and fans to disperse heat quickly so are designed to withstand the heat so dont worry about that.
The fuses can be taken out by gripping the sides and simply pulling em out. They are quite cheap and most electronic stores stock them.
Its possible that your amp is the problem, if you are in a position of obtaining a temporary replacement to use as a check it might help.
Is it only the sub thats affected or the whole system?? if its the whole system it could be your head unit