Make sure you visit the Trip to Jerusalem - oldest pub in the country, built into the rock underneath the castle. Good live venues include the Rescue Rooms and the Maze - the latter, especially has some great roots and Americana acts on, on a fairly regular basis. The Music Room have a shop just down the road from the Maze, should you want to pick up some good quality folk instruments..!
If you want a good real ale pub out of the centre, I can recommend the Lion in Basford - it also has live music.