Common sense says leave 'em off. As Big Lee points out, why carry around a load of extra weight for no gain. On the other hand, as Thunder points out, common sense also says leave'em on to act as protection and a visual deterant.
It's not about protection and Looking Big (although those are useful and entertaining, respectively), but the reason I leave my panniers on is so I can just throw crap onto the bike and not care about preparation. I can still filter through all the gaps I can fit the bars through, and I don't have to worry about preparation if I randomly decide to go shopping on the way home, as there's a fair old chance I'll just be able to throw everything in the boxes and ride home when done.
Plus it makes for an easier setup in the morning - I just bundle the bike cover in one pannier, the locks and work stuff in the topbox, and I've an empty pannier to leave the lid and gloves in when I'm at work.
........fluoxetine 20mg daily........fine Scotch single malt 300ml evenings......3 good pinches Hamsterjam's finest twice weekly........and the occasional Fray Bentos steak n kidney pie............works for me!
David
PS Further recommendations on the fine malts can no doubt be found in the Scottish section
Do what your comfy with. Personally I just use the topbox daily but can see the merits of having all three boxes on the bike all of the time. Whatever you decide it has to be your decision cos everyones needs are different. Doesnt help really does it????????