The first annoyance for me is not riding at my "normal" pace, at my natural speed things flow a lot better, positioning for corners (and the lines) make more sense as the speed is matched accurately to the bend (not mated to an exact number on the speedo) my brain wakes up and goes in full bore information processing mode instead of me having to keep forcing it out of standby.
This is not a criticism of the IAM as they obviously have to work within the speed limits set by the law - you know the ones originally designed for Morris Minors and Royal Oilfields, only we now have predominantly lower speed limits these days, yet the vehicles are capable of travelling a fair bit quicker with better safety margins.
The hardest thing for me was the need to be so exact with speed, generally I do slow right down for 30's and 40's, but still use my brain, if the 30 limit starts a mile from the village (like so many these days do) I may not be bang on 30 as I go through the signs.
With modern bikes 30mph is a real crawl, the GS is not too happy at that speed, third gives little control and second is a bit revvy - and both have the bike surging a bit, with 40's not being much better as they seem to sit between third and fourth gear.
As roads are rarely dead flat I find myself looking at the speedo a lot to make sure I don't creep up a bit too quick, or get penalised for not making progress, it seems like you spend half your time looking down at the speedo instead of where your going.
Well done for passing.
I think that it is worth being precise with the 30s and 40s for various reasons. With experience you can then dial in 30 or 40 without having to look at the speedo. A good test is to set yourself up at 30 or 40 and then check how accurate you can be.