Riding a smaller bike just needs a different mindset. Why do you NEED to go rip roaring about the place? You miss so much of what you are going through and past. I can get by a caravan or truck by planning the move, otherwise doing 50 on an A road is no slower than doing 60, not in the real world of things. It doesn't matter if you get to the pub 1 minute later.
The difference between 70mph and 60mph is only 6 seconds per mile. You would have to do 100 miles at a steady 70 to make 10 minutes difference... try and maintain 70 in this country, for 100 miles! Think about those long trips you've had and passed a noticeable truck, then 20 miles down the road gone in fuel and as you're putting your gloves back on that same truck goes trundling by..... reality check time! Big bikes are only needed to fill the bank balance of those that make and sell them. When I was a rookie biker the 750 was a behemoth. Now it is a 'middleweight' - no it isn't it is still a behemoth, just because there are leviathans bigger than it does not make it a 'small' bike. A 650 is an ideal size for real world biking. Christ alive, I did 4000+ miles from Plymouth, through Spain across the Strait and all along the N African coast and back through Greece on my Morini 350 (344 actually). I never felt undergunned. Keep it real.