Oil seals keep oil in, they're crap (ish) at keeping water out.
My guess, based on my own similar problem will be water ingress past the rubber boots into the swinging arm. Water collects therein, ride the bike, final drive warms up then, when it cools it sucks water in, past the input seal.
1, drain the final drive oil, if it's milky then it's confirmed.
2, drop final drive, if water has got in then the input bearings will be running in water and not oil = not good. can you move the input shaft back and forth or up, down, sideways? If so it's shagged the bearings.
3, whilst in there check the UJ's as if that's been running in water it's also probably shagged.
Sort it soonest as continued water exposure will pile on the damage....
Roger,