because you cant have anything in front of the headlight glass that could cause any distortion in the beam pattern or create the possibility of headlamp light being reflected into the vision of the rider (or in the case of mesh grilles, into the vision of the rider and oncoming drivers.)
The light from headlights, spot lights and foglights are meant to be completely invisible to the rider / driver and are not allowed to reflect back to the rider from any part of the vehicle ... thus allowing your eyes to be better adjusted to the dark, and making better use of the light reflecting back from the road surface / road hazards. If there is any other light hitting your eyes (i.e. from an oncoming drivers high beams (or low beams) - or a bright reflection from your own headlights hitting a plastic headlight protector) then it reduces your ability to see the road surface and road hazards. The fact that the effect of reflections from the edge of a headlight protector is significantly smaller than an oncoming vehicles headlights is not factored in by the rules.
So all headlight protectors (whether clear or mesh) are technically non conformant to ECE regs and therefore theortically illegal. However I have never heard of anyone being pulled over for them or for a bike failing an MOT because of them - there is no MOT box to tick that says "no no-ECE compliant headlight accesories".
If you are touratech or BMW or wunderlich or anyone like that however, you will sell them for use on road bikes and cover your @rse legally by saying "for off road use only" on the packaging and on the website / catalogue you buy them from.
That way everyone is happy.
