Col de la Faucille on the north side of Lake Geneva, (on the N5 north from Gex in Switzerland) is like a mini Alpine pass. From the top on a clear day you get a great view over the lake to the higher Alps on the other side.
There are some similar, although slightly less extreme, road leading down to Lac Neuchatel.
Overall, the roads in France are very nice - it's a seriously hilly area and many of the roads are just a series of bends joined together. Haven't really ridden on the Swiss side too much, although the countryside looks retty similar.
All in all, probably better roads for riding bikes than just about anywhere in the UK. No idea about horses though.