Brixham isn't easy to get to in summer, from anywhere except Brixham (even by bike).
Dartmouth is ridiculously expensive and full of grockles for 7 -8 months of the year. Kingswear, the Brixham side of the Dart, would be better but the queues in summer for the ferries to Dartmouth clog the roads solid for hours.
Anywhere on the southern edge of Dartmoor means that you will either have to navigate the couple of roads to Dartmouth and then use the Dart ferries to get to the Torbay side, or navigate via the complete bottle-neck that is Newton Abbot.
Exeter direction? You'll be okay on the M5 and the first part of the A380 but where it gets to Newton Abbot and the Pen Inn roundabout it will be at a standstill. Then you will have to take the Newton Road towards Torquay (average speed in summer: backwards) to get on the Marldon By-Pass. This will deliver fairly rapidly (30 mph if lucky) to the next standstill. From there to Brixham is a lot of single-carriageway road being used by every other commuter trying to avoid the other snarl-ups.
Starting out from Newton Abbot will mean getting stuck on the Newton/Torquay road and the Marldon By-pass (before you get stuck on the way in to Brixham).Torquay is nice enough but prices are not cheap and you have to get to Brixham via Paignton (where you will get stuck).
Paignton is cheaper and for a reason. You will get stuck between Paignton and Brixham.
Even navigating these routes on a bike will mean that you'll get stuck quite a lot.
If you are working in Brixham, try and find a place to live there before you even think of elsewhere.
If you come down here at this time of year and spot a nice place, try and imagine all the roads and lanes (which most of the roads actually are) being used by gormless holidaymakers crawling along at 15 mph (if the traffic is moving at all) because: a) they are looking out of their car ooh'ing and aah'ing at how picturesque everything is. b) crawling down the middle of the lane because the high banks scare them shitless. c) they've just got a glimpse of the sea (again). d) they're on holiday, what's the rush?
Have fun.
How about Totnes
If you're worried about the NA to Torquay Road amd Marldon................cut through the back lanes via Compton or Ipplepen
)

. Only one club if your desperate but easy access to m5. And the beach. 

) in case the missus ever signs to this forum ...