Yes mate, I got caught out the first year I went. Better to get there Friday morning at the earliest. But you may struggle with a spot if you pitch up late on the Saturday, as it is generally heaving by then.
The main town is literally a 10 min walk, so handy for breakfast in one of the many cafes.
It is in a beautiful part of the country, with an amazing array of Alpine passes on the doorstep. Getting there and returning, I have tended to use the Black Forrest route. It is also a beautiful region to transit through by bike and make a stopover. Just avoid lake Konstantse.