There are quite a few proper decent restaurants in or close to Boulogne. La Matelotte, Hostillerie du Riviere, L'Epicure and Hotel Atlantic (Leigoise fine dining and a bistro) - both in Wimmereaux, La Ferme du Verte in Wierre-Effroy. Between Calais and Boulogne is La Chaloupe in Wissant, a more bistro-y type of rustic moderne, serves a great fish stew, sort of Atlantic bouillabaisse, but other stuff too.
A previous highlight of any French trip is Le Cygne in St Omer. The new owner has changed it just enough to be irritating to those of us who liked it the way it was - excellent value and excellently cooked and presented food with regional character. Great unobtrusively attentive service. If you never went before you might find it great, the menus are much the same, but the service dropped a notch and there is something now quirky to the point of gimickry about the food or its presentation.
The previously mentioned Froggy's in Montreuil was good value too the time I visited. Conveniently located opposite the big central car park. A walk round the ramparts will either give you an appetite or help walk the meal off. I have been somewhat disappointed trying several other establishments in Montreuil food wise.
The Chateau du Montreuil excepted, even there the special menu is <€40.
All the places mentioned offer really good value menus - 3 to 5 course for between <€30 to €50 for lunch times. Evenings are usually more.
There are lots more places but those compromise my personal favourites. I know L'Epicure doesn't have a www, but the food is sublime. One chef, one front of house server, a husband and wife team. They get a bashing on trip advisor occasionally - I can only think it is some malicious troll, as the food has always been superb ever since I first went there in the mid 80's.
La Matelotte is more of showy place, food slightly under L'Epicure. Check them all out, wwwwise. Have a look at the menus and offers and see what you fancy.
The coast road from Calais to Boulogne is an attractive little ride. The hinterland of Boulogne is quite hilly, at least undulating, and if you travel in the general direction east, and if you follow Wapping's advice, and have a look at a map, you will find lots of possibilities, Cassel being only one. Its a nice area for bimbling.
Most restaurants close Sunday evening and Mondays, although in tourist areas this may sometimes change.
Double check you are not clashing with French or Belgian Bank Holidays - everyone likes a holiday! Especially if you normally live in a small apartment in a tower block in Paris.
Other places to visit to fill in the time. Eperleques and La Coupole, both not far from St Omer.