Stay in Mont-Louis for a few days. The camping is convenient with the Gulf of St. Lawrence on one side and the Appalachians on the other, and the restaurants are good. You can also buy delicious smoked fish at Atkins et Frères.
Gaspé is a bit dull, but it does boast a museum with public archives, as well as the Jacques Cartier monument, which documents the arrival of the Europeans and their interaction with the locals.
Percé is also worth more than one day. This little town has survived more than one storm that devastated the seashore. There are informational signs along the temporary boardwalk discussing the effects of climate change on the coastline. You can see remains of the former boardwalk.
As well as damage to neighboring property.
It is probably worth the overpriced boat tour to Rocher Percé and Îsle Bonaventure. The latter is where the gannets nest. Just the corner of the island is visible here.