There are "BoardWalk Bakery" signs up near the awnings.
The front of the BoardWalk Bakery building. There is some outdoor seating to the left.
You can get your Resort Mugs refilled at the BoardWalk Bakery. (A "Resort Mug" let's you get free refills in that mug, for a predetermined amount of time. It might be a good deal if you drink a lot of soda.)
The BoardWalk Bakery is located between Ample Hills Creamery (to the left) and Trattoria (to the right).
Here's a photo of the BoardWalk Bakery taken from the Yacht & Beach Club (i.e., there's a giant lake between the two locations).