Just outside of Ludington, there is a place called Bortell's Fisheries. It is one of those places that you would miss unless you knew to actually look for it. I ate there the last time I went over to Ludington for the Fourth of July and I recalled that it was pretty good. It came on the recommendation from another blog I used to read.
So if you are in Ludington, I would highly recommend this place.