Frederik VII was the master of scandals himself, so I dont think his mothers reputation, good or bad mattered that much. Frederik liked his mother, and they were very much a like. Much more than him and his father.
Christian VIII and Caroline Amalie had a very happy life together, the only thing missing was the lack of children as you mentioned.
During CVIIIs reign she wasn't popular, but that changed after she became a widow.