The Empress Marie used the Palace quite often in the summer, but really only for day visits. She met other members of her family there for tea in the garden, overlooking the water. I have an old photograph of one of these tea parties, but cannot scan it as it is in a very old book and I dont have a flatbed scanner. I think the palace was probably a fairly safe meeting place for the Imperial family.
The palace survived relatively unscathed by WWII, the Soviet regime caused more damage to the building than anything else. The first Palace on Elagin island was designed by Quarenghi, for Ivan Elagin, after whom the Island is named. The present building was designed by Carlo Rossi in 1817, when the island was aquired by the widow of Tsar Paul. The gardens were designed by the English landscaper, Joseph Bush, who also worked at Pavlovsk.
I think the choice of grey for the exterior is quite unusual, as it makes the building look very bleak in the Russian winter.