Yes indeed, the difference between the partners in both cases was very great. Marie Festetics (one of Elisabeth's ladies in waiting), summed up Emperor Franz-Joseph as follows: "The Emperor possesses intelligence and quick understanding. Yet he gives his imagination no time to develop. Thus he sometimes acts abruptly, overworked as he is and incredibly devoted to his duties. For him only primitive concepts exist: Beautiful - Ugly, Dead - Living, Healthy - Sick, Young - Old, Clever - Stupid. These are all separate notions to him and he is unable to form a bridge leading from one to the other......His ideas know no nuances" (Marie Festetics' Diary: Entry of Nov. 28th. 1873). Although he loved her dearly, he never really understood Elisabeth and described her ideas as "Wolkenkraxlerei" ("Cloud climbing"), using the word in a pejorative sense. For her part Elisabeth always felt misunderstood. Ironically, the only member of her immediate family to share her high ideals and political views was her son, Crown Prince Rudolf. He most resembled his mother, while the girls took after their father. However, for whatever reason, Elisabeth failed to realize this until it was too late.
I think the Brigitte Hamann biography of Elisabeth is available in an English translation (as far as I know the English title is "The reluctant Empress")