Although not as well documented on the more modern Hapsburgs as many of you are, I had always thought, as a well known fact, that Franz Josef had several affairs during his lifetime. I even thought his wife Sisi not only did know about them, but that she also encouraged them. Didn't she call Schratt "the friend" in a somewhat ironic way? And I might be wrong, but she wrote to someone that Franz Josef was in love with Katharina and that was great (although this part I might have understood it wrongly, I can't really recall where I got it from).
Franz Josef might have been religious, but so were (and much more fanatically), kings in the 16th and 17th centuries, and they indeed had illegitimate children, as it has been posted earlier. Meaning no offense, CFH, I don't think religious principles should be shown as an absolut theory to prove Franz's faithfulness.
I think it's much more probable that he DID have this lovers. I think it's natural. By all accounts he was a passionate man (let's remember that one of the only times in his life in which he defied his mother was when choosing Elisabeth as a wife, and they didn't know each other, so her intelligence or culture wasn't obviously what made him take that decision). How could a man how was sensible to woman beauty, not have a lover when his own wife was refusing to let him in her bed? And given that this man was not only atrractive, but also powerful... I think it's just the perfect combination.