Yes, I agree. We can not know, what really happened between them. Supposedly Franz Joseph infected Sisi with an venereal disease around 1859/60, though I don't consider this information for trustworth.
It's claimed, that he had some affairs with women (one polish countess was rumoured) in that time. But he definitely didn't have syphilis. Syphilis is very serious illness with many symptoms and neither Franz nor Elisabeth suffered them. If he would have been infected, he didn't cartainly reach so advanced age of 86. In some biographies which I have read, was mentioned gonorrhoea. That would be more likely, although I still doubt about such rumours. I have never read anything about the symphtoms typical for those venereal diseases appeared on Elisabeth's or her husband's side. I admit, that the Emperor was not all the time of marriage absolutely faithful husband, he had few mistresses like A. Nahowsky or K. Schratt. But according my opinion, he was more careful and didn't choose disreputable women like for example Crown Prince Rudolph.