Reading this, I simply had to laugh, trying to imagine the poor Queen as a young mother and a new monarch even having the energy to have an affair!
Seriously, think of the logistics! Unlike the other royals, the Queen doesn't simply "pop over" to spend a night with private friends. She sleeps every night at BP, Windsor, Hollyrood, Sandringham or Balmoral. When she rarely spends the night elsewhere, she is accompanied by dressers, maids, security, footmen, pages, private secretaries, equerries, and even her own doctor. (which probably explains her lack of invitations to weekend house parties!!).
Sorry - a woman like the Queen simply wouldn't allow herself to play out an affair in full view of an army of courtiers, staff, and servants.