When I originally wrote it, it was done under the assumption that it was for when the user would rotate their phone/device rather than resizing a window. This way, it would use the shortest edge to consider the framing.
In your case, it sounds like you don’t want to flip it at all unless the user resets the view to frame the content.
An alternative idea is to move the camera back/forward when resizing the window instead of changing the fov.