I don't think so. You've noted that you would feel more comfortable with "more athletic" ladies versus "bumbling" ladies. You're already making judgments about their riding abilities. I'm saying that I don't get into whether they are a good rider or not, I am concerned about someone based on who they are, not what they are doing. Guys who are concerned about their wives or GFs riding should consider the reverse aspect and the impact it could have on themselves. ie - if wife or GF shouldn't ride alone because he is concerned about her, maybe he shouldn't ride alone either because she is concerned about him. It goes both ways.
I'm the wrong person to ask anyway. My risk threshhold doesn't track with most people's.