This mostly only happens over the phone because the beard is usually enough of a hint for people to get it right in person, but the name should also be a pretty good clue if you're going to make a guess. (Or, you know, you could just not use gendered pronouns where they're not needed or if you must, ask?)