An ex-Japanese envoy said the "sexist" comments "betrayed the close bond" between the countries.