It needs to be done in the heat of the day, when the flowers are fully open and dry. You can use a soft bristle brush, too, like the kind used to apply facial blush makeup. Dip a clean brush into the male flower, then dust lightly into the females.
Since you're not there in the midday, though...