As a general rule, the best size for house numbers & letters is 4 inches in height. However, it also depends on how far from the road your door is:
- If it's right next to it, 3 inches will do;
- If it's 130-160 feet away, aim for 6 inches or more;
- If it's over 300 feet away, it should be at least 10 inches high.
To be on the safe side, you might want to consider adding the house number to your front gate too, if your house is really far from the road.
To maximize visibility, don't forget about the kerning, which is the space between letters and numbers: they might be the correct size, but, if you mount them too close together, they could still be hard to spot.