XD-3 here as well. It's only a little bit louder in clean air than my RF-1000 was, but in rough air it was REALLY loud; once I fixed the windshield, life was good. Some air gets in even with all the vents closed; you can't seal it off quite as well as a pure street helmet.
Goggles fit under the shield, and the rubber seal is even designed so you can close the shield over straps. Shields are a bitch to change, so I just use tinted goggles. A pinlock takes care of the fogging on rainy days (no goggles tho).
Having the peak is great--I never realized how helpful it was until I went back out in the RF-1000 early in the morning and was constantly cursing the sun in my face. Once I got my windshield adjusted correctly, I've never had any issues with the wind catching the peak when I turn my head or look up. Even on my KLR with the stock (tiny) shield, I can ride all day without any more fatigue than I had with a street helmet.
If I lived somewhere where it actually got cold, I'd probably have a street helmet with a chin curtain for cold weather, but the XD-3 is fine for California.