I've ridden a buddies XR-L quite a bit, both pavement and gravel. Nice bike, peppier than my KLR 685. This summer I bought a tagged 07 XR-R and it is a hoot, different bike entirely. I'm commuting on the -R now, but it's not much highway. Seems to have plenty of grunt, even at 70 mph. If you can tag an XR-R in your state, I'd go that way.
btw, I think the oil change intervals on the XR-R are like every 600 miles, the XR-L is probably higher. I'm changing the XR-R every 1000, it's quick and easy. Valve adjustments are easy on the -R too. I took the knobs off and am running Pirelli Scorpions, thinking about changing the front rim to a 19" since I plan to do some track days on it.
The -R doesn't have much of a subframe, so no passengers. I've got an aftermarket subframe for carrying a tailbag, but I wouldn't haul a passenger. I've got kickstarting down easy peesy!