When I did the swap, I rode with the XR hub and never noticed any difference. The XR wheels can be picked up cheap, so you can have one set mounted with street tires, and a second with knobbies.
But if you have the money, there is an aftermarket hub company (Talon maybe?) that builds an alloy cush drive unit for the XRL that should fit. Way too expensive in my opinion for the benefit.
1993 RD07 Africa Twin
1994 XR650L farkled
1985 XL600R (sold!)
1985 XR600R tagged
1983 XL100S, 1980 XL80S, and 1985 ATC250ES