I pass at the first safe opportunity, especially if I'm solo or with one friend. Drop the hammer and dispose of them with a quickness. I think most drivers (and many riders) are clueless to the thought that it's common courtesy to let faster traffic pass. If slower drivers/riders regularly moved over for faster traffic, then there would be a lot less aggro passing going on.
If I happen to be on one of the "destination" sections of good road on that day's ride and I can't pass immediately, I tend to pull over for a minute or two and let a gap build up so that I can enjoy the good bit at my pace. There are few really good sections of road around here that are more than 10 miles long. IMO, that works much better than following along being frustrated. Some days you're just F'd with traffic no matter what you do.