I got the "remove helmet" thing from a lady on the trail who told me that we look like "Predator" to the horses with them on. Ran into the same lady a couple of years later and she told me the exact same thing... guess she had a fascination with that.
I don't have any problem going out of my way to accomodate the horse riders, but I think it's a mutual deal. My encounter this past weekend was peculiar in the fact that the riders didn't do anything at all for a few moments, other than stare at me. I don't know if they didn't know what to do or what. I've run into several organized trail rides and there is always one or two horses that react to the bike or myself. Last year my wife and I were on the side of a "designated" forest road, off of the bikes and helmets off taking a break when a group of riders came by. I thought one horse was going to pitch it's rider. Found out later that there were stables on down the road and this was just one of their rides with what appeared to be inexperienced riders. That scares me as I don't want to see someone get hurt.