I would hate to go through life not doing something I REALLY wanted to do because someone else is scared of it.
this is a valid statement, and I feel the same way, but you never should have made an agreement if you didn't mean to keep it. You effectively bargained to get your foot in the door only to go back on your word now simply because you want something new. Not good for maintaining your integrity.
Their are way too many sporty standards for this to even be an issue. Many options have been suggested. A street fighter or SM sound like the best options. They are quick but top speed is still limited due to the naked factor.
Last, you will ride faster on a sport bike. You already said you leave your buddies behind because they go too slow. You will ride as fast as you feel comfortable, and a sport bike simply makes it much easier to go faster comfortably. Your own words imply you will go as fast as you feel comfortable so give that argument some further thought.