Personally...I would go for comfort and take the gravel sections or dirt track a bit slower and enjoy the ride over having to sit on a bike without those comfort aspects for the amount of time it takes to get to the harder riding. Just my 2 and a half cents worth of .....well you know....lol.
If this is a first time ride of the harder places why not use it as a scouting trip perhaps for a trailered trip on the lighter bike at a later date? That way you can get there faster possibly and spend more time on the harder more challengind trails once there.....I know...it's not a bike adventure if you add in a trailer to carry said bikes....but.....you sometimes have to accept things you normally wouldn't to get the best out of a fun vacation.
For me it's the whole trip as I'm not into a possible bike damaging situation far far away from home.