I bought my TE 250 at 49. Been riding street since the 70's and my only prior dirt experience was as a kid on a mini bike.
I really enjoy it and it is a great work out.
You just have to ride within your skill level. The good news is when you do go down, dirt is softer pavement.