Trent,
Spending extra energy on longer rides takes away energy that can be spent on getting faster. If your goal is to complete your races and getting faster is not a goal it is ok to extend the long rides. If you want to get faster then put the work into the speed sessions and follow the plan on the long ride days.
Lynda