As a followup, everything is working good with this approach, just need to make sure to allow adequate driving time between appointments or they will be unassigned.
"cannot be visited within time window"
Same with first appointment of the day - driver needs to start before the appointment or that appointment needs an increased time window for driving time