...but either way the PC is going to get caught and dealt with.
This is as good a reason as any to threaten a PC for a small amount of coin: it sets up an ongoing antagonistic roleplay between the victim, the highwayman, and their respective friends and allies.
It's not fun to lose, but losing is part of every hero's journey before their ultimate triumph. That's okay; it gives you a motive to come back and win this time.
Try not to get too caught up in the specific wrong - coin is easily replaced - and think about this as a story arc that you and the highwayman's player are creating with each other. If you have the opportunity, perhaps reach out to the player OOC to make arrangements to continue the antagonistic RP (if you're interested in that). Do keep the hostility IC - the offense was in character, the response should be as well!
Even if you never get resolution with the original highwayman himself, how does this encounter impact Caylin? If it's had a strong impression on you, chances are it did on him too. Does it make him hate bandits with a vengeance? How might that hatred play out? A personal vendetta, a bounty?
These are some approaches I'd take to turn what might have been a frustrating IC encounter into fodder for more exciting roleplay.