AITA for not paying my dad back after he technically owes me tens of thousands?

For context, my dad (60M) cheated on my mom, had another family, and spent the college fund my mom had been saving for me since birth on the other woman. That’s actually how my mom figured out he was cheating when I (29F) was around 15 years old. He was diagnosed as a narcissist and never talked about it with me, never directly apologized. We sort of act like it's never happened, he's not emotionally open; he wants to be close to me but I have a wall up.

Last year, when I was unemployed, he lent me $300. The other day, he called me saying that his wife told him it was actually $800. I sent him the receipt showing it was $300. But honestly, I’m just sitting here like… it’s $300, you’re my dad, and you technically owe me tens of thousands.

Would I be the asshole if I just didn’t pay him back? He says a year is too long and I need to learn about responsibility. What should I tell him?