Hate to do it, but...

I let it go and explained it off as a miscommunication, since I told him where I would be cutting prior to having done it, even of he didn't remember.

Just looking, really, to save any future frustrations on misconstrued border lines.

The law says if it intrudes onto your property you can do whatever you want with THAT part of it, even if the tree itself is on his property.
I know a guy that cut the top off only his half of lilac bushes on a fence line.
My Brother didn't want them cut at all as it was a privacy bushes/wall, but the other guy wanted them cut down to the top of the 4 foot fence so he cut the stuff on his side of the fence and left the other side with the rounded tops.
We laughed about it actually, because we know he did it just to make them look like hell so my Brother would go ahead and cut his side, but nooooo.
They still looked normal from my Brothers side, but goofy as hell on that guy's side.:D