I was always under the impression that when someone shorts a stock, they borrow the shares, sell them immediately, and then buy them back when they want to end their short position and return them to the original shareholder. (Hoping that the stock went down and they can buy them back at a lower price than when they sold them.)

Therefor the person who bought the shares from the short at the time of the vote controls the vote.

Am I wrong?