Details below: following a negative interaction with two NCOs, should I drop it or pursue it further?

So today we we're loading up a bus with people. Some of us were early. A sergeant said that we'll need to double up because it's going to be full (it was no where near). I told him I'll move when I need to. I wasn't trying to be insubordinate or disrespectful but I guess he took it that way. He said "no you move now. Who the f**k do you think you are?" So I moved. Then his SSG busy told me he needed my seat (all to himself) so I moved again. And all they did was talk trash on me for the day. So my question is am I wrong, or was he? If he was do I take it up with someone higher up the chain of command? Or do I ignore it? I have been told way worse things from people better than him. He's never liked me. and lots of people think he's a bit of a jerk To put it tactfully.