When an E-4 calls me "hey" or "hey man" knowing that I am an officer, should I bother to correct him?

The fact that you have to ask that question, Captain, says volumes about today's Army. And Not in a good way. I am not sure how it should be handled, but ignoring it is absolutely not the way.

If you are off duty and in civvies even though he knows who and what you are, you might want to gently mention that he should respect the rank difference. When I was on active duty as an NCO, even though others were friends, we still had a certain courtesy between us. I remember that even in this type of situation, I would greet my commander with a greeting appropriate to our situation unless the higher ranking individual told me to address him or her as and gave me a name. Of course, on duty it was Sir or Mam or by rank. For enlisted persons it was Sarge or Sgt. It is sometimes a difficult thing to know how to address someone you know when in civilian clothes, and it is up to you to make the decision as to how you would like to be addressed.

I have always enjoyed officer's that believe they are better than enlisted personnel when off base and in civilian attire. If not in uniform and not a direct superior, it would seem more of an ego stroke for that officer to expect a sir or maam. Saying that I will say this, I refer to everyone as sir or maam in public when addressing them because that was the type of respect I grew up with.

Lacking the context of the situation....
I would go with a quick, on the spot correction.
Since it is outside of work, I can only assume out of uniform, I would go with a more subtle approach. At least the first time.

Context:
Outside of work,
You do not work with him directly.
Where you in uniform, PT's, or civvies?
Did he recognize you as an officer, at the time?
Was it a formal, informal or casual conversation?
I could come up with more...

SFC O Malley said it point on!

The E-4 knows better, should be reminded of his rank and your rank.

May be he doesn't recognize you. Reintroduce your self

Saying that to an officer of any rank is a good way to find your butt in a sling, both with the Officer and either the First Sgt. or commander of your unit

Yes. Customs and Courtesy.
I was on Belvoir leaving a meeting and was approaching about 8 US Army 1-2 stipers walking the opposite direction towards me. I was obviously ready to return their salute as appropriate but all walked straight by me (that silver star does stick out BTW). I called out "Gentlemen" and had them come over. I wasn't mad. My first thought is they are Army and no one may have simply explained that the ranks crossover for customs and courtesy. And after talking with them...I believe that was the case. They simply didn't know what to do. I had a pleasant conversation with 8 young men who were willing to do what 95% of the nation isn't. I thanked them for their service and we exchanged salutes. My closing comment was "Just be grateful a Sgt Major wasn't here.

Absolutely. That's an unacceptable example of a soldier not following proper MCC. He's an E4, he should know all this stuff by now. It ain't rocket surgery.