When an E-4 calls me "hey" or "hey man" knowing that I am an officer, should I bother to correct him?

That would be CORRECT SIR ! YES SIR

Sir as a Commissioned Officer it is your right to correct an Individual that blatantly shows disrespect for you.

I cannot even begin to conceive of what would have happened to me in the way, way back, if I had spoken to an officer with "hey man"....

Y’all are more worried about your ego than the standards. Don’t get it confused.

Never walk past a mistake

Yes, I believe you should correct him. Your name is not, "Hey Man."

That is a no-brainer in all branches of service. Remind the E-4 that it shows respect for the person or his /her position. It is not proto-cal to call them anything, other than that. Even in civilian attire, I did that.

Yes.

No, especially since you don't work with him. People need a break from the caste system when they're not at work.

Lack of respect!