When an E-4 calls me "hey" or "hey man" knowing that I am an officer, should I bother to correct him?

If you were in uniform he should have been lit up. Insubordination is insubordination. In or out of uniform respect should always be maintained. My uncle was a COL, my father was a 1SG. In uniform it was always COL B and Top; out of uniform uncle or dad. You have to remember where you are at all times and show respect.

Don't let "Top" correct the Specialist....'cuz, then "Top" is going to rip you a "a new one" !
Know what I mean. Be decisive, Sir !
Not "Wishy-Washy".

Tolerance leads to acceptance and that acceptance sets new standards

meany years Ago i saw how Colin Powell handled it when 2 young men gave him the Black Power sign

IT ALL DEPENDS ON THE "SETTING" AND THE RELATIONSHIP BETWEEN THE INVOLVED PARTIES. THIS WORLD "NEEDS" MORE "FRIENDSHIPS", THAN "SUB-ORDINATES".

Something my grandfather taught me... as well as a 1SG... Sir doesn't always apply to rank.
Sir is a term of respect given to your fellow ... hu ... man.
Context is everything.
"outside work" could be a party, a bar... anywhere...
MY way is to always show proper respect to commissioned officer... unless told otherwise.
I.E. when at the Berchtesgaden Training center in Germany, we were told to not use rank while in civilian areas.
Everyone knew we were military, no sense pointing out who was in charge ;)
But ... as Top said: "when in doubt, whip it out".
We're all people working as a team, sometimes it's okay to 'let your hair down', in public...
In moderation... it's good for moral for us to just be human out of uniform. ;)
Never forget that Rank goes with you, no matter where you are, and represents the discipline of the Forces civilians see as their protectors.
Try to not be a bad example?

Gentle correction is appropriate. Ensure the NCO chain is advised. Let the NCOs deal with the remediation/adjustment , if needed.

"outside work"!? The E4 needs to be shown a new level of attention, along with its attendant discomfort level.

You are an officer in the United States army sir no sir not hey man

LT, I need to pick myself from the deck. Not too sure how things go in your organization, but I can assure you that if that E-4 were observed by a Chief Petty Officer that E-4's life would become a bucket of s--t. Who do they think they are, your drinking buddy? Somebody needs to lock their heel and square them away. When I was a Seaman Nobody if I had done something like that the 1st Class Boatswain would have taken me to the Bo'sun Hole for an education. The next time some Yahoo does that to you I recommend you lock their heels and put the fear of God in them. This Woke stuff is getting out of hand.