When an E-4 calls me "hey" or "hey man" knowing that I am an officer, should I bother to correct him?

Absolutely.
IT IS THE MILITARY.

A complex situation. Normally you praise in public and go behind doors to punish.
In uniform this requires an immediate correction; on the spot, in public - by the time you’re an O-3 you should know how to “professionally” correct this behavior.
I played both softball and soccer on unit teams throughout my career. The teams I played on had teammates who were from E-2 to O-6; during the game it was usually first name. I don’t ever remember being told to call an O-6 Captain; but off the field and in uniform - proper military courtesies were followed.

Yes, you should always show respect to those appointed over you, but as one that holds rank higher than those under you, you should show respect to them as well as a valued member of your team/unit and not just some underling to order around. You'll find when you show their value to you in a positive way, the more they will respect you.

Depends if you were in uniform or in civvies. If he sees you as an Officer then he should show you the respect that you are due. If you are both out of uniform and off base Hey Man is OK, but you have the ability to acknowledge and respond accordingly. Respect is a two way street. If it is not given then you do not need to respond but if the SPC knows you then he should give the respect that you are due and you should call him out on it. ;-) I SALUTE All My Fellow Veterans!!!

Well I'm not sure how to respond to this
1) you are outside of work is one
2) You don't state if your in uniform
If you are he should he corrected
If not in uniform no because your are off of work which means no rank is involved unless you want to be an ass like some of the officers I've encountered and don't get me wrong my uncle is a retired Colonel and is the nicest guy around

Respectfully sir, yes. Correct him and subtly reinforce it while on duty should you come across him again. Customs and curtesy are there for discipline and this soldier has none apparently nor any respect for the rank you’ve attained.

I always say "hey sir/ma'am/sgt" to any Officer/NCO I know, at least in uniform. Out of uniform, depends on the environment and location I'm in, also if the Officer/NCO is out of uniform. But I would rather be corrected than let it slide because sometimes even I slip up and forget about the basics.

While I never was one to "jack someone up", I have on occasion quietly brought it to the attention of the individual that they might want to maintain military decorum outside the office as well. I, additionally tried my best not to refer to my senior NCOs by their first name, because it puts THEM at a disadvantage. Now in the retired world , I just say please call me "Drew".

ABSOLUTELY! Put him in the leaning rest first!

By outside of work, I assume you mean off duty and in street clothes. Which if that's the case, unless you can cite verbatim a regulation that requires him to address you by military title while in civvies, you should just let it be. If you are in uniform, then obviously you should correct him. But use some tact in doing so and lead by example in showing some decorum when redressing the situation. Otherwise, you just continue the epidemic in the military of do as I say not as I do leadership.