What is the proper way to address NCOs in different branches?

I grew up in a Navy family. Dad was BM1 and spent his last seven years training recruits at Bainbridge and San Diego NTC. Things may have changed since the late 50s, but he would address anyone E-7 or higher as "Chief". He used the same form of address toward anyone likely to be Navy in civilian clothes, and I never saw a negative response to it. Perhaps sensitivity has changed, but no one seemed to mind the assumption that they were a chief petty officer.

I have worked with all 4 branches at one time or another. When in doubt, just ask the NCO. Grow a set and just ask them. We won't bite. But otherwise, by observation, I learned that the Navy Petty Officers I met went by last name unless they were E-7 and above. Then, it was Chief, Senior Chief, Master Chief, respectively. Marines were always full rank. Staff Sergeant, Gunnery Sergeant, etc. unless they liked you, then you could use gunny or such nicknames (but don't get creative or assume. Make sure they are okay with it first or you will get "sternly" lectured, lol). Army E-5 to E-8 is just Sergeant unless they were 1SG or a SGM/CSM. Then it was First Sergeant or Sergeant Major. Again, if the 1SG liked you, then you could call them Top (again, don't assume, get permission). Every Air Force Sergeant I met went by Sergeant. I sometimes heard Chief used for anyone above Chief Master Sergeant. This is solely from my own experiences but I hope it helps somebody perhaps.

I call him gunney.

I always addressed them by rank. NEVER by sir or ma'am. Those titles are for Officers. The only times an enlisted person is addressed as sir/ma'am is by recruits in basic training.

In the Marine Corps, recruits (because they are in bootbcamp) are required to call every Marine or Sailor Sir/Ma'am. Once a recruit has graduated and earned the title of US Marine, then they are allowed to address a person by their rank.

I usually, for the Navy, ask them what their rank is and how they are addressed. Recently I binge watched JAG. Between that and NCIS, I have a much better idea of what they are called. :)

ONLY THE BOOT ADRESSES THE D.I.AS SIR,AND ONLY IN BOOT CAMP.
REGARDLESS OF THE RANK OF THE NCO HE/SHE ARE ADRESSED AS THEIR RANK,PVT,CPL,SGT,ETC.ETC.ONLY OFFICERS ARE ADRESSED AS SIR/MAAM........
OR AS GUNNY OR SGT/MAJ.FIRST NAMES OFF BASE.....

I was always called "Petty Officer" by my subordinates and the chiefs in my department always referred to my rank before my name when talking to someone about me.
It is my title. The E6's above me were also called "Petty Officer" to me.

We only called petty officers Sir, or Mam in boot camp in the CG. I'm sure that's was the case when you wittnessed those Marine recruits.

Commission Officers these are Sir or Ma'am, Non-commission Officers these are NCO's. There is only three "3" ways to address an NCO, these are - Sergeant / 1st Sergeant and Sergeant Major.