Military working directly with GS Civilians / Contractors... Thoughts and or experiences?

I do work directly with them and only them.  The only other "green suitor" in my section is a Sergeant Major, whom is the Deputy Director of the section (DOET or glorified S-6) and also my first line supervisor.  When I first started the position, I was the only military member of the section and found it extremely difficult to develop a good working rapport with them.  I believe some of it was me ,and how my previous duty assignment impacted the style of work and communications I tried to employ with them.  I also believe that their "laid back" mentalities and lack of solid work ethics had an impact as well.  These issues obviously created a great deal of tension in the room, almost to a point of fist-a-cuffs. The tension has since died down and we are mostly at an understanding with each other now, even more so with the arrival of the above mentioned SGM.  Still I cannot help but always feel as if there will always be sort of a rift between myself and them.