Can an MP carry their firearm off duty in any state?

If not under orders, then you become a simple civilian like everyone else and you need to comply with local weapons carry laws. In Florida, course attendance is waive if you provide weapons training documentation. Still need fingerprints and the background check.

Even USCG petty and commissioned officers, who are federal law enforcement officers, cannot use such authority while off duty. I once had to get one of our sailors out of the Bronx jail for carrying a concealed private handgun while off duty. Plea bargained a felony down to a misdemeanor with probation and no jail time.

I have my LEOSA ID, and use it to carry in CT and RI, and I am in the process of getting my CT Card. Funny thing is, if your are a CT resident, you cannot use your retired LEOSA ID to buy a weapon. Only an active duty LEOSA ID can be used. I need to get the CT license to buy. (And it costs $140 just to get the temporary license - you need to pay to exercise your right in this fascist state.)