I have an X-terra; great trucks. We used to tow our boat with ours and it did great. One thing to consider is that the factory rear suspension is VERY soft. Great for ride comfort, terrible for payload control, we hit our bumpstops all the time. This can be a significant contributor to sway in the trailer while towing.
So I agree that you should have your setup weighed, and I agree that a sway control device is good, cheap insurance. But I would add that you should consider upgrading your rear suspension as well. I've seen it done with everything from helper springs to entirely new leaf spring packs, and air bags are popular. I was going to change out my leaf pack (can't remember which one I was going to get, and can't find the link right now) and replace the rear shocks with Bilsteins. I decided against it because we pretty much use the X for commuting and I want to do some upgrades to my truck (TV) instead.
Check out this forum for pointers on different options: http://www.thenewx.org/
I'm assuming you're using a 2nd generation X (model year 2005 +) as you don't see a lot of the 1st gens doing any heavy lifting...
2014 Jay Flight 28 BHBE
2015 RAM 2500 6.4L HEMI, Tradesman 4x4, 3.73
Blue Ox SwayPro (BXW 1503)
Upgraded from an REI internal frame backpack and a Eureka 1/2 dome tent!