Quick question (and my 2 cents), if you were between the X17A and the X20E, did you consider the X19H? It's 'only' two feet longer than the X17A, but that does give you a bit more inside for not much more weight. We sleep 2 adults and 4 kids in it....
I won't get into the towing aspect, others chimed in on that. I'll just say that we tow our X19H with our Sienna. It has the same motor and same transmission as your Highlander. It tows just fine...
As an FYI, this guys setup makes me feel easier about towing 4000# with mine....
The Toads of Towed Haul - Home He's hauling 7500#!
Again...same motor...same transmission...same uni-body type framework.