It'll be safe, but...
I towed our X20E for one season with an '03 Trailblazer EXT, I6, 3.42 gears. My towing capacity was 4900#, the trailer's GVWR is 4750#. So I was close to my capacity. It worked, but it struggled on hills and in the wind. Mine had 165k miles on it and was going to need an exhaust and a couple of other things so it became a no-brainer to upgrade. My Sierra tows our X20E like they were made for each other. I don't miss the TB in the slightest.
A couple of other things you haven't considered.
The weight on the yellow sticker is not the weight you will be towing. When you add your "stuff", it will at least 500# more, probably higher than that. the GVWR of that trailer is 5500#. If you max out the weight of the trailer, you will be several hundred pounds over you rated max.
Ignore the weight sticker on the hitch. That's the max of the hitch itself, go by the manual (Tongue wt 400lbs / 912 with weight dist hitch). Also ignore the 400#. That's the max dead weight. You will not be towing this thing without a WD hitch. The suspension on GMT 360/370s is too soft to tow a trailer this size without one.
Plan your tongue weight to be 13% of your trailer weight. Just assume 5000# ready to camp, that's 650# of tongue weight. So that's not an issue.
However the other thing you need to look at is payload. Look at the sticker in the door, it'll tell you your payload. IIRC, mine was something like 1200#. Sounds good? Subtract the 650# tongue weight, and say you have a wife, 2 kids, and a dog, total weight 225#. Your leftover payload is 325#. Oh wait, we forgot to add in the grill and the bikes! You can see how we run out of payload real fast. That's why a lot of people go into full size trucks, more payload. My Sierra has a 1621# payload and now if we want to add "stuff", we don't even think about it.
I can tell you that you'll probably be OK on short trips (2-3 hours) over relatively flat terrain. Don't even think about mountains. You'll be thinking about a bigger truck by next season.