How old it the belt? If it's old you may want to change it out. Also if it was over tightened it may have snapped the threads in the belt and it only held together with the rubber not letting it tighten properly and causing it to stretch after readjusting it. So, you may want to try a new belt. If you can find a Gator Back belt you will be better off. Let us know how it turns out.

 

I replaced it a few months ago with a NAPA belt. The belt looks just fine.

It seems like tons of cherokees have this issue, I figure there has to be something that can be done to fix it 100%.

I'll find a gatorback next.

 

The problem is it may look fine but the treads that are in the belt may be broken and you won't be able to see if they are bad, Unfortunately the only way is to try a new one.

They do. The bad thing is that the belt has to be really tight to keep them from making a lot of noise.

 

Often overlooked is 'glazing' on the belt or the pullys. This will cause squeal on a properly tensioned belt. Look for any build up on the idler pulley.
I like to wire brush all the pulleys when changing a belt or dealing with squeal.