I have that exact same problem with pants. It’s not so much my waist as it is my thighs.