Okay, certain guys have moved easily into trousers and sport jackets and they look great, but others can't quite pull it off. What's your advice?

Go to a professional. It's not the case that only certain guys can wear a sport jacket and trousers; it's just more likely that the rest of us need some guidance. Go to a specialty shop selling men's clothing and meet with a salesman whose style you like. He deals with fit and fabric every day. Be open to change, listen to his advice.