The torque due to gravity is:
However, when the center of gravity does not coincide with the center of mass, this is not true.
An example of when this happens is when a pole is tilted to the side. The center of mass is at the center, but one side of the pole is closer to the earth, and therefore more gravitational force is exerted on it. The center of gravity is shifted towards it.
However, at our scales, this difference is negligible.