R. Aizel Charif of Slonim was a very colorful gadol, who lived in Lita around a century and a half ago. He was born into a Hasidic family (his father was a Chasid of the Alter Rebbe of Habad), but became a Misnaged, an opponent of Hasidism (that is a good subject for a possible future posting, Chasidim who left Chasidus and became Misnagdic gedolim, with Hashem's help).
A number of years ago, a descendant of his put out an interesting book about him in English, called "The Modest Genius: Reb Aisel Harif", portions of which can be seen via google books. One chapter of it is about Chasidim and Misnagdim (Chasidim and their opponents, the Misnagdim), and has some interesting stories, which display Reb Ayzel's sharpness. One of them (p.117-118) tells that Reb Aizel was once rebuked by a Chasid for not wearing a gartel. He responded that the pants belt he was wearing already performed the separation between the upper and lower parts of the body. The Chasid, however, kept bothering him about it, which led him to remark that if a sefer Torah is kosher, the gartel is under the mantel (coat, cover, jacket), whereas if the gartel is over the mantel, it is a siman, a sign that the sefer Torah is not kosher. והמבין יבין.
Another issue with wearing gartels that I notice is, especially in some cases, that the way the gartel is worn accentuates the shape of the body in an immodest way, which doesn't seem like an appropriate way with which to approach Hashem.
There are also other aspects of why Litvaks (generally) don't wear gartels. The idea of a gartel is a separation between the higher and lower parts of the body. In ancient times, clothing styles were different than today. If someone wore a robe like garment, they might not have such a separation between upper and lower regions. However, later on, prevalent clothing styles changed, and they already incorporated separation between those areas.
