Peter Singer's 1980 paper Utilitarianism and Vegetarianism might answer your concerns. Here's a relevant quote from the conclusion of his paper:"I advocate vegetarianism as something which "underpins, makes consistent, and gives meaning to all our other activities on behalf of animals" (Animal Liberation, p. I71). I remain convinced that for those concerned to change the situation of animals in our society, vegetarianism is of real practical importance. It provides an irrefutable answer to the oft-repeated claim that we need factory farms to feed our growing population. It allows the animal welfare campaigner to defeat ad hominem attacks, for instance: 'How can you object to killing seals when you eat pigs and calves?' By eliminating one's personal involvement in the production of animals for food, it makes it easier to take a detached view of the animal industry, and to avoid compromising the interests of the animals with one's own interest as a consumer of animals. Calling on the public not to buy the produce of factory farms can be an important part of a campaign against factory farming. It holds out a threatening prospect to farmers-one which is beginning to be noticed in farming magazines-and it enables those who support the campaign against factory farming to make a personal commitment which goes beyond signing petitions and writing letters to their elected representatives. One cannot convincingly ask others to do this if one does not do it oneself. (Unless one eats animal flesh in secret-which hardly seems worth the hypocrisy and risk of discovery involved.)
Finally, becoming a vegetarian is a way of attesting to the depth and sincerity of one's belief in the wrongness of what we are doing to animals. Perhaps in a society of sophisticated philosophers there would be no need to attest to one's sincerity in this way, because sophisticated philosophers would understand that one can sincerely oppose the exploitation of animals in factory farms while continuing to buy and enjoy the product of these very farms. But to most of the members of our society this would mean, as it seemed to Oliver Goldsmith's fictitious Chinese traveler, a "strange contrariety of conduct."