Sudharshan Seneviratne a leading Sri Lankan expert says that the rise and fall of Buddhism in India was linked to trade and fortunes of the mercantile community. According to him, Buddhism was an urban phenomena and it owes its spread to the movement of the merchants all over the India. So for example Buddhist sites came up in Andhra Pradesh and Tamil Nadu due to the expansion of trade to the Mediterranean.
“The Buddhist temple was not merely a place of worship. It brought into contact foreign merchants and local groups. It funnelled resources and endowed wealth for investment by acting as a bank where merchants and guilds deposited money. The temples used to lend money for interest and the interest was used to maintain the temples,” he said.[Buddhism linked with traders in South India: Lankan scholar]
He also mentions that Buddhism was a movement against brahminical hegemony but later, around 4 AD, there was a revival of Brahminism and he claims that it was because kings started seeking the Kshatriya status and that could only be conferred by Brahmins.
But many scholars do not agree with this black and white distinction between Buddhism and Brahminism as enemies.
The notion of continious rebirths and the challenge of escaping from their endless cycle were common to both orthodox teachings derived from the Upanishads and to the Buddha’s teaching. Buddhism was not a belief system, not a rival faith to the post-Vedic cults and practices which prevailed under brahminical direction, but more a complementary discipline[John Keay, India, a History]
Abraham Eraly in his book The Gem in the Lotus writes that the early disciples of the Buddha were people from the upper crust of the urban society such as Brahmins, Kshatriyas and wealthy merchants. Even though Buddhism did not recognize any status claimed by birth, he never challenged the caste system. Eraly says that Buddhism was not a movement against the established order, but instead its concerns were of a different plane altogether.
Buddha was against Brahminism in certain ways. For one the Buddha never could accept the fact that people could claim sanctity because of birth and because they could recite the Vedas. He also could not accept their blind belief in the Vedas while he advocated on experiencing the truth individually. Buddhisms insistence on non-ritualistic sect, definitely affected the livelihood of Brahmins, but it never resulted in any serious confrontation between them. Buddhism never considered Brahmins as their real opponents, but their rivalry was more with other sectarians like Jains and Ajivikas.