There is research on the links between downward social mobility and happiness, however:
These empirical studies show little consensus when it comes to the consequences of intergenerational social mobility for SWB: while some authors suggest that upward mobility is beneficial for SWB (e.g. Nikolaev and Burns, 2014), others find no such relationship (e.g. Zang and de Graaf, 2016; Zhao et al., 2017). In a similar vein, some researchers suggest that downward mobility is negatively associated with SWB (e.g. Nikolaev and Burns, 2014), while others do not (e.g. Zang and de Graaf, 2016; Zhao et al., 2017)
This paper suggests that differences in culture may influence the connection between downward social mobility and happiness:
the United States is an archetypical example of a success-oriented society in which great emphasis is placed on individual accomplishments and achievement (Spence, 1985). The Scandinavian countries are characterized by more egalitarian values (Schwartz, 2006; Triandis, 1996, Triandis and Gelfand, 1998; see also Nelson and Shavitt, 1992)...A great cultural salience of success and achievement may make occupational success or failure more important markers for people’s SWB.
And they claim to find this:
In line with a previous study from Nikolaev and Burns (2014) we found that downward social mobility is indeed associated with lower SWB in the United States. This finding provides evidence for the “falling from grace hypothesis” which predicts that downward social mobility is harmful for people’s well-being. However, in Scandinavian Europe, no association between downward social mobility and SWB was found. This confirms our macro-level contextual hypothesis for downward social mobility: downward social mobility has greater consequences in the United States than in the Scandinavian countries.
This is, of course, just one study so not very conclusive.
In these sorts of discussions, I don't think comparing ourselves to the rest of the population is a great guide. It should probably be our base rate but many other factors can affect how income impacts our happiness.
If we look at the overall population the income level required to get the maximum benefit from consumption is pretty high. However, there is some evidence that for people who adopt voluntary simplicity can achieve greater life satisfaction on less income. Boujbel (2012) explanation for this is 'that the control of one’s consumption desires is a significant mediator of the relationship between voluntary simplicity and life satisfaction among consumers who have limited financial resources'.
So then the question is can you reduce your consumption desires if you start life with high consumption? I've seen a few people achieve this and I think I've reduced consumption desires over time as well. But this is pretty weak evidence to make broader inferences so I don't put too much weight on it.
I'd be much more interested in studying how income (specifically consumed rather than donated income) effects life satisfaction and value drift amongst EAs. I'd weight this much more than the general population for my own decision making. If I had to bet I would expect similar findings to Boujbel.