Developers develop when it's profitable. It's profitable right now to develop luxury housing because owners can be sure that they will make a shitload of money in rent. It's not as profitable to develop affordable housing, since the rent recouped is far lower.

Yes, we should build. And one of the arguments for building is that rich people will stop occupying otherwise affordable housing and move into luxury developments. But there are only so many rich people, and at a certain point, building luxury housing will no longer be profitable and the filtering effect will diminish. Without that incentive, developers will have to accept a far lower profit and build housing for normal people, and it's not clear that they will do so.

That's why we need a public developer unmoored from the profit motive. It's also why we should fight to make sure larger allocations of affordable housing are included in luxury developments, because that's a unit you know will be affordable, as opposed to a theoretical unit that might open up because a rich person moves into a luxury development.