If you are paying for the API something like "RWKV" might be an alternative hosted on a GPU Cloud provider. The model is currently only at 14B parameters but technically has "unlimited context" which in theory is probably not actually unlimited but for what I saw in your use case it might be worth looking into