If their family usually speaks Spanish in their home, they will continue to do so unless you're specifically in the conversation. That's just how it is.

When I visit my wife's family, it's the same way with everyone speaking Vietnamese even though I only know a small number of words. When they want to include me in the conversation, they switch to English, but they are used to speaking Vietnamese together so they switch back as soon as possible.