There's actually not much oxygen in the water. That's why fish are moving even when they are sleeping, they have to constantly keep fresh oxygenated water moving over their gills or they would suffocate. Not enough to keep a fire going either.

Oxygen is not the only reason for a flame. Alternatively you could have any oxidizing agent (which most commonly is oxygen but not necessarily only one) that could produce heat and light. Thats all fire is. A reaction that produces heat and light. Another challenge for an underwater fire should be elevating and sustaining the high temperatures. The reaction should produce light and self sustain heat at lower temperatures. For example, IIRC magnesium burns and sustains fire underwater or any conditions because it can react with nitrogen, water and oxygen as well and have flames in each of these environments.