are you trying to use FP16 ? because in most GPUs it is same or faster than FP32, FP64 is much slower

ex: RTX 3090 (https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622)

FP16 (half) performance 35.58 TFLOPS (1:1)
FP32 (float) performance 35.58 TFLOPS
FP64 (double) performance 556.0 GFLOPS (1:64)