Viewing a single comment thread. View all comments

puppet_pals t1_j0kjvqn wrote

Reply to comment by Ok-Teacher-22 in [R] Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow by Ok-Teacher-22

is there a bug report for this? definitely file one if there is not.

​

I encountered a similar silent failure years ago where the gradient some operation on a complex # was 0. https://lukewood.xyz/blog/complex-deep-learning

2

Ok-Teacher-22 OP t1_j0mwqk7 wrote

there have been many filed. just go search and stop being lazy

−4

puppet_pals t1_j0p5ai3 wrote

> stop being lazy

please keep in mind that people on reddit are usually browsing in their free time and might be on mobile.

---

I dug into this for you...

The issue is that the complex numbers are casted to the default data type of each individual metric, which is usually floats. This is consistent with the behavior of all Keras components. Each component has a `compute_dtype` attribute, which all inputs and outputs are casted to. This allows for mixed precision computation.

Complex numbers are a weird case. The complex numbers get casted to the metric's native dtype, which is float by default, causing the imaginary components to be dropped. For most dtypes theres a logical translation from one to another; i.e. 1->1.0, 2->2.0, etc. There is not one of these to go from complex->float.

In my opinion TensorFlow should raise an error when you cast complex->float, but this is not the case in TensorFlow. I have a strong feeling that we can't change this due to backwards compat, but would have to dig deeper to verify this.

In short this is not really a Keras bug but is rather a weird interaction between Keras' mixed precision support and TensorFlow.

I hope this helps - maybe we can make a push to raise an error when casting from complex->real numbers and force users to call another function ? (i.e tf.real()). I don't know what the "Right" solution is here, but that is the history of why this issue exists.

2