Unfortunately the answer is that it depends. IMO 4k bluray is always worth it when it is the most accurate reproduction of the film at the time of release. Part of this is because it has a MUCH higher bitrate, so you'll be getting more of the original picture out of it. If you have a high end HDR setup, I also think thats worth it. Normal blurays have to fit into the rec709 signal standard that only gives you around 5 stops of latitude/dynamic range. An HDR version can have 12+ stops of dynamic range and most closely approximate the film print or theatrical DCP.

Now, where it gets wonky is that some blurays (1080 and 4k), especially ones made further away from the original release, sometimes the color grade gets "updated." The new Hurt Locker "master" is now more green than its first release. The new edition of LOTR movies have been re-graded and de-noised to look more modern. Granted, in that case it was the director making the changes, but it is a certifiably different experience than what you got originally. At least with blurays released closest to the theatrical release, they tend to be what we call a "trim pass", aka the original color off the print or DCP is being as faithfully reproduced as possible in the 5 stop rec709 color space. The only time you dont have to worry about this is if its a Criterion release, they're pretty nuts about preserving the original experience.

That all being said, 1080 blurays use to be (and I think still are) 8 bit color depth, where as 4k is 10bit (or is at least 10bit capable). its the different between being able to encode millions of colors vs billions of colors.