Léon Rosenfeld is maybe mostly known as devoted adherent and fierce defender of Niels Bohr’s interpretation of Quantum mechanics. But he also had a past, where he pioneered in a remarkable way in the area of quantization of gravitation, while he later distinctly criticized this program altogether. In this talk we want to examine how this all goes together and in particular outline Rosenfeld's distinct view on the usage of mathematical methods in physical theory development.