The connection between beauty and truth has a tradition reaching back to ancient times, but in the last decades it has featured increasingly often as a motivation for pursuing certain research programs in High Energy Physics for which an empirical confirmation is still lacking, such as Supersymmetry or String Theory. Unsurprisingly, this trend has been questioned by philosophers and harshly criticized by physicists following alternative research lines. In my paper I am not interested in criticizing or upholding the beauty-equal-truth argument, but rather in contextualizing it, showing how it is not a singular deplorable departure from alleged scientific values of rationality and objectivity, but that, just like those values, it is a historically and culturally situated feature of the modern scientific worldview. More specifically, it is one of the many verbal or visual modes of expression that modern science shares with modern Western religion, and that can be traced back to their common heritage from nineteenth-century Romantic culture. From this point of view, uses of religious wordings or imagery in science should not be regarded as superficial, perhaps misguided borrowings, as they have an essential role in the construction of scientific knowledge and its perceived validity. I will discuss this topic using as an example the emergence during the second half of the twentieth century of a notion of symmetry which gained prominence not only in the popular press, but first and foremost in scientific discourse, and which combines a technical definition as mathematical invariance with a more qualitative characterization as an inner principle of nature.