Most of mathematics, if not all of it, reduces to logic. This is especially clear in Set Theory, Group Theory, Topology, and Geometry as these theories have their foundations in the study of logic.
Ancient philosophers such as Plato and Aristotle clearly saw parallels between the material world and world of ideas. If there is a rational structure behind reality, then it might be possible to speak about it in rational terms.
Some thinkers would refer to things like "the mind of God" when they expressed the opinion that there might be rational structures behind reality.
The possibility that there might be a rational foundation to the universe has been the subject of speculation throughout history.