This paper attempts to demonstrate that the conviction about the harmony and order of the world was a fundamental metaphysical principle of the Pythagoreans. This harmony and order were primarily sought in the structures of arithmetics, yet following the discovery of incommensurable magnitudes (irrational numbers, as we now call them), the Pythagoreans began to see geometrical structure as a fundamental part of the world. On the example of the Pythagoreans’ metaphysics and science, the paper shows the mutual relations between metaphysics and science. It demonstrates— on the one hand—the necessity of the first as a guide for the latter, and—on the other—how our scientific research can change its basic metaphysical principles when these are found to be inappropriate. The paper also tries to show the need for a realistic approach in our scientific research by means of the same example of the Pythagoreans, that is, the need to discern something which is below the surface appearance.