Anarchism, properly understood, is just the consistent application of common sense moral principles. Most anarchists base their beliefs on the Non-Aggression Principle, which states that no one has the right to initiate force against another. This, in turn, is based on the principles of self-ownership and inviolable private property rights. So to initiate force against someone's person (murder, assault, rape, kidnapping, slavery) or property (theft or fraud) is ethically wrong. This does not preclude the use of defensive force. Most people recognize these principles in their daily lives, but the anarchist recognizes that the institution known as the State must, by definition, violate this Non-Aggression Principle. It must make use of taxation, which is the taking of private property by force or the threat thereof (theft), and holds a monopoly of ultimate decision making in a territorial area (which necessitates violations of the right of property owners to decide their own rules on their own property). The reason that people often think that anarchy=chaos is because they conflate the State with society. You're right in saying that human kind has already chosen order over chaos. Widely-accepted norms such as those outlined by the Non-Aggression Principle are recognized as valid regardless of legislation; that's why we can differentiate between good law and bad law. State legislation is not needed for this discernment of justice within a society.
Many serious thinkers throughout history have been anarchists. Modern anarchism of the libertarian strain, such as I'm referring to here, and in contrast to former brands of socialist anarchism, is simply an extension of western classical liberalism. If you want to learn more, there's a wealth of information at Mises.org and the Mises YouTube channel. It's not just revolutionary-minded kids in their mother's basements. Anarchism as a philosophy has a long and very interesting history.
In other words, anarchism is primarily concerned with the use of force in society. The State is an institution that relies on the initiation of force, and so is seen as illegitimate. Anarchism does not reject social organization or leadership. It only insists that such relationships be voluntary and not coercive.