I think a lot of the confusion comes from people not understanding the definitions of what they are talking about.
A ponzi scheme is "a form of fraud in which belief in the success of a nonexistent enterprise is fostered by the payment of quick returns to the first investors from money invested by later investors."
In other words, there is no product. The returns to the first investors are paid by new investors. This scheme can be very successful because it can sustain itself until new investors dry up. By then, the initial investors have already made their money and they disappear. A good example of this would be Bitconnect. They claim that their product is some sort of trading bot but there is no evidence of this. If there was one, it would be a legitimate business. But because there (likely) isn't one, it is simply a ponzi scheme.
There are also "Pyramid Schemes" which are defined as "a form of investment in which each paying participant recruits two further participants, with returns being given to early participants using money contributed by later ones."
In a pyramid scheme, you pay the person that recruited you, and anyone you recruit pays you. Very similar to a ponzi scheme but slightly different. In a pyramid scheme you know that you only get paid if you recruit new people where in a ponzi scheme you think that there is some sort of investment going on that is generating your returns.
Referral programs are somewhat similar to pyramid schemes except that there is actually a product and (as you pointed out) the company/organization would still continue to function without new investors/customers. (Of course, in today's business world if you are not gaining new customers you are in trouble because you are always losing customers but that is different than a company that must continually grow to function ie ponzi scheme)
Hopefully that helps clear up a bit of confusion. We should always know the definitions of the words we use.