It is not fully understood how cooperation emerges in a population of individuals with no connections or prior experience with each other. Strategy selection that is purely based on accumulated payoffs promotes free riders who put their self interests above that of any group. How could cooperation persist in these settings? Researchers have posited direct or indirect reciprocity as possible explanations but these theories fail if interactions are not repeated or reputations are ignored. Altruistic punishment may provide an answer. Punishers impose a penalty, such as a fine, on defectors. The idea is a sufficiently high enough penalty—or even the threat of a high penalty—will convince defectors that cooperation is more beneficial. The punishment is altruistic because punishers pay a cost to impose a penalty and expect nothing in return (including no future reciprocity). Empirical and human studies have shown when some individuals are punishers, and they are common, cooperation levels tend to increase. But it has never been specified exactly how large this majority of punishers must be to promote cooperation. Here we analyze cooperation, defection and punishment in a social dilemma, public goods game, and precisely identify the necessary conditions to make altruistic punishment an effective strategy for improving group cooperation levels.