Purpose This study aimed to assess the activity of A. austriaca flowers in treating polycystic ovary syndrome (PCOS) in rats. Methods A letrozole-induced PCOS rat model was used to evaluate the activity potential of A. austriaca flowers. For this purpose, extracts of different polarity were prepared from A. austriaca flowers using n-hexane, ethyl acetate, and methanol. Serum luteinizing hormone, follicle-stimulating hormone, progesterone, testosterone, estradiol, serum leptin, lipid, and glucose levels were tested. Moreover, the antioxidant activity was evaluated by calculating superoxide dismutase, malondialdehyde, catalase, and glutathione peroxidase levels. Following the biological activity studies, phytochemical studies were conducted on the active extract to detect the compound(s) responsible for the activity. Results The treatment with n-hexane extract contributed to regulating serum gonadotropin and steroid hormone levels. The plasma level of high-density lipoprotein cholesterol was significantly higher than that of the control group, while the levels of low-density lipoprotein cholesterol, leptin, and glucose were significantly lower than those of the control group. Also, the n-hexane extract showed significant antioxidant activity in the PCOS rat model. Since the n-hexane extract was found to be active, isolation studies were performed on this extract and three main fractions were obtained from the n-hexane extract. Those fractions also were tested on letrozole-induced PCOS rat model. As a result, three triterpenoids, beta-amyrin palmitate, taraxasterol acetate, and taraxasterol were isolated and identified from Fr. B which is the most active fraction. Conclusion n-Hexane extract and Fr. B obtained from this extract showed statistically significant activity in the letrozole-induced PCOS rat model and three triterpene-type compounds were isolated from Fr. B.