My default answer is that all supplier monitoring, including product audits, should be performed at a frequency determined by risk-based thinking. You need to consider things like the criticality of the product (is there an effect on human safety?), the severity of an escape, the likelihood of a quality issue, the detectability of those issues, as well as historical performance of your supplier. If you have data that one or more failure modes tend to occur with some frequency, you might decide to migrate your attention from detection to prevention. Depending on the failure modes in question, there are several options. In my company, repeat or critical to quality characteristics often switch on reporting requirements of the supplier. Those characteristics are required by contract to be inspected and reported by the supplier. If failures continue to occur, engage the supplier and work through what is now clearly an inspection challenge.
Hope this all helps. The question was pretty general, so I may have gone down entirely the wrong rabbit hole for you. Remember that product audits don't need to review all characteristics, just those deemed CTQ or that show repeat failures. Good luck!