Limiting Call Notifications

Could you add feature request where we would be able to limit call notifications to a maximum of [N] at a time?
For example, if more than 10 checks go down (e.g., due to a network outage causing 100 checks to fail), I would like calls to be sent only for 10 of those checks, not all of them.