By default, yes, budgets can be exceeded. The analytics for a budget will indicate that a budget has been exceeded, and by how much.
However, to prevent users from exceeding a budget:
Whenever a transaction is initiated that would surpass the limit for that budget while the capped spending option is enabled, that transaction will not be processed, and a notification will be provided to indicate that one or more budgets are being exceeded.
Tags: