You can immediately:
- Block authentication attempts are made from specific network ranges, like the ones from where the attack is being sourced.
- Restrict what users can access the affected application.
- Disable the application service principal in your tenant.
Depending on your specific scenario and licensing, more robust solutions or actions could be taken. For more information take a look at: