Hi @Jefin Mathew
I have reproduced your issue on my PC. The first click could always succeed but after that it will need double click or refresh the browser. This issue seems to be by design and I could not find any official document to explain it. So I would suggest you to send feedback to Microsoft SharePoint Feedback.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.