Best solution to capture all requests/responses of Asp.Net Web API hosted on Azure

Question

question

ChauTieu-6500 asked Jan 13, '22 AnuragSingh-MSFT edited Apr 28, '22

Best solution to capture all requests/responses of Asp.Net Web API hosted on Azure

My system is an Asp.Net Web API (.Net 5) hosted on Azure kubernetes service. I have a requirement to log all requests/responses for all controllers of the system. Currently, the system logs only exceptions to Application Insights using Middleware and the default logger injected by the framework.

My questions are:

Should I develop the new requirement as the current implementation of logging exceptions by adding an additional middleware to log all requests/responses to Application Insights?
Are there any services/configurations on Azure to log all requests/responses so that I don't need to modify the code?
What are the best practices for that kind of requirement? Is this ok to save all requests/responses into Application Insights to easily query/trace/investigate issues later or it should be somewhere else such as databases. etc. since the requests/responses may have sensitive info?

azure-monitor

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ChauTieu-6500 · Jan 15 at 06:55 AM

Does anyone have any idea? Thanks.

0 ·

Answer 1 · 2022-01-17T08:49:28.94Z

AnuragSingh-MSFT answered Jan 17, '22 AnuragSingh-MSFT edited Jan 17, '22

Hi @ChauTieu-6500

Welcome to Microsoft Q&A! Apologies for the delayed response.

Application Insights is an extensible Application Performance Management (APM) service. This is for mainly instrumentation (measure of performance) purpose. Here is a list of all the scenarios monitored by Application Insights: What does Application Insights Monitor?.
Generally, the record of **all* request and response are monitored from client side for diagnostics and troubleshooting purpose as mentioned in this article: HTTP trace from client.

In view of the points above, please find below my response to your queries:

1. Should I develop the new requirement as the current implementation of logging exceptions by adding an additional middleware to log all requests/responses to Application Insights?
It depends on your requirement. If it is absolutely required to record all the requests and responses on the server side, you will have to develop custom solution. From Application Insights standpoint, you may use Trace logs to record these details

2. Are there any services/configurations on Azure to log all requests/responses so that I don't need to modify the code?
No

3. What are the best practices for that kind of requirement? Is this ok to save all requests/responses into Application Insights to easily query/trace/investigate issues later or it should be somewhere else such as databases. etc. since the requests/responses may have sensitive info?
The choice of service will be up to you. You may use Application Insights trace logger to store it in AppInsights Workspace. Since this data is going to be unstructured, you may also use Azure storage OR Azure Cosmos DB. All the service mentioned have well defined .NET APIs that can help you achieve the goal

Please note, recording all the requests/responses from all the controllers will negatively impact the performance of your application. I would also like to understand the need for such a requirement. A stateful application with security as required should be able to provide you with most of the information required (For example - who performed what?)

Please let me know if you have any questions.

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ChauTieu-6500 · Jan 17 at 08:54 AM

Thank you for the answer! The purpose of tracking requests/responses is just for later ease of investigation. We may apply for some controllers not all. For sensitive data, we have decided to mask these fields before writing to trace logs.

0 ·

question