question

GoelAkanksha-9319 avatar image
0 Votes"
GoelAkanksha-9319 asked PRADEEPCHEEKATLA-MSFT commented

Not able to limit the size of STDERR and STDOUT logs for Azure HDInsight deployed Spark Application!!!

I have tried editing the log4j properties file in

https://CLUSTERNAME.azurehdinsight.net/#/main/services/SPARK2/configs,

as mentioned in the following link:

https://docs.microsoft.com/en-us/azure/hdinsight/spark/troubleshoot-debug-wasb

89494-image.png
I have done the following additions as shown in screenshot:


log4j.appender.audit.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.audit.triggeringPolicy.maxFileSize=50MB

![89416-image.png][2]
But, the changes are not working, what can I do to limit the size of logs as you can see the stderr log size is increasing


azure-hdinsight
image.png (438.2 KiB)
image.png (68.6 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @GoelAkanksha-9319,

Welcome to the Microsoft Q&A platform.

We are reaching out to the internal team to get more help on this, I will update you once we hear back from them.

1 Vote 1 ·

Hello @GoelAkanksha-9319,

We are still awaiting for the response from the product team. I will update you once we hear back from them.

Thank you for your patience.

1 Vote 1 ·

Hello @GoelAkanksha-9319,

Apologize for the delay in response.

Unfortunately, I haven't got any response from the product team.

For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket, else please do let us know.

0 Votes 0 ·
Show more comments

0 Answers