This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hi All
We are running an ESP-enabled HDInsight Spark cluster in Azure. We have no clue why some of our domain users are not able to use Zeppelin notebooks (usint the pyspark interpreter in our case). This is the very simple code that results in errors:
%livy2.pyspark
x = 2
print x
We have followed and tried the advices explained here without success: https://community.cloudera.com/t5/Support-Questions/Error-when-running-a-Spark-job-as-a-different-zeppelin-user/td-p/191846
I have attached the log files from Yarn and Zeppelin below. For some reason whatsoever, no Yarn application is created.
The output from command hdfs dfs -ls /user/ (for the not-working user) on the cluster's head node returns:
hdfs@hn0-upcch:~$ hdfs dfs -ls /user/
drwxrwxrwx+ - sdptest-cloud 00000000-0000-0000-0000-000000000000 0 2020-08-12 08:57 /user/sdptest-cloud
Any help / pointers / references about what else we could try are highly appreciated.
Christoph
17232-yarn-yarn-resourcemanager-hn1-upcchlog.txt
17038-zeppelin-interpreter-livy2-livy-zeppelin-hn0-upcch.txt
17186-zeppelin-zeppelin-hn0-upcchlog.txt
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Hi @Christoph Kiefer ,
Thank you for reaching out. The users who are having this issue, were they able to run a simple query in spark shell? And I see that you have mentioned it results in errors when a simple query is ran in notebook, could you please share the error message that was returned to the notebook? There could be many causes for this, check the shiro.ini file to confirm they're in the groups listed. I think a support request would be the best route for this issue. If you have a support plan, I would recommend to file a support request for deeper investigation. In case if you don't have a support plan, please email me below details at AzCommunity[@]microsoft[dot]com.
Please let me know here once the details are shared.
Thank you
Hi @Christoph Kiefer ,
Following up to see if you have got a chance review my previous comment. In case if you are still experiencing the issue and if you don't have a support plan, please do share the details requested above, in case if you have already filed a support request, please share the SR# with us so that we can track the issue internally.
Thank you
Hi @Christoph Kiefer , Following up to see if you still need assistance on this query. In case if you have already found a solution, please do consider to share it here with community, as it would be helpful for others reading this thread.
Thanks
Thx for the reply and sorry for the long delay. It acutally worked when I implemented the changes mentioned here: https://community.cloudera.com/t5/Support-Questions/livy2-zepplin-issue/td-p/220980
However, I am not sure if turning of impersonation is a good idea ...
Hi @KranthiPakala-MSFT ,
Glad to know that your issue was resolved and thanks for sharing your findings as it would be helpful for other community members reading this thread.
Please feel free to accept your answer so that other readers with similar issue can find your solution quickly.
Thank you & Stay safe.