Start task fails when setting up batch node for ADF python batch script activity

Shannon Moran 96

Hello - Trying to set up data factoring executing Python via batch activity and the start task failing on node.
When setting up the pool/node, I need to install Python as well the packages used by the script. I have included the following in start up command:

cmd /c “python-3.9.6-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 Include_test=0” & "pip install azure-storage-blob pandas json delighted”

I receive start task failure error and not sure how to accomplish both.

Thanks in advance for your help

PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee

2021-07-26T09:33:59.147+00:00

Hello @Shannon Moran ,

Welcome to the Microsoft Q&A paltform.

Can you please go to Tasks under your Batch pool and check the stderr.txt file if you are getting any error logged in that file?

Meanwhile, you can checkout the MS Q&A thread, which addressing similar issue.
PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee

2021-07-27T10:57:59.817+00:00

Hello @Shannon Moran ,

Just checking in if you have had a chance to see the previous response. We need the following information to understand/investigate this issue further.
Shannon Moran 96 Reputation points

2021-07-27T12:41:07.51+00:00

Apologies for delayed response. I am working on a trial account and believe that there are limitations in number of attempts/pools that can be created and deleted. See this message when trying to reproduce the issue to provide the stderr.txt file.

Code: AccountVMSeriesCoreQuotaReached

Message:
The specified account has reached VM series core quota for standardFSv2Family

this is the tutorial:
https://learn.microsoft.com/en-us/azure/batch/tutorial-run-python-batch-azure-data-factory

I will try again and post results once I convert my trial to subscription.

Is the above a good tutorial to run python in data factory? If not, I would greatly appreciate any suggestions.

thanks,
PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee

2021-07-28T06:34:02.113+00:00

Hello @Shannon Moran ,

Yes, this is the tutorial to run python batch jobs using ADF.

You can try again post you converting your free trial account and do let us know the status.

Thank you.
Yomar Vega 1 Reputation point

2023-01-06T02:29:10.967+00:00

hello, i have the same error.
in the stderr.txt I get the following message "dpkg: error: dpkg interface is locked by another process"