question

EddynsonVega-8570 avatar image
0 Votes"
EddynsonVega-8570 asked ShaikMaheer-MSFT commented

Dataflow fails when executing pipeline and presents the following error

{"message":"Job failed due to reason: The store configuration is not defined. This error is potentially caused by invalid parameter assignment in the pipeline.. Details:","failureType":"UserError","target":"Data flow","errorCode":"DF-Executor-StoreIsNotDefined"}

137078-screenshot-8.png



When I debug the dataflow it works fine,but when I execute it though a trigger it presents the error.

azure-data-factory
screenshot-8.png (15.0 KiB)
· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @EddynsonVega-8570 ,

Thank you for posting query on Microsoft Q&A Platform.

It seems you configuration inside your dataflows are parameterized and when you are debugging you might be passing that configurations information correctly in to that parameters.

But when you are triggering pipeline, you may be passing the values in to these parameters runtime by fetching data from somewhere may be. I feel your runtime values in to you parameters are incorrect and hence you are ending up with this issue.

  • Could you please review your values what you are passing in to dataflow runtime and see if that causing issue?

  • Could you please share more details on your dataflow implementation step by step so that we can try to repro scenario and see? Thank you

0 Votes 0 ·

Hi @EddynsonVega-8570 ,

Following up to see if you get chance to look above comment and share details?

0 Votes 0 ·

Hi @ShaikMaheer-MSFT ,

In the execution trigger I have carefully reviewed the dynamically defined parameter of an activity of type Lookup and sends it as a string. Example: 137861-screenshot-9.png


0 Votes 0 ·
screenshot-9.png (9.8 KiB)

I use this parameter in the dataflow filter, to filter the data that have dates greater than the date of the parameters.

0 Votes 0 ·

Hi @EddynsonVega-8570 ,

Thank you for sharing details. So, the value which are passing runtime is matching with the format of the value which you are using during your debug dataflow. I guess that might some difference between the format of value between runtime and debug. You might need to format it accordingly to correct your error.

Could you please share your debug dataflow details. Like what is your parameter type? and what value you are passing to it during debug data preview? and how you are using that parameter in filter transformation expression?

0 Votes 0 ·

@ShaikMaheer-MSFT

The date value that is being sent at run time does not match the format of the value at data processing time.

- Date format that is sent at run time: 2020-01-26T00:00:00Z

- Date format in data processing: 2020-02-27

Note: Something important to note is that both types of data are different, the one that is sent dynamically is of type string, and the one that is compared in the processing is of type date.

That is why in the filter step there is a date type convesion for the parameter that is received in string and that is why you have a split expression to make a cut in the letter T and in this way achieve that both formats coincide, but I do not know if it is really okay.

Example: 137864-screenshot-10.png


0 Votes 0 ·
screenshot-10.png (34.2 KiB)

1 Answer

ShaikMaheer-MSFT avatar image
1 Vote"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @EddynsonVega-8570 ,

Thank you for Posting query on Microsoft Q&A and sharing few insights on issue.

I tried similar implementation. by having a parameter in data flow and supplying string value to that parameter from data flow activity. "

I see error when I check Expression box inside Dataflow Parameters tab. When I Uncheck Expression box then I am not seeing any Issue.

When you uncheck expression box that means the string parameter values will be wrapped into single quotes using the interpolation syntax

You try to execute your data flow by checking and un-checking expression box to see how it behaves.

Below are screenshots:
138023-image.png

137906-image.png


How ever, Your error message in the question still makes little confusion as it says store not defined. That means you may be using some dataset which may be pointing to a storage which is not defined. Kindly review this point as also once. Thank you.

Hope this will help. Please let us know how it goes and if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators


image.png (27.7 KiB)
image.png (65.2 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@ShaikMaheer-MSFT Thank you, this has solved my problem.

0 Votes 0 ·

@EddynsonVega-8570 - Happy to know that your issue resolved. Thank you.

0 Votes 0 ·