question

DivyaRajendran-5109 avatar image
0 Votes"
DivyaRajendran-5109 asked DivyaRajendran-5109 commented

Help with a ADF error

Hi Team,

I am running a pipeline that has a metadata activity which in turn calls on a dataflow that uses a date parameter as shown below and this pipeline always failed with the below error.

Parameters:


  1. startDtTm --> @addhours(pipeline().TriggerTime, -24)

  2. endDtTm --> @pipeline().TriggerTime

129212-parameters-dt.png

Error


{"StatusCode":"DF-Executor-StoreIsNotDefined","Message":"Job failed due to reason: The store configuration is not defined. This error is potentially caused by invalid parameter assignment in the pipeline.","Details":""}


On research, I found a similar ticket on GitHub, link mentioned below. In this ticket, the issue was resolved by checking the Expression checkbox
However, when I tried to do the same the checkbox is disabled for me.

https://github.com/MicrosoftDocs/azure-docs/issues/79325


Could you please guide me on how to overcome this issue?

Thanks!




azure-data-factory
parameters-dt.png (258.4 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @DivyaRajendran-5109,
Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

1 Vote 1 ·
DivyaRajendran-5109 avatar image DivyaRajendran-5109 SathyamoorthyVijayakumar-MSFT ·

@SathyamoorthyVijayakumar-MSFT

The solution you provided worked beautifully :-) Thank you so much!

0 Votes 0 ·

Please do not focus on the 24 hours subtracted from pipeline trigger time, I usually use startDtTm --> @addhours(pipeline().TriggerTime, -2)

0 Votes 0 ·

1 Answer

SathyamoorthyVijayakumar-MSFT avatar image
0 Votes"
SathyamoorthyVijayakumar-MSFT answered

Hello @DivyaRajendran-5109 ,

Thanks for the question and using MS Q&A platform.

You could use the below expression for startDtTm & endDtTm respectively :

 toTimestamp(left('@{addhours(pipeline().TriggerTime,-2)}',23), 'yyyy-MM-dd\'T\'HH:mm:ss.SSS')
 toTimestamp(left('@{pipeline().TriggerTime}',23), 'yyyy-MM-dd\'T\'HH:mm:ss.SSS')

Code, @{addhours(pipeline().TriggerTime,-2)} that needs to be run is passed as string. Left is used to handle the limitation of the DataFlow Expressions. The totimestamp is used to convert the resultant string to type timestamp - that matches the parameter datatype.

I tested at my end and was able to get the above expression work successfully. The error is because there is a crossover between the dataflow expression and pipeline expression when using pipeline().triggerTime or utcnow(). The reference can be found in this document.

129753-image.png

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.




image.png (99.8 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.