question

56220211 avatar image
0 Votes"
56220211 asked 56220211 commented

Column functions not supported in configurations - MDF Expression Error

I am building a pipeline with a Mapping Data Flow activity in synapse analytics. I am trying to create a derived column with an expression similar to below. However, it fails with the error "Column functions not supported in configurations".

array(toString(byNames(columnNames())))

I need to dynamically get the list of column names and get the values on the fly. Looks like it is not working when I use byNames() function along with columnNames() function. Is there any workaround for this?

PS: I cannot use columns() function because I need to reorder or filter some of the columns.



azure-data-factoryazure-synapse-analytics
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Just columnNames() alone will give you the column names

0 Votes 0 ·
56220211 avatar image 56220211 MarkKromer-MSFT ·

Thanks for your response. However, just the columnNames() function does not work in my case because I actually want the values of the columns. I want to get all the columns dynamically first and then reorder the columns before calculating the hash value. This is to make sure the hash value is correct even if the column orders are different in the source file.

0 Votes 0 ·

I'm having the same issue.

Trying to generate a hash of all incoming columns using- columns_hash = md5(byNames(columnNames())) which throws the error above.

If I pass a parameter in to generate the hash as in - columns_hash = md5(byNames(split($columns,','))) it works fine.

Why, when both the function split() and columnNames() return an array of string, which is then passed to the byNames() function, does the second expression work but the first fail?

0 Votes 0 ·
MarkKromer-2402 avatar image MarkKromer-2402 PhilipLee-8962 ·

Wouldn't you use just want to hash the column values instead? md5(columns())

0 Votes 0 ·
PhilipLee-8962 avatar image PhilipLee-8962 MarkKromer-2402 ·

Yep - that worked :) Stupidity error on my part doh!!!

Thanks

0 Votes 0 ·
Show more comments

0 Answers