The sharepoint list is empty when import from Azure Data factory, but I can only see specific coloumns and import them, not the full list

Question

question

JanBoldt-5975 asked May 15, '22 KranthiPakala-MSFT commented May 20, '22

The sharepoint list is empty when import from Azure Data factory, but I can only see specific coloumns and import them, not the full list

I have followed the guideline to import a sharepoint list using ADF:

https://docs.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory

I have created the Linked service. When I need to define the source in the pipeline, then when selecting the list, then no data is shown in the PreviewData. I can select a column which is also shown and see the data. I do not know what it is that I have done wrong, is there some setting on the list that needs to be specified.

azure-data-factory office-sharepoint-online

image.png (12.1 KiB)

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 1 · 2022-05-16T23:20:56.787Z

KranthiPakala-MSFT answered May 16, '22 JanBoldt-5975 commented May 17, '22

Hello @JanBoldt-5975,

Thanks for the question and using MS Q&A platform.

I tried to reproduce the issue on my end but no luck. I was able to see the data if I do a $select=* query on the selected List

Below is my dataset configuration, please make sure to select the List name in your dataset settings and try to do a preview data from the data set and then try to use the query in Copy Data activity source settings. If you are able to see the data from dataset preview tab then you should be able to see the data using Query under Copy Activity Source preview section.

If you still notice this issue, to work around, I would suggest to select the Table option instead of Query under Copy activity Source settings as you are trying to view the whole list data. If you want to filter specific data related to your list then use Query option.

Hope this will help. Please let us know if any further queries.

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

image.png (59.8 KiB)

image.png (19.6 KiB)

image.png (18.7 KiB)

· 3

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT · May 16 at 11:24 PM

I even tried to remove the complete query and did a preview and was able to view the data.

I would suggest to preview the data from your dataset and check if you can view from there.

0 ·

image.png (59.1 KiB)

JanBoldt-5975 · May 17 at 11:21 AM

Thank you for your reply. I have tried all possible ways (table and query) of extracting the data without luck. What is included in the "GetBearerToke" and would you have a link to a guide on how to set it up. I cannot see that as included in the guide I have used:

https://docs.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory

Could you provide a link to the guide you are using.

I am not getting any data when I preview the dataset of the sharepoint list, but when I choose one of the columns which is available, then data is shown.

0 ·

JanBoldt-5975 · May 17 at 12:09 PM

The picture above shows that there is no data in the list.

The picture above shows that there is data in one of the columns.

The picture above shows the columns in the list.

0 ·

picture1.jpg (13.6 KiB)

picture3.png (6.1 KiB)

picture4.png (38.2 KiB)

Answer 2 · 2022-05-19T13:17:47.917Z

JanBoldt-5975 answered May 19, '22 KranthiPakala-MSFT commented May 20, '22

I found the mistake. I had replaced the address in the Scope="..." with the direct reference to the sharepoint site. It should be as specified as below.

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT · May 20 at 11:07 PM

Hello @JanBoldt-5975,

Glad to know that you were able to find the actual root cause and appreciate much for sharing your findings with us here as it can be helpful for others who read this thread.

Thank you and have a good day!

0 ·

question