Synapse Copy Activity Logging option change the file name and capture source file name

Naga 66 Reputation points
2021-10-21T16:00:26.96+00:00

Hi Team -

We need to move .csv files from one container to another container. Also, we have enabled logging mechanism in Copy Activity to view the error logs.

We have used For Each and Copy Activities to perform above use case. However, in log directory for each file it shows File Name as activity id , which is unable to map with source file which consists bad data. How to get the file name as source file names for log files?

Also, it generates log file for all files where it does not have bad data. How to minimize the issue and display log files which consists bad data.

Thanks in advance !!!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,425 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee
    2021-10-22T08:01:01.07+00:00

    Hello @Naga ,

    Thanks for the question and using MS Q&A platform.

    Unfortunately, you cannot provide log file name while Enable Logging in the copy activity.

    Reason: Log file name is [auto-generated-GUID]

    After the copy activity runs completely, you can see the path of log files from the output of each copy activity run. You can find the log files from the path: https://[your-blob-account].blob.core.windows.net/[logFilePath]/copyactivity-logs/[copy-activity-name]/[copy-activity-run-id]/[auto-generated-GUID].txt.

    For more details, refer to Copy activity - Log.

    I would suggest you to provide feedback on the same:

    Feedback - Azure Data Factory

    All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators