question

PingpongSet avatar image
0 Votes"
PingpongSet asked ShaikMaheer-MSFT commented

Publish Error via Azure Data Factory after Terraform Deployment Pointing to existing GitHub Repo

I have an error below when I tried to publish from Azure Data Factory after It is deployed via Terraform.

Publishing error: Invalid references or dependencies found. This is
likely due to publishing outside of Git mode or editing and deleting
linked services in other branches. To recover from this state, please
refer to the errors below and our Git troubleshooting guide
https://docs.microsoft.com/en-gb/azure/data-factory/source-control#stale-publish-branch

The repo settings in the Data Factory is below:

1RzTX.png

The Data Factory is created and deployed by Terraform. All pipelines in JSON files created in the DF are stored in a GitHub repo like below:

 resource "azurerm_data_factory" "data-factory" {
   name                = "data-factory"
   location            = azurerm_resource_group.data-factory.location
   resource_group_name = azurerm_resource_group.data-factory.name
   github_configuration {
     account_name = "xxxx"
     branch_name = "main"
     git_url = "https://github.com/xxxx/azure-data-factory-etl-demo"
    repository_name = "azure-data-factory-etl-demo"
     root_folder = "/"
   }

I publish from Data Factory after the deployment without making changes to the repo and anything in Data Factory.

I tried both Options 1&2 in the link below, but it is still not working.

https://docs.microsoft.com/en-gb/azure/data-factory/source-control#stale-publish-branch


The Github repository is public

Data Factory in Github for Authorized OAuth Apps is below (GitHub -> Settings -> Applications -> Authorized OAuth Apps)

7vOiu.png
I am able to create a new branch in my Github repo from Data Factory, which means it has write permission.

The example is from the link below:
https://medium.com/twodigits/deploy-your-azure-data-factory-through-terraform-2a9c2bd2d75d

The issue has been raised from below:

https://stackoverflow.com/questions/70854827/publish-error-via-azure-data-factory-after-terraform-deployment-pointing-to-exis/70862483



azure-data-factoryazure-sql-databaseazure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @PingpongSet ,

Thank you for posting query in Microsoft Q&A Platform.

If any of our pipelines referring to datasets or linked services or other pipelines which are not existing in our pipeline, then in that case you usually end up with this error.

Could you please cross verify all your pipelines and see if they are referring to any of non existing datasets or linked services or triggers or pipelines ?

Kindly hit validate button as well in your ADF to validate your pipelines.

Hope this helps. Please let us know how it goes. Thank you.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @PingpongSet - Did you get chance to try above and share updates?

0 Votes 0 ·