Copy Data From MongoDb 3.2 -> MongoDb 3.6 Fails

Question

question

DariuszJalowiec-2129 asked Apr 7, '22 Luka2345-0885 commented Apr 25, '22

Copy Data From MongoDb 3.2 -> MongoDb 3.6 Fails

I have a running Azure Cosmos DB API for MongoDB version 3.2 provisioned with throughout. I created a new Azure Cosmos DB API for MongoDB version 3.6 using serverless and tried to copy the data from the 3.2 DB to the second DB using data factory copy job.

the result:

The two of them that failed both have the same error of

Failure happened on 'Sink' side. ErrorCode=MongoDbOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=>Failed to operate data via MongoDB client.,Source=Microsoft.DataTransfer.Runtime.MongoDbAtlasConnector,''Type=MongoDB.Bson.BsonSerializationException,Message=Element name '' is not valid'.,Source=MongoDB.Bson,' Source Pipeline Copy serverless

When I do the same copy from a cosmo mongo DB 3.2 to another cosmo mongo db 3.2 it works fine and no errors. (same data factory job just destination modified to use 3.2 cosmo db). So knowing this I can assume the data on source is okay.

For the copy job I am using the standard build-in copy task just added a sort on the source.

I did look around some, and there was a suggestion of changing tsl=true to ssl=true which I did and it did help (before this change entire copy failed).

azure-data-factory azure-cosmos-db

image.png (54.1 KiB)

· 5

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SaurabhSharma-msft · Apr 11 at 08:59 PM

Hi @dariuszjalowiec-2129,

Thanks for using Microsoft Q&A !!
As per my understanding you are trying to copy documents (using ADF) from Azure Cosmos DB for MongoDB 3.2 to Azure Cosmos DB for MongoDB 3.6 version (serverless) and getting errors.
I have tried this myself by copying from Mongodb 3.2 to 3.6 (serverless) and it worked without any issues.
How many documents you have in the collection? Also, are you getting error against the same collections all the time and do you have sharding enabled in both source and destination db collections ?
Could you please provide a sample data so that I can further look into. Additionally, you can create a Azure support ticket for this issue. In case you have any limitations please let me know and I will help you providing a one time free support ticket.

Thanks
Saurabh

1 ·

SaurabhSharma-msft SaurabhSharma-msft · Apr 22 at 09:34 PM

Hi @dariuszjalowiec-2129,

We haven't heard back from you. Just wanted to check if you are you still facing the issue? In case If you already found a solution, would you please share it here with the community?

Thanks
Saurabh

1 ·

DariuszJalowiec-2129 SaurabhSharma-msft · Apr 25 at 05:15 PM

I fixed the problem by looking into the mongo driver source code. There is a check if element name is at least one char which was causing my problem, I was able to fix the documents causing this issue and it is okay now.

So this can be closed now

1 ·

Show more comments

Luka2345-0885 · Apr 25 at 08:32 PM

Actually you will doing some minor error which will not be showing properly. As I am not techy so when I was transferring the data of types of parrots so was facing the same issue. Then my assistant who is expert in coding transferred it easily because he told me that you have putted something wrong in the code configuration. So have a look on all the configured data.

0 ·

question