question

admin-0962 avatar image
0 Votes"
admin-0962 asked ·

Unable to Make Change Stream Request(WATCH) when Connection Pool Size is Less(CosmosDB with MongoDB API)

Team CosmosDB,

Background:
We are using Cosmos with MongoDB API ver 4.0 with change streams
From the Microsoft Documentation we understood that Default MAX Cosmos Connection Pool Size =100

Queries:
If we open 'n' change stream to WATCH on any collection , Will CosmosDB consider creating 'n' connections ?
ie.. will each WATCH operation on Cosmos consumes one dedicated connection from the Connection Pool?

We set the Max Connection Pool Size as '2' & tried creating more than 2 Change Streams. We observed that the call is never returned , For the 3rd change stream .
Is this expected result?


If we open more than 100 change streams (Considering the Pool Size as 100) Assuming that all WATCH processes are running indefinitely without
releasing the connection back to the POOL , Will this affects the Performance ?

Reference Links:
a)https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/cosmos-db/create-mongodb-dotnet.md
b) https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.documents.client.connectionpolicy.maxconnectionlimit?view=azure-dotnet

Kindly confirm ,

Query#1::
Will more number of change streams required larger pool size to be set on CosmosDB ?
Similar to mongodb -https://jira.mongodb.org/browse/DOCS-11270
Querey#2:
Whether opening multiple Change streams will affect performance ?
Similar to mongodb https://jira.mongodb.org/browse/SERVER-32946

Awaiting for your suggestion/Feedback

Thanks & Regards

azure-cosmos-db
· 4
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@admin-0962

We are checking this and will get back to you.

Regards
Navtej S

0 Votes 0 ·

@admin-0962

Can you please share the documentation that you checked for the default MAX pool size. Also convey what driver you are using to connect with the Mongo API. We need to get this info to check this futher.

Our Team did say that "For each poller (watch) that's created, one will get used from the max pool size that the driver (mongodb driver) maintains. Performance wise, this is the same as querying cosmos DB."

Please provide more info to help you better,

Regards
Navtej S

0 Votes 0 ·

@admin-0962 Please check the reply from our team and get back to us with info if further help is needed.

0 Votes 0 ·

Dear NavtejSaini,

a)We got the information about the default MAX pool size from below URL
URL:: https://azure.github.io/azure-cosmosdb-java/1.0.0/index.html?com/microsoft/azure/cosmosdb/ConnectionPolicy.html. (setMaxPoolSize Function )
b) As part of the PoC, We have used NET Mongodb.driver 2.12.2 .
c) Thank you for confirmation about WATCH. We understood that each WATCH uses one connection from the pool
d) Can you also please confirm with us the MAX pool we can set ?
During our development phase we observed that max value allowed is = 2147483647 (int max range ) .
e) We are expecting half a Million Connections with WATCH .Will this affect the Performance ?
f ) If we create a watch for User1 and if we try to create another watch with same condition for User2, will CosmosDB treats this as same existing connection or will it create new connection?
Kindly clarify our queries

Thank you

0 Votes 0 ·

1 Answer

NavtejSaini-MSFT avatar image
0 Votes"
NavtejSaini-MSFT answered ·

@admin-0962

We have received response from our team.

"The link you sent appears to be from the cosmos DB core (SQL) driver, which is not related to the API for MongoDB - since it cannot be used to connect to the API for MongoDB Database. The max pool setting is entirely based on the third party MongoDB (client) driver being used to connect to Cosmos DB. We didn't see the a max value for the pool size in those docs.

We think that the performance is ultimately impacted by how the driver treats the connections from the connection pool. If each user is creating a new connection over and over again, that's another TLS handshake that has to take place. It sounds like you might need some guidance/help setting this up through the support channels. Please raise the support ticket and share the ticket number with us and we can get it checked."

Regards
Navtej S

· 2 ·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi,

Below are the links for the MongoDB Drivers

1) .NET MongoDriver
URL: https://mongodb.github.io/mongo-csharp-driver/2.13/apidocs/html/P_MongoDB_Driver_Core_Configuration_ConnectionPoolSettings_MaxConnections.htm
Defaultis not specified .During DevPhase we got 100
2) NodeJS
URL :https://mongodb.github.io/node-mongodb-native/3.6/api/MongoClient.html
Default 5
Max pool Size 10
3) JAVA
URL http://mongodb.github.io/mongo-java-driver/4.2/apidocs/mongodb-driver-core/com/mongodb/connection/ConnectionPoolSettings.html#getMaxSize()
Default is 0.
Max pool 100

But as mentioned above thread, we planned for 1M connections.Can we meet this ?
Kindly address below Queries

e) We are expecting half a Million Connections with WATCH .Will this affect the Performance ?
f ) If we create a watch for User1 and if we try to create another watch with same condition for User2, will Cosmos DB treats this as same existing connection or will it
create new connection?

0 Votes 0 ·

@admin-0962

At this time we would like you to contact support to provide the guidance regarding this. Please share the ticket number and we will get more traction for the same.

Regards
Navtej S

0 Votes 0 ·