Cosmos: bulk execution

[AndriySvyryd]

What problem are you trying to solve?

https://devblogs.microsoft.com/cosmosdb/introducing-bulk-support-in-the-net-sdk/
https://devblogs.microsoft.com/cosmosdb/bulk-updates-with-optimistic-concurrency-control/

Describe the solution you'd like

No response

[JoasE]

In #37033 I have code for adding an option to enable bulk execution in the sdk to EF. However, in order to properly make use of bulk mode: write operations should be sent in parallel to the sdk and not be part of a transactional batch.

Currently the CosmosDatabaseWrapper always executes all writes sequentially and by default (AutoTransactionBehavior.WhenNeeded) will batch non-trigger document writes. It will perform a normal/non-batched write if the amount of items that can be batched for a partition key is 1, but it will also do this sequentially. We will have to introduce (conditional) parallelism into the saving of non-batched documents if we want to correctly leverage the bulk execution mode of the sdk.

EF does sent a write as a transactional batch even if the batch only contains 1 document due to the size of the document. I think we should actually keep it that way as bulk will also sent a document of that size as a single request. Bulk even aims to keep the request size way lower then 2MiB, aiming for ~220KB (215KiB). If a document is larger than 215KiB it will sent a bulk request with a single operation in it.

My proposition would be to:

• When bulk mode is enabled, any groups of batchable items that contain only a single item should be categorized as non-batchable writes during the grouping phase (instead of determining this during the execution phase, as is currently done).
  This change will affect write order. Performing this operation during the grouping phase in non-bulk mode would increase computational overhead and make write order management more complicated for users, which is why I am suggesting to only do this when bulk mode is enabled. However, we could also do this for non bulk scenarios for consistency reasons
• When bulk mode is enabled, all documents classified as non-batchable writes should be executed in parallel.

There is a small issue here with the sync savechanges api and the current design of the CosmosDatabaseWrapper. The CosmosDb sdk does not support sync operations, and this is abstracted by the CosmosClientWrapper. The CosmosClientWrapper exposes sync methods which are used by the CosmosDatabaseWrapper. Implementing parallelism in the sync savechanges method only to synchronously invoke async methods in the CosmosClientWrapper seems inefficient in many ways. I would suggest either removing the sync CosmosClientWrapper methods and simply invoking CosmosDatabaseWrapper.SaveChangesAsync().GetResult() in the sync CosmosDatabaseWrapper.SaveChanges, or throwing an exception for when bulk is enabled.

Since batching will skip bulk, I think users will want to use AutoTransactionBehaviour.Never in most cases when having bulk execution enabled. I already included this in the summary on the options method to enable bulk execution, but I would suggest adding a warning from EF aswell when SaveChangesAsync is called with bulk mode enabled and not using AutoTransactionBehaviour.Never. The warning shouldn't throw by default imo, but you could argue that it should.

[roji]

The CosmosDb sdk does not support sync operations, and this is abstracted by the CosmosClientWrapper. The CosmosClientWrapper exposes sync methods which are used by the CosmosDatabaseWrapper.

I don't think we should worry about sync. EF 9 disabled sync support by default, and I believe that EF 11 would be the right time to remove it entirely, as users will have had a full LTS release - 10 - which obsoleted it but allowed opting back in.

@AndriySvyryd does that make sense to you? I've opened #37059 to track doing this in 11.

As to the rest and the relationship between bulk and batch, @AndriySvyryd (and you) probably know more about the details.

[AndriySvyryd]

Since batching will skip bulk, I think users will want to use AutoTransactionBehaviour.Never in most cases when having bulk execution enabled. I already included this in the summary on the options method to enable bulk execution, but I would suggest adding a warning from EF aswell when SaveChangesAsync is called with bulk mode enabled and not using AutoTransactionBehaviour.Never. The warning shouldn't throw by default imo, but you could argue that it should.

I think the warning should throw by default, the two modes are too different in their performance to assume that the user will be aware that sometimes one will be used and not the other.