Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,16 @@

## Publishing

* To publish the library, update the `<VersionPrefix>` in [`Directory.Build.props`](Directory.Build.props), add a corresponding section to the top of [`ReleaseNotes.md`](ReleaseNotes.md), commit, and push.
To publish an official release:

## Template
* Update the `<VersionPrefix>` in [`Directory.Build.props`](Directory.Build.props).
* Add a corresponding section to the top of [`ReleaseNotes.md`](ReleaseNotes.md).
* Push or create a PR for review.

* This repository uses the [`faithlife-build`](https://github.com/Faithlife/CSharpTemplate/tree/faithlife-build) template of [`Faithlife/CSharpTemplate`](https://github.com/Faithlife/CSharpTemplate).
### Prereleases

Certain changes can be hard to unit test and are better tested in a real consumer project. In this case, you can publish a beta version of the library for testing.

To publish a beta, add a `<VersionSuffix>` below `<VersionPrefix>` in [`Directory.Build.props`](Directory.Build.props), e.g., `beta.1`. Publish as above.

When beta testing is done, delete the `<VersionSuffix>` and publish again.
70 changes: 66 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,69 @@
# DapperUtility
# Faithlife.Utility.Dapper

**Faithlife.Utility.Dapper** is a utility library for [Dapper](https://github.com/StackExchange/dapper-dot-net).
**Faithlife.Utility.Dapper** is a utility library for [Dapper](https://github.com/DapperLib/Dapper).

[![Build](https://github.com/Faithlife/DapperUtility/workflows/Build/badge.svg)](https://github.com/Faithlife/DapperUtility/actions?query=workflow%3ABuild) [![NuGet](https://img.shields.io/nuget/v/Faithlife.Utility.Dapper.svg)](https://www.nuget.org/packages/Faithlife.Utility.Dapper)
[![NuGet](https://img.shields.io/nuget/v/Faithlife.Utility.Dapper.svg)](https://www.nuget.org/packages/Faithlife.Utility.Dapper)

[Documentation](https://faithlife.github.io/DapperUtility/) | [Release Notes](https://github.com/Faithlife/DapperUtility/blob/master/ReleaseNotes.md) | [Contributing](https://github.com/Faithlife/DapperUtility/blob/master/CONTRIBUTING.md)
## Overview

Faithlife.Utility.Dapper adds [BulkInsert](#bulkinsert), a Dapper-like API for inserting many rows efficiently while still using command parameters.

## Installation

Faithlife.Utility.Dapper should be installed [via NuGet](https://www.nuget.org/packages/Faithlife.Utility.Dapper).

## BulkInsert

The [BulkInsert](./src/Faithlife.Utility.Dapper/BulkInsertUtility.cs) and [BulkInsertAsync](./src/Faithlife.Utility.Dapper/BulkInsertUtility.cs) extension methods allow efficient insertion of many rows into a database table with a familiar Dapper-like API.

### Problem

Dapper already has a mechanism for "bulk insert". Calling `Execute` with an `IEnumerable` will execute the specified `INSERT` command once for each item in the sequence. Unfortunately, this can be an extremely slow way to insert a large number of rows into a database.

Each DBMS has its own preferred approaches for efficiently inserting many rows into a database, but the most portable way is to execute an `INSERT` command with multiple rows in the `VALUES` clause, like so:

```sql
INSERT INTO widgets (name, size) VALUES ('foo', 22), ('bar', 14), ('baz', 42)
```

Building a SQL statement for a large number of rows is straightforward, but runs the risk of SQL injection problems if the SQL isn't escaped properly.

Using command parameters is safer, but building and executing the SQL is more complex. Furthermore, databases often have a limit on the maximum number of command parameters that can be used, so it can be necessary to execute multiple SQL statements, one for each batch of rows to insert.

### Solution

`BulkInsert` is a simple Dapper-like extension method that builds the SQL commands for each batch and leverages Dapper to inject the command parameters.

```csharp
var widgets = new[] { new { name = "foo", size = 22 }, new { name = "bar", size = 14 }, new { name = "baz", size = 42 } };
connection.BulkInsert("INSERT INTO widgets (name, size) VALUES (@name, @size) ...", widgets);
```

The `...` after the `VALUES` clause must be included. It is used by `BulkInsert` to find the end of the `VALUES` clause that will be transformed. The call above will build a SQL statement like so:

```sql
INSERT INTO widgets (name, size) VALUES (@name_0, @size_0), (@name_1, @size_1)
```

The actual SQL statement will have as many parameters as needed to insert all of the specified rows. If the total number of command parameters exceeds 999 (the maximum number that [SQLite](https://www.sqlite.org/) supports and an efficient number for [MySql.Data](https://www.nuget.org/packages/MySql.Data/)), it will execute multiple SQL commands until all of the rows are inserted.

All of the transformed SQL will be executed for each batch, so including additional statements before or after the `INSERT` statement is not recommended.

Execute the method within a transaction if it is important to avoid inserting only some of the rows if there is an error.

### Reference

The `BulkInsert` and `BulkInsertAsync` methods of the [BulkInsertUtility](./src/Faithlife.Utility.Dapper/BulkInsertUtility.cs) static class are extension methods on `IDbConnection`. They support these arguments:

* `sql`: The SQL to execute, which must have a `VALUES` clause that is followed by `...`.
* `commonParam`: The values of any command parameters that are outside the `VALUES` clause, or are common to every row. Optional.
* `insertParams`: The values of the command parameters for each row to be inserted.
* `transaction`: The current transaction, if any. See `Dapper.Execute`.
* `batchSize`: If specified, indicates the number of rows to insert in each batch, even if doing so requires more than 999 command parameters.
* `cancellationToken`: The optional cancellation token for `BulkInsertAsync`.

The method returns the total number of rows affected, or more specifically, the sum of the numbers returned when executing the SQL commands for each batch.

## Contributing

See [Contributing](./CONTRIBUTING.md) for setup and contribution guidelines. For a history of notable changes, see the [Release Notes](./ReleaseNotes.md).
Loading