-
-
Notifications
You must be signed in to change notification settings - Fork 549
Add C# Source Generator #1726
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add C# Source Generator #1726
Conversation
Directory.Packages.props
Outdated
| <PackageVersion Include="Newtonsoft.Json" Version="13.0.3" /> | ||
| <PackageVersion Include="NodaTime" Version="3.1.9" /> | ||
| <PackageVersion Include="NSwag.Core.Yaml" Version="14.0.0-preview011" /> | ||
| <PackageVersion Include="Parlot" Version="0.0.25.0" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Source generators do not support transient dependencies: dotnet/sdk#17775
Because of that need to explicitly install some transient packages.
For example, it'll be used here to include Parlot into NuGet package https://github.com/RicoSuter/NJsonSchema/pull/1726/files#diff-ead3877feb09b2619c61d0feb8b85e0c85c627756febe0dda0829dafe13d5325R39
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <AdditionalFiles |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this project you can see how Source Generator actually works
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\NJsonSchema.CodeGeneration.CSharp\NJsonSchema.CodeGeneration.CSharp.csproj" PrivateAssets="All"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With PrivateAssets="All" it will not add NJsonSchema.CodeGeneration.CSharp as a dependency of NJsonSchema.SourceGenerators.CSharp NuGet package as I assume it's not needed. Code from NJsonSchema.SourceGenerators.CSharp should be enough to generate C# classes. For more advanced scenarios NJsonSchema.CodeGeneration.CSharp can be installed explicitly in client's projects.
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.CodeAnalysis.CSharp" PrivateAssets="all" /> | ||
| <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" PrivateAssets="all" /> | ||
| <PackageReference Include="Newtonsoft.Json" GeneratePathProperty="true" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deliberately didn't add PrivateAssets="all" for Newtonsoft.Json so it will be included as NuGet dependency. In the generated code Newtonsoft attributes will be used. That way client don't need to install Newtonsoft.Json in their project explicitly.
| @@ -0,0 +1,74 @@ | |||
| <Project Sdk="Microsoft.NET.Sdk"> | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My main concern is whether we should create a separate project for the source generator or add them to the existing NJsonSchema.CodeGeneration.CSharp. Frankly speaking, I didn't try the second approach at all, but in theory, it should work.
Below are my thoughts.
Add Source Generator to new NJsonSchema.SourceGenerators.CSharp
Pros:
- Clients will install that package only if they know they need to use a source generator.
- Easier to maintain as source generator projects require additional configuration (
IsRoslynComponent, exclude NU5128 warning, etc.)
Cons:
- The names of the NuGet packages could be confusing.
NJsonSchema.CodeGeneration.CSharpandNJsonSchema.SourceGenerators.CSharpmay sound a bit synonimical.
Add Source Generator to existing NJsonSchema.CodeGeneration.CSharp
Pros:
- We don't introduce a new NuGet package.
- By installing
NJsonSchema.CodeGeneration.CSharpclient can use either classes from the library or/and Source Generator.
Cons:
- It'll automatically install the source generator when the client does a package upgrade. (However, the source generator will not do much without additional configuration. Not sure if this is an issue).
- The size of the NuGet package will slightly increase as it'll include DLLs for the source generator.
- The project might be harder to maintain as it will be a mix of regular code and code that will go to the source generator.
|
You can check my implementation of Source Generator for NSwag to see how to support transient packages automatically https://github.com/HavenDV/H.NSwag.Generator |
Thank you @HavenDV. Amazing library 🙂 However, I have some issues with project references. I asked about them here: HavenDV/H.Generators.Extensions#15 |
| <ItemGroup> | ||
| <None Update="NJsonSchema.SourceGenerators.CSharp.nuspec" /> | ||
|
|
||
| <None Include="$(PkgFluid_Core)\lib\netstandard2.0\Fluid.dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is how dependencies are added to the NuGet package. More details here https://github.com/dotnet/roslyn/blob/main/docs/features/incremental-generators.cookbook.md#use-functionality-from-nuget-packages
|
|
||
| <Target Name="GetDependencyTargetPaths"> | ||
| <ItemGroup> | ||
| <TargetPathWithTargetPlatformMoniker Include="$(PkgFluid_Core)\lib\netstandard2.0\Fluid.dll" IncludeRuntimeDependency="false" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In case if Source Generator needs to be added as a project reference (in NJsonSchema.Demo) it needs additional configuration. More details here https://www.thinktecture.com/en/net/roslyn-source-generators-using-3rd-party-libraries/
| /// </summary> | ||
| [EditorBrowsable(EditorBrowsableState.Never)] | ||
| public static class IsExternalInit | ||
| internal static class IsExternalInit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without that change build of NJsonSchema.SourceGenerators.CSharp.Tests was failing. Here suggests to make it internal if code is a part of a library
9e35119 to
56c5c54
Compare
56c5c54 to
de1a601
Compare
|
@valchetski, has this effort been abandoned? If not, I recently tried to implement a source generator for a project I was working on and I thought I could share the source code. It's not as elegant as yours because I did not implement the customization properties. Source GeneratorImplementation Codenamespace NJsonSchema.CodeGeneration.CSharp.Analyzers;
using System.Text;
using Humanizer;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.Text;
using NJsonSchema;
using NJsonSchema.CodeGeneration.CSharp;
[Generator]
public class SchemaTypeGenerator : IIncrementalGenerator
{
public void Initialize(IncrementalGeneratorInitializationContext context)
{
var pipeline = context.AdditionalTextsProvider
.Where(static (text) => text.Path.EndsWith(".json", StringComparison.OrdinalIgnoreCase))
.Select(static (text, cancellationToken) =>
(
name: Path.GetFullPath(text.Path).Dehumanize().Pascalize(),
code: Compile(text.GetText(cancellationToken))
));
context.RegisterSourceOutput(pipeline,
static (context, pair) =>
context.AddSource(
hintName: $"{pair.name}.g.cs",
sourceText: SourceText.From(pair.code, Encoding.UTF8)));
context.RegisterPostInitializationOutput(static postInitializationContext =>
{
postInitializationContext.AddEmbeddedAttributeDefinition();
});
}
private static string Compile(SourceText? sourceText) =>
AddEmbeddedAttribute(
Compile(
sourceText?.ToString()
?? throw new ArgumentNullException(nameof(sourceText))));
private static string AddEmbeddedAttribute(string compiled) =>
compiled.Replace(
"[System.CodeDom.Compiler.GeneratedCode",
"[Microsoft.CodeAnalysis.Embedded][System.CodeDom.Compiler.GeneratedCode");
private static string Compile(string json) =>
new CSharpGenerator(
rootObject: JsonSchema.FromJsonAsync(json).GetAwaiter().GetResult(),
settings: new CSharpGeneratorSettings
{
Namespace = $"{typeof(SchemaTypeGenerator).Namespace}.Generated",
ClassStyle = CSharpClassStyle.Poco,
JsonLibrary = CSharpJsonLibrary.SystemTextJson,
TypeAccessModifier = "internal",
UseRequiredKeyword = false,
ArrayType = nameof(IEnumerable<>),
ArrayBaseType = nameof(IEnumerable<>),
ArrayInstanceType = nameof(List<>),
GenerateDataAnnotations = true,
GenerateNativeRecords = true,
GenerateDefaultValues = false,
GenerateOptionalPropertiesAsNullable = true,
GenerateNullableReferenceTypes = true,
InlineNamedArrays = true
}).GenerateFile();
}Project Configuration (NuGet Package)<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<LangVersion>preview</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsRoslynComponent>true</IsRoslynComponent>
<EnforceExtendedAnalyzerRules>true</EnforceExtendedAnalyzerRules>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<PackageId>NJsonSchema.CodeGeneration.CSharp.Analyzers</PackageId>
<Version>0.0.0</Version>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Fluid.Core" Version="2.24.0" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="Humanizer.Core" Version="2.14.1" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="4.14.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
<PackageReference Include="Microsoft.Extensions.FileProviders.Abstractions" Version="9.0.5" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="Namotion.Reflection" Version="3.4.2" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="NJsonSchema" Version="11.3.2" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="NJsonSchema.Annotations" Version="11.3.2" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="NJsonSchema.CodeGeneration" Version="11.3.2" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="NJsonSchema.CodeGeneration.CSharp" Version="11.3.2" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="Parlot" Version="1.3.6" PrivateAssets="all" GeneratePathProperty="true" />
</ItemGroup>
<PropertyGroup>
<GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>
<Target Name="GetDependencyTargetPaths">
<ItemGroup>
<TargetPathWithTargetPlatformMoniker Include="$(PKGFluid_Core)\lib\netstandard2.0\Fluid.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGHumanizer_Core)\lib\netstandard2.0\Humanizer.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGMicrosoft_Extensions_FileProviders_Abstractions)\lib\netstandard2.0\Microsoft.Extensions.FileProviders.Abstractions.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNamotion_Reflection)\lib\netstandard2.0\Namotion.Reflection.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNewtonsoft_Json)\lib\netstandard2.0\Newtonsoft.Json.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNJsonSchema)\lib\netstandard2.0\NJsonSchema.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNJsonSchema_Annotations)\lib\netstandard2.0\NJsonSchema.Annotations.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNJsonSchema_CodeGeneration)\lib\netstandard2.0\NJsonSchema.CodeGeneration.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGNJsonSchema_CodeGeneration_CSharp)\lib\netstandard2.0\NJsonSchema.CodeGeneration.CSharp.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGParlot)\lib\netstandard2.0\Parlot.dll" IncludeRuntimeDependency="false" />
</ItemGroup>
</Target>
<ItemGroup>
<None Include="$(OutputPath)\$(AssemblyName).dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" />
</ItemGroup>
</Project>Example UsageProject Configuration<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="NJsonSchema.CodeGeneration.CSharp.Analyzers" Version="0.0.0" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
<PackageReference Include="YamlDotNet" Version="16.3.0" />
</ItemGroup>
<ItemGroup>
</ItemGroup>
<ItemGroup>
<AdditionalFiles Include="schema.json" />
<EmbeddedResource Include="reference/**/*.yml" />
</ItemGroup>
</Project>Example YAML file being parsed## NoSQLQueryReference
type: command
name: count
description: |-
The `count` command is used to count the number of documents in a collection that match specific criteria.
summary: |-
This command is useful for obtaining quick statistics about the data stored in your collections, such as the number of documents that meet certain criteria.
syntax: |-
db.collection.count(
<query>,
<options>
);
parameters:
- name: query
type: object
required: true
description: |-
A document specifying the selection criteria using query operators.
- name: options
type: object
required: false
description: |-
A document specifying options including, but not limited to `limit` and `skip`.
examples:
sample:
set: stores
filter: |-
{}
items:
- title: Counting all documents in a collection
explanation: |
Use the `count` command with an empty document to count **all** documents in a collection.
description: |-
In this example, all documents in the `stores` collection are counted.
query: |-
db.stores.count({ "_id": "00000000-0000-0000-0000-000000003002" })
output:
value: |-
1
- title: Counting documents that match nested criteria
explanation: |-
The `query` parameter supports nested parameters.
description: |-
In this example, the command counts documents that match the string value `"Incredible Discount Days"` for the `promotionEvents.eventName` field.
query: |-
db.stores.count({ "promotionEvents.eventName": "Incredible Discount Days" })
output:
value: |-
2
- title: Counting documents that match multiple criteria
explanation: |-
The `query` parameter also supports multiple parameters.
description: |-
In this example, the `locationLatitude` and `locationLongitude` parameters are used to count documents that match on these specific values.
query: |-
db.stores.count({ "location.lat": -2.4111, "location.lon": 72.1041 })
output:
value: |-
1
related:
- reference: /operators/query/regexExample JSON schema{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "NoSQLQueryReference",
"type": "object",
"required": [
"type",
"name",
"description",
"syntax",
"parameters",
"examples"
],
"properties": {
"type": {
"type": "string",
"enum": [
"operator",
"command"
]
},
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"summary": {
"type": "string"
},
"syntax": {
"type": "string"
},
"parameters": {
"type": "array",
"items": {
"type": "object",
"required": [
"name",
"type",
"required"
],
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string",
"enum": [
"object",
"string",
"number",
"pattern"
]
},
"required": {
"type": "boolean"
},
"description": {
"type": "string"
}
},
"additionalProperties": false
}
},
"examples": {
"type": "object",
"required": [
"items"
],
"properties": {
"sample": {
"type": "object",
"required": [
"set",
"filter"
],
"properties": {
"set": {
"type": "string",
"enum": [
"products",
"stores",
"employees"
]
},
"filter": {
"type": "string"
}
},
"additionalProperties": false
},
"items": {
"type": "array",
"items": {
"type": "object",
"required": [
"title",
"description",
"query"
],
"properties": {
"title": {
"type": "string"
},
"explanation": {
"type": "string"
},
"description": {
"type": "string"
},
"query": {
"type": "string"
},
"output": {
"type": "object",
"required": [
"value"
],
"properties": {
"devlang": {
"type": "string",
"enum": [
"bson",
"json",
"plaintext"
]
},
"value": {
"type": "string"
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
},
"related": {
"type": "array",
"maxItems": 4,
"items": {
"type": "object",
"required": [
"reference"
],
"properties": {
"reference": {
"type": "string"
}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
}Example Codeusing System.Reflection;
using NJsonSchema.CodeGeneration.CSharp.Analyzers.Generated;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;
IDeserializer yamlDeserializer = new DeserializerBuilder()
.WithNamingConvention(CamelCaseNamingConvention.Instance)
.WithCaseInsensitivePropertyMatching()
.IgnoreUnmatchedProperties()
.Build();
Console.WriteLine("Parsing YAML files...");
static bool filter(string resource) =>
resource.Contains(".reference", StringComparison.OrdinalIgnoreCase) &&
resource.EndsWith(".yml", StringComparison.OrdinalIgnoreCase);
IEnumerable<string> resources = Assembly
.GetExecutingAssembly()
.GetManifestResourceNames()
.Where(filter);
foreach (string resource in resources)
{
Console.WriteLine($"Reading {resource}...");
using Stream yamlStream = Assembly.GetExecutingAssembly().GetManifestResourceStream(resource)
?? throw new InvalidOperationException($"Resource '{resource}' not found.");
using StreamReader yamlReader = new(yamlStream);
NoSQLQueryReference reference = yamlDeserializer.Deserialize<NoSQLQueryReference>(yamlReader)
?? throw new InvalidOperationException($"Failed to deserialize resource '{resource}'.");
Console.WriteLine(reference);
Console.WriteLine();
} |
No, still waiting for this PR to be reviewed |
What's in this PR
This change aims to add a C# Source generator that automatically creates classes based on JSON schema. The idea is to make code generation more straightforward for generic scenarios. I've tried that approach on my project and it worked quite well. I thought it'd be good to make it a part of the library. This Source Generator has few configurable properties and can be extended in the future if needed.
Current approach
To generate C# classes you need to create a separate console project that will generate a
.csfile and put it somewhere in your Solution. Then you need to manually run that console application to generate/regenerate the models. Here's exampleNew (additional) approach
Add the following elements in the
.csprojfile of your project where the generated.csfile will exist:After that Source Generator will automatically create a
.csfile during compilation. No additional console project or manual job is needed.Concerns
I have some questions/concerns regarding the implementation and have put them in the comments. We can start from this main one.
Also added some explanatory comments about why something was implemented that way. As tooling for the Source Generators is not the best for now and there is a lot of msbuild magic in the
.csprojfiles.