`schema/apply/` raises RecursionError when a document creates multiple new, cross-referencing Custom Object Types in one request

[cruse1977]

Plugin Version

0.6.1

NetBox Version

4.6.9

Python Version

3.14.4

Steps to Reproduce

1. On a NetBox instance with no existing Custom Object Types, POST a portable
   schema document (see docs/portable-schema.md) to
   /api/plugins/custom-objects/schema/apply/ that defines two or more brand-new
   Custom Object Types in the same request, where at least one type has an
   object/multiobject field whose related_object_type points at another
   new type in the same document via custom-objects/<slug>.

   Minimal example body:

   {
     "allow_destructive": false,
     "schema": {
       "schema_version": "1",
       "types": [
         {
           "name": "ospf_instance",
           "slug": "ospf-instances",
           "fields": [
             { "id": 1, "name": "name", "type": "text", "primary": true, "required": true, "unique": true }
           ]
         },
         {
           "name": "ospf_area",
           "slug": "ospf-areas",
           "fields": [
             { "id": 1, "name": "name", "type": "text", "primary": true, "required": true, "unique": true },
             {
               "id": 2,
               "name": "instance",
               "type": "object",
               "required": true,
               "related_object_type": "custom-objects/ospf-instances"
             }
           ]
         }
       ]
     }
   }

2. Observe the response.

Notes on scope:

• Applying the exact same two-type document works fine if you split it into two
  separate schema/apply/ requests — one per type, in dependency order (create
  ospf-instances first, then ospf-areas).
• Calling netbox_custom_objects.schema.executor.apply_document() directly from
  manage.py nbshell with the same multi-type document also succeeds — the bug
  only manifests when the call happens inside a live Django request/response
  cycle (i.e. via the SchemaApplyView API endpoint).
• Reproduced with a 2-type and a 3-type document (3rd type
  ospf_interface -> custom-objects/ospf-areas); both fail identically.

Expected Behavior

POST /api/plugins/custom-objects/schema/apply/ should create all of the new,
cross-referencing Custom Object Types described in the document in a single
atomic call, returning 200 with {"applied": true, "diffs": [...]}.

This is not just an assumption — docs/portable-schema.md documents multi-COT,
single-request apply as the supported path. Its "Dependency Ordering" section
(under "Executor (Developer Reference)") states:

When a schema document contains multiple new COTs that reference each other
via related_object_type: "custom-objects/<slug>", the executor performs a
topological sort to ensure referenced COT tables exist before any referencing
field is added. Cycles among new COTs raise CircularDependencyError.

The only failure case called out there is a dependency cycle. Nothing in
docs/portable-schema.md (Usage, Typical End-to-End Workflow, or the API
request/response reference) suggests that a document with multiple new,
non-circular, cross-referencing COTs needs to be split into separate requests.

Expected Behavior

POST /api/plugins/custom-objects/schema/apply/ should create all of the new,
cross-referencing Custom Object Types described in the document in a single
atomic call, returning 200 with {"applied": true, "diffs": [...]}.

This is not just an assumption — docs/portable-schema.md documents multi-COT,
single-request apply as the supported path. Its "Dependency Ordering" section
(under "Executor (Developer Reference)") states:

When a schema document contains multiple new COTs that reference each other
via related_object_type: "custom-objects/<slug>", the executor performs a
topological sort to ensure referenced COT tables exist before any referencing
field is added. Cycles among new COTs raise CircularDependencyError.

The only failure case called out there is a dependency cycle. Nothing in
docs/portable-schema.md (Usage, Typical End-to-End Workflow, or the API
request/response reference) suggests that a document with multiple new,
non-circular, cross-referencing COTs needs to be split into separate requests.

Observed Behavior

The request fails with 500 Internal Server Error:

{
  "error": "maximum recursion depth exceeded",
  "exception": "RecursionError",
  "netbox_version": "4.6.9-Docker-5.0.2",
  "python_version": "3.14.4"
}

Full traceback (captured by calling the view directly via Django's test
Client, bypassing the API's generic 500 handler):

  File ".../netbox_custom_objects/__init__.py", line 597, in get_models
    model = custom_type.get_model()
  File ".../netbox_custom_objects/models.py", line 1819, in get_model
    self._after_model_generation(attrs, model)
  File ".../netbox_custom_objects/models.py", line 1538, in _after_model_generation
    own_ot = ObjectType.objects.get_for_model(model)
  File ".../netbox/core/models/object_types.py", line 98, in get_for_model
    ot = self.get_or_create(
         ...
    )[0]
  File ".../django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File ".../django/db/models/query.py", line 994, in get_or_create
    return self.create(**params), True
  File ".../netbox/core/models/object_types.py", line 34, in create
    return super().create(**kwargs)
  File ".../django/db/models/query.py", line 659, in create
    self.model._meta._reverse_one_to_one_field_names
  File ".../django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File ".../django/db/models/options.py", line 1030, in _reverse_one_to_one_field_names
    field.name for field in self.related_objects if field.one_to_one
  File ".../django/db/models/options.py", line 621, in related_objects
    all_related_fields = self._get_fields(
  File ".../django/db/models/options.py", line 925, in _get_fields
    for obj in parent._meta._get_fields(
  File ".../django/db/models/options.py", line 943, in _get_fields
    all_fields = self._relation_tree
  File ".../django/utils/functional.py", line 47, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File ".../django/db/models/options.py", line 843, in _relation_tree
    return self._populate_directed_relation_graph()
  File ".../django/db/models/options.py", line 810, in _populate_directed_relation_graph
    all_models = self.apps.get_models(include_auto_created=True)
  File ".../django/apps/registry.py", line 185, in get_models
    result.extend(app_config.get_models(include_auto_created, include_swapped))
  File ".../netbox_custom_objects/__init__.py", line 597, in get_models
    model = custom_type.get_model()
  File ".../netbox_custom_objects/models.py", line 1819, in get_model
    self._after_model_generation(attrs, model)
  ... (repeats, cycling back through get_for_model / get_or_create / create /
       _relation_tree / _populate_directed_relation_graph / get_models /
       get_model, growing the stack each time) ...
  File ".../django/db/models/fields/related.py", line 1191, in get_prep_value
    return self.target_field.get_prep_value(value)
RecursionError: maximum recursion depth exceeded

The transaction.atomic() wrapper in the executor does correctly roll back —
after the 500, GET /api/plugins/custom-objects/custom-object-types/ shows
"count": 0, so no partial/orphaned COTs or tables are left behind. The bug is
a hard failure of the request, not data corruption.

Suspected Cause

CustomObjectsPluginConfig.get_models() (netbox_custom_objects/__init__.py:597)
overrides Django's app-registry model enumeration and, for each CustomObjectType,
calls get_model() to lazily generate the dynamic model class. get_model() in
turn calls _after_model_generation(), which calls
ObjectType.objects.get_for_model(model) to ensure a NetBox ObjectType
(ContentType wrapper) exists for the newly generated model.

ObjectType.objects.get_or_create() → .create() accesses
model._meta._reverse_one_to_one_field_names, which is a cached property that,
on first access, triggers Django to build the global relation graph via
self._meta._relation_tree → apps.get_models(include_auto_created=True).
Because get_models() is overridden by the plugin, this re-enters
custom_type.get_model() for every CustomObjectType row, including ones
newly created earlier in the same apply_document() call whose FK
(related_object_type pointing at custom-objects/<slug>) has not finished
being fully wired up yet.

For those still-being-generated models, get_model() re-triggers
_after_model_generation() → get_for_model() → the same relation-graph build
→ get_models() again, and so on — an unbounded mutual recursion between
Django's relation-graph cache population and the plugin's dynamic model
generation, specific to the window where 2+ new, cross-referencing COTs exist
in the same uncommitted transaction and haven't each individually completed
_after_model_generation() yet.

This does not reproduce when:

• Only one new COT is created per apply_document()/API call (no other
  not-yet-finalized COT for get_models() to trip over), or
• The call happens outside a fresh request context where Django's relation-tree
  cache (Options._relation_tree) may already be warm from prior activity in
  the same process (e.g. an nbshell session that has already touched the app
  registry), which appears to avoid re-entering
  _populate_directed_relation_graph() altogether.

Proposed Fix

A few options, roughly in order of invasiveness:

1. In the executor (netbox_custom_objects/schema/executor.py), when applying
   a document with multiple new COTs, fully finalize each COT (including its
   ObjectType/ContentType registration) one at a time, in the existing
   topological order, before moving on to the next — rather than creating all
   COT rows/fields first and letting model generation/ObjectType resolution
   happen lazily and interleaved.
2. Pre-create the ObjectType row for each new COT's model up front (before
   any field with a related_object_type reference to it is added), so that by
   the time a later COT's FK needs to resolve that ObjectType,
   get_for_model() short-circuits on the existing row instead of calling
   .create() and rebuilding the relation graph.
3. Guard CustomObjectsPluginConfig.get_models() against re-entrancy (e.g. a
   thread-local/context-var flag set while a get_model() call for a given COT
   is already in progress), returning already-registered models without
   recursing into generation for COTs currently mid-construction.

As a workaround in the meantime, callers of schema/apply/ can submit each new,
cross-referencing Custom Object Type as its own request, in dependency order
(referenced types before referencing types) — this was verified to succeed
reliably against 4.6.9 / plugin 0.6.1.

[jnovinger]

My view is option 3 (the re-entrancy guard) is the right fix. #686 hits this same get_models() recursion through a completely different callsite (_wire_polymorphic_reverse_descriptors → related_object_types.all()) on schema save/modify, nowhere near the executor.

An executor-only fix (options 1 and 2) resolves this report but leaves #686 live, along with any future path that touches the relation graph during model generation. The guard covers all of them at the source.

Regression tests should exercise both re-entry paths: the executor multi-COT apply here, and the polymorphic descriptor-wiring path from #686.