Propagate singleton class creation to descendants

[vinistock]

Background

We were creating broken descendant chains for singleton classes. The essence of the issue is that when a parent singleton class was created, it did not propagate the creation of singletons to its descendants, resulting in missing data in the graph and non-sensical results.

For example:

class Parent
  # Singleton class created because it is explicitly opened here
  def self.foo; end
end

# This class has no explicit opening of its singleton. Because we didn't
# propagate, it would end up never being created
class Intermediate < Parent; end

class Child < Intermediate; end

That would result in graph["Parent"].descendants to return Intermediate and Child (correct), while graph["Parent::<Parent>"].descendants would return nothing, which is incorrect and inconsistent.

The fix

When we are linearizing singleton ancestors, we need to figure out what is the parent class of the singleton, for which we have a dedicated method singleton_parent_id. That method recurses and determines the parent class based on the entity we are looking at. Eventually, we reach the ultimate attached object, read its parent class and use it as the ancestor.

That is the point where we need to grab the attached descendants and ensure that we're creating singletons for all of them, matching all depth levels so that Parent::<Parent>::<<Parent>> (and any other level) is also handled correctly and lazily (without creating unnecessary singleton classes).

The major gotcha that makes this change require context to apply is that simply propagating to descendants immediately results in infinite recursion. The reason is not very trivial to trace, but I tried to document it as best as I could in a comment (please let me know if it's not clear).

Finally, to ensure proper behaviour in incremental resolution scenarios, we also need a tiny modification to the parent ancestor linearization helper, so that when a class appears later, we correctly create all necessary singleton levels.

Impact

Naturally, this change is not 100% free. We were "avoiding" work by building an inconsistent graph, which is faster, but not correct. That said, the impact is acceptable and we need the correctness (+ ~2s resolution + ~200MB graph). I'm sure we can shave this off with more optimizations.

[soutaro]

@vinistock This looks good overall, but the reported +200 MB increase in graph size feels unexpectedly large. I think we should validate it against the number of additional singleton-class declarations. It may ultimately be acceptable, but I’m concerned the increase could be disproportionate to the additional graph structure required.

For reference, my previous PR (#893), which I believe should produce a broadly comparable graph shape, added 17,858 declarations for a +25 MB increase.

[Morriar]

# resolution 1

class Bar
  def self.foo; end
end

class Other
  def self.foo; end
end

class Child < Bar; end

# resolution 2

class Child < Other; end

Do we update the parent of Child::<Child>?

[Morriar]

Can we add a test like this:

# First resolution
class Child; end

# Later resolution
def Object.foo; end

I'm not sure we're properly updating the Child::<Child> parent.

[Morriar]

Can you try this:

# resolution 1

# parents.rb
class OldParent
  def self.old_method; end
end

class NewParent
  def self.new_method; end
end

# child_a.rb
class Child < OldParent; end

# child_b.rb
class Child < OldParent; end

then

# resolution 2

# child_a.rb
class Child < NewParent; end

# child_b.rb
class Child < NewParent; end

Is the parent of Child::<Child> updated?

[Morriar]

Since descendants is already transitive, aren't we repeating a lot of work here? Creating the singleton for A visits all of its descendants, and then each singleton we enqueue does another overlapping scan when it is linearized. For a deep hierarchy, this looks quadratic.

Could we track the singleton depth already propagated for each declaration during this resolution and skip these repeated scans?