RFC: Remove Nested(many=True) to keep only the List(Nested()) form

[lafrech]

See discussion in #493 (comment).

We have two ways to do the same thing. They sometimes behave differently, which generally means one of them is buggy.

IMHO, and from what I have seen from users around me, it is more natural for a new user to write List(Nested) than Nested(many). Besides, it is consistent with Dict(values=Nested).

I realize this is quite a breaking change, both from user perspective, because Nested(many=True) is probably the most commonly used form, and from developer perspective, because it involves a lot of code changes (but also a lot of code removal, which is nice).

There could be things that can be done with Nested(many=True) but not with List(Nested), because in the former case, the nested Schema has access to the whole data, while in the latter, it is called on each element. Perhaps some pre_load edge cases might suffer from the change. Any real-life example, anyone?

Anyway, since the question was raised in a comment in another issue, I figured I'd give it more visibility.

[seansfkelley]

Is there any documentation outlining the differences between these two? I find the distinction confusing as well.

[deckar01]

I don't think there is supposed to be a difference.

[sloria]

I think deprecating Nested(many=True) is a good idea. Would someone be up for doing an analysis of the current differences between the two APIs?

[lafrech]

Do you mean deprecate in 2.X and remove in 3 or deprecate in 3?

Here are a few differences:

• Validation error issue with Nested(many=True) (Can't json encode some validation errors, "unorderable types: str() < int()" #493 (comment)) (List(Nested) does the right thing.)
• Different error message when deserializing an invalid input (not a list) (Fixes for #906 (TypeError with many=True) and #907 (return [] not {} on many=True error) and #930 (disallows strings and dicts on many=True) #954 (comment)) (List(Nested) does the right thing.)
• List(Nested) prevents pre/post processors in nested schema to work on the whole data collection] (List(Nested()) doesn't use many in the nested schema #1801)

---

• Partial data if an item fails validation [List field does does not put anything in valid_data if an item fails validation #766] (Fixed in Fix List and Dict valid_data #1052)
• List is not propagating the context between a nested field and the parent schema [Context is cached improperly on nested schemas #922] (Fixed)
• only argument inconsistent between Nested(S, many=True) and List(Nested(S)) [only argument inconsistent between Nested(S, many=True) and List(Nested(S)) #946] (Fixed in Fix propagation of options to Nested containers #1229)
• partial not propagated to List(Nested) (Fixed in Fix propagation of "partial" to Nested containers #1230)

[sloria]

@lafrech I was thinking of deprecating in 3. As you pointed out, Nested(many=True) is the probably the more common usage, and removing it would break many users' code.

[sloria]

How would a self-referencing schema work with this? In other words, how would we express

friends = Nested('self', many=True, exclude=('friends',))

[deckar01]

I would expect friends = List(Nested('self', exclude=('friends',))) to work the same way, but it currently errors. Using a reference to the schema with python 3 works, so I suspect this is just an edge case that hasn't been tested.

[ex5]

Perhaps some pre_load edge cases might suffer from the change. Any real-life example, anyone?

I've got production code that relies on the fact that a nested field with many=True and @pre_dump(pass_many=True) on its schema will execute this pre_dump only once.

The benefit is making sure expensive code that is needed to be executed before nested fields can be dumped is not repeated for each element in the fields.List.

I don't really see this as an edge case though, pretty sure doing something to the whole set of related objects before dumping/loading them is a common pattern.

[deckar01]

Currently a list of nested fields does not propagate the schema context correctly, but nested many does. See #424 and #922.

[lafrech]

Yep, I just updated the list above.

[deckar01]

I saw the ref on the issue, but I couldn't figure out where it came from. Thanks.

[rooterkyberian]

@anka-sirota yes it seems like a common use case, but it very well could be done on parent schema or by subclassing List and doing custom pre_dump there. All that is possible without handling a separate many case, which I would I agree with @lafrech, just source of inconsistent in the library.

Looking at bigger picture, that this issue already have been here for couple months now, and there still few bugs before many=True can be complete eliminated from the library, I would suggest just calling it deprecated in marshmallow 3 release.
Such move will:

• prevent further delay of marshmallow 3
• make it so that there is more interest in List(Nested) and that it works consistently with Nested(many=True)