Flatten nested data arrays with a single sub-element?

[timokoch]

We have sometimes field that are represented by a nested data structure similar to std::vector<std::array<double, 1>>. The current default behavior of gridformat is to write such fields out as vector field zero-padding the "missing" entries.

Would it make sense to change the behaviour to flatten all fixed-sized arrays of size 1 (e.g. having constexpr size()) if coorddim > 1? The two only colliding cases I can think of is writing a 1D vector on a 1D-in-2D/3D grid (this could be distinguished if the dimension of elements can be checked); and relying on the default to zero-pad a vector an lazily only pass a shortened vector. The latter case is in my opinion less useful than the flattening.

Of course both flattening and zero-padding can be also implemented outside gridformat via copying to a new data structure or by writing an adapter for data structure, but this complicates things significantly on the caller site.

@dglaeser What do you think?

[dglaeser]

Hi @timokoch, thanks for the discussion! Since the writer interface does not take raw ranges/vectors (AFAIR :D), I am not entirely sure if this is an issue we can/should address on the gridformat side. If you use the point/cell function interface, you can simply return the zero-th entry:

std::vector<std::array<double, 1>> field_values(...);
writer.set_point_field("field", [&] (const auto& point) { 
    return field_values[id(point)][0];
});

This turns it into a scalar field? If you have a custom field implementation that you pass in, then you could handle it in there. I guess one use case that purely relies on the library would be to read in a 1D vector field using a gridformat reader, but write that field out again as a scalar field. I think there exist two field transformations, however, that you could readily use:

auto field = some_reader.cell_field("field_name");
auto flattened = GridFormat::transform(field, GridFormat::FieldTransformation::flatten);
auto reshaped = GridFormat::transform(field, GridFormat::FieldTransformation::reshape_to(GridFormat::MDLayout{{
    field->layout().extent(0)
}});

The transformations can also be used for a custom field implementation of course.

Not sure if this addresses your use case? I guess we could also add an option to the writers to unfold all 1D vector fields (and then do the transformation automatically). I wouldn't add it as an option to set_cell_field/set_point_field because one can simply transform manually, e.g. writer.set_point_field("f", transform(f, FieldTransformation::flatten)).

What do you think? And please let me know if I missed the point you raised...

[timokoch]

I've also used the lambda version, and yes, no problem to realize the wanted behaviour. But I'm wondering what is the expected thing if I simply pass the range. I think, I agree that having a data structure with a nested fixed size 1 element may be unusual. In Dune, it happens often and always with the meaning that it's actually just a scalar field, not a vector field. I have fixed this downstream now (where we have the interface that accepts a vector). The behaviour is probably user-specific, so ok to do nothing from gridformat side .

[timokoch]

Aha the interface for a range doesn't even exist. Yeah then this doesn't make much sense.

[dglaeser]

Yeah the interface with a range doesn't exist because it's not always obvious how to do the mapping, and with the lambda interface it's clear that the user has to return the value for a single grid point - then we can internally put it in the right place (e.g. for structured grid output)... And I didn't see a real drawback because as a user I can just use the point to access my range.