Open
Conversation
Enabling the groupByEnabled option allows CSV records with multiple rows per account to be aggregated by uid attribute and read as JSON string.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I added
groupByEnabledoption, which uses the uid attribute for grouping. Enabling this option allows CSV records with multiple rows per account to be aggregated by uid attribute and read as JSON string.Motivation
In our experience with IDM projects, when importing data into the IDM system via CSV, we often deal with multiple rows per account. For example, the following csv:
In the above example, the first row means that
johnbelongs to theabcdepartment asengineer, and the second row means thatjohnbelongs to theefgdepartment asmanager.The current CSV connector cannot handle this kind of one-account, multiple rows data. Therefore, in this use case, we currently had to create custom BulkAction to read CSV files and update midPoint user data.
We would be happy if the CSV connector natively supports such use cases, so I have created this pull request.
How does this option work?
When
groupByEnabledis enabled, then the executeQuery method is called, it will perform grouping by the value of the Uid and map it as a JSON string to a special attribute,__RAW_JSON__. For example, in the CSVjohnexample above, the following JSON string is mapped. Note that other attributes are mapped from the first row of the grouped rows.Example inbound mapping on midPoint side
Here is an example of inbound mapping of a CSV resource definition that searches for OrgType by the value of the dept column in the CSV and assigns that organization. The value of the title column of the CSV is set to the
subtypeof the assignment to express which position in the organization is being assigned.