Skip to content

Is big Decimal in Parquet big endian? #3246

@4ertus2

Description

@4ertus2

Describe the usage question you have. Please include as many useful details as possible.

It's not clear from description https://parquet.apache.org/docs/file-format/data-pages/encodings/ how are big Decimals (precision >= 18) placed. As I could understand they can be encoded as BYTE_ARRAY or FIXED_LEN_BYTE_ARRAY. But there're two ways to place "the bytes contained in the array" from memory: big endian and little endian. I.e. they are in LE in int128 in memory. Could I place the bytes into FIXED_LEN_BYTE_ARRAY as is?

It's also not clean if there's some encoding over these decimal's bytes, should the data be swapped to correct endianness before of after the encoding?

Component(s)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions