[Bug]: Yaml writeToBigQuery - java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET

[derrickaw]

What happened?

When running this pipeline:

• pipeline:
  type: chain
  transforms:
  - type: Create
  config:
  elements:
  - {label: "4a", rank: 3, timestamp: "2024-07-14 00:00:00 UTC"}
  - {label: "5a", rank: 4}
  - {label: "6a", rank: 5, timestamp: "2024-07-14T02:00:00.123Z"}
  - type: WriteToBigQuery
  config:
  table: "{BQ_TABLE1}"

E Caused by: java.lang.IllegalArgumentException: Unexpected type_info: TYPEINFO_NOT_SET
E at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:503)
E at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:340)
E at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProtoWithoutNullable(SchemaTranslation.java:500)
E at org.apache.beam.sdk.schemas.SchemaTranslation.fieldTypeFromProto(SchemaTranslation.java:340)
E at org.apache.beam.sdk.schemas.SchemaTranslation.fieldFromProto(SchemaTranslation.java:333)
E at org.apache.beam.sdk.schemas.SchemaTranslation.schemaFromProto(SchemaTranslation.java:303)

This only happens when a field is missing in a row.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[derrickaw]

#35692 merged