[client-v2] Data Centric Tests (Phase 1)

clickhouse-client/README.md

@@ -37,3 +37,89 @@ client.connect("http://localhost/system")
 
 ## Examples
 For more example please check [here](https://github.com/ClickHouse/clickhouse-java/tree/main/examples/client).
+
+## Test Data Generator
+
+The test sources contain a generator of test data sets (`com.clickhouse.client.testdata`) covering boundary,
+edge-case and random values of ClickHouse data types. `GenerateTestDataSet` writes such a data set to a file,
+with column types selected on the command line.
+
+Run it from the repository root with Maven:
+
+```bash
+mvn -q -pl clickhouse-client test-compile exec:java -Dexec.classpathScope=test \
+    -Dexec.mainClass=com.clickhouse.client.testdata.GenerateTestDataSet \
+    -Dexec.args='-o data.sql Int32 "s Nullable(String)" "dt DateTime64(3)"'
+```
+
+Relative output paths are resolved against the directory `mvn` runs in. Types with quoted
+arguments, like `DateTime64(3, 'Asia/Kolkata')`, are easier to pass with plain Java.
+
+After `mvn -pl clickhouse-client test-compile` it can also run directly with Java (the generator only needs the JDK):
+
+```bash
+java -cp clickhouse-client/target/test-classes com.clickhouse.client.testdata.GenerateTestDataSet \
+    -o data.sql Int32 "s Nullable(String)" "dt DateTime64(3, 'Asia/Kolkata')" "m Map(String, Array(UInt8))"
+```
+
+Each column is a ClickHouse column definition `"name Type"`, or just `Type` (named `c1`, `c2`, ...). Every
+data set also has an `id Int32` column with the row number. The number of rows is defined by the generated
+values: the column with most values wins and other columns repeat their values cyclically.
+
+| Option | Description |
+| --- | --- |
+| `-o`, `--output <file>` | output file, stdout by default |
+| `-f`, `--format <format>` | `sql` (default) - `SET` statements for required settings, `CREATE TABLE` and `INSERT`; `values` - rows as SQL tuples only; `tsv` - rows in the `TabSeparated` format, exactly as ClickHouse writes them |
+| `-t`, `--table <name>` | table name for the `sql` format, `test_data` by default |
+| `-s`, `--seed <long>` | random seed; the same seed produces the same data |
+| `-h`, `--help` | print usage and all supported types |
+
+Supported types:
+
+- `Int8`..`Int256`, `UInt8`..`UInt256`, `Float32`, `Float64`, `Bool`
+- `Decimal(P[, S])`, `Decimal32(S)`, `Decimal64(S)`, `Decimal128(S)`, `Decimal256(S)`
+- `String`, `FixedString(N)`, `BinaryString` (not a ClickHouse type: a `String` column with arbitrary bytes)
+- `Date`, `Date32`, `Time`, `Time64(P)`, `DateTime[('tz')]`, `DateTime64(P[, 'tz'])` (timezone defaults to `UTC`)
+- `IPv4`, `IPv6`, `UUID`, `Enum8`, `Enum16` (with built-in constants) or `Enum8('a' = 1, ...)`
+- `Point`, `Ring`, `LineString`, `Polygon`, `MultiLineString`, `MultiPolygon`
+- `Array(T)`, `Tuple(T, ...)`, `Tuple(name T, ...)`, `Map(K, V)`, `Nested(name T, ...)`, `Nullable(T)`, `LowCardinality(T)`
+- `Interval<Unit>` and `Nothing` - cannot be stored in a table, so only with `-f values` or `-f tsv`
+
+The `sql` output can be loaded with `clickhouse client --multiquery < data.sql`.
+
+### Pre-generated data sets
+
+`src/test/resources/datasets` contains a data set for every supported type, with the data in a single
+`value` column. Files are named after the type with words separated by underscores, for example
+`array_nullable_string.*` for `Array(Nullable(String))` and `interval_day.*` for `IntervalDay`. Every data set
+has two files with the same rows:
+
+- `<type>.sql` - `SET` statements for required settings, `CREATE TABLE test_data` and `INSERT ... VALUES`.
+  Interval and `Nothing` types cannot be stored in a table, so for them it is `<type>.txt` with the rows as
+  SQL tuples only.
+- `<type>.tsv` - the rows in the `TabSeparated` format, byte for byte what ClickHouse returns for
+  `SELECT * FROM test_data ORDER BY id FORMAT TSV` after running `<type>.sql`. Use it to verify how a client
+  reads or writes the values.
+
+The TSV text follows the server defaults, for example `output_format_decimal_trailing_zeros = 0`, and was
+verified with ClickHouse 26.10. Older versions write some values differently, for example 25.8 and 26.3 write
+`DateTime64` values after 2299 in a non-UTC timezone with a wrong offset. In Java the same text is available
+from `TestDataSet.getTsv()`.
+
+In `.sql` files floats are written as their bits, for example `reinterpretAsFloat64(toUInt64(4607182418800017408))`
+for `1`, and maps with the `map()` function: older ClickHouse versions parse float literals inexactly, so this is
+the only way to load the same values into every supported version.
+
+`client-v2` reads every data set with each binary format reader and verifies all reader accessors, see
+`com.clickhouse.client.datatypes.readers.AbstractBinaryReaderDataSetTest`.
+
+To regenerate them after changing the generators, or to add a type to the `STORABLE_TYPES` or
+`VALUES_ONLY_TYPES` lists in `GenerateTestDataSet` (client tests use the same lists), run:
+
+```bash
+clickhouse-client/scripts/generate-datasets.sh              # builds test classes first
+clickhouse-client/scripts/generate-datasets.sh --skip-build # uses already compiled test classes
+```
+
+The script replaces all `.sql`, `.txt` and `.tsv` files in the directory. With the default seed the output is
+reproducible, so a diff of the regenerated files shows exactly what changed in the generated values.

clickhouse-client/scripts/generate-datasets.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+#
+# Generates a test data set for every type supported by GenerateTestDataSet into
+# clickhouse-client/src/test/resources/datasets, one file per type:
+#   <type>.sql  - SET statements for required settings, CREATE TABLE and INSERT (storable types)
+#   <type>.txt  - rows as SQL tuples only (Interval and Nothing types, which cannot be stored in a table)
+#   <type>.tsv  - the same rows in the TabSeparated format, exactly as ClickHouse writes them, for example
+#                 the output of `SELECT * FROM test_data ORDER BY id FORMAT TSV`
+#
+# Usage: clickhouse-client/scripts/generate-datasets.sh [--skip-build] [--seed <long>]
+#   --skip-build  do not run `mvn test-compile`, use already compiled test classes
+#   --seed        random seed passed to the generator, its default seed otherwise
+
+set -euo pipefail
+
+MODULE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+ROOT_DIR="$(cd "${MODULE_DIR}/.." && pwd)"
+OUTPUT_DIR="${MODULE_DIR}/src/test/resources/datasets"
+CLASSES_DIR="${MODULE_DIR}/target/test-classes"
+MAIN_CLASS="com.clickhouse.client.testdata.GenerateTestDataSet"
+
+SKIP_BUILD=false
+SEED_ARGS=()
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --skip-build) SKIP_BUILD=true; shift ;;
+        --seed) SEED_ARGS=(--seed "$2"); shift 2 ;;
+        -h|--help) sed -n '2,12p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
+        *) echo "Unknown argument: $1" >&2; exit 1 ;;
+    esac
+done
+
+# Array(Nullable(String)) -> array_nullable_string, DateTime64(3, 'UTC') -> date_time64_3_utc,
+# IntervalDay -> interval_day
+file_name() {
+    echo "$1" | sed -e 's/\([a-z0-9]\)\([A-Z]\)/\1_\2/g' | tr '[:upper:]' '[:lower:]' \
+        | sed -e 's/[^a-z0-9]\{1,\}/_/g' -e 's/^_//' -e 's/_$//'
+}
+
+generate() {
+    local type="$1" format="$2" extension="$3"
+    local file
+    file="${OUTPUT_DIR}/$(file_name "${type}").${extension}"
+    java -cp "${CLASSES_DIR}" "${MAIN_CLASS}" "${SEED_ARGS[@]+"${SEED_ARGS[@]}"}" \
+        --format "${format}" --output "${file}" "value ${type}"
+}
+
+if [[ "${SKIP_BUILD}" == false ]]; then
+    (cd "${ROOT_DIR}" && mvn -q -pl clickhouse-client test-compile)
+fi
+
+# Type lists are defined in GenerateTestDataSet, so tests can use the same types
+mapfile -t STORABLE_TYPES < <(java -cp "${CLASSES_DIR}" "${MAIN_CLASS}" --list-types storable)
+mapfile -t VALUES_ONLY_TYPES < <(java -cp "${CLASSES_DIR}" "${MAIN_CLASS}" --list-types values-only)
+
+mkdir -p "${OUTPUT_DIR}"
+rm -f "${OUTPUT_DIR}"/*.sql "${OUTPUT_DIR}"/*.txt "${OUTPUT_DIR}"/*.tsv
+
+for type in "${STORABLE_TYPES[@]}"; do
+    generate "${type}" sql sql
+    generate "${type}" tsv tsv
+done
+for type in "${VALUES_ONLY_TYPES[@]}"; do
+    generate "${type}" values txt
+    generate "${type}" tsv tsv
+done
+
+echo "Generated $(ls "${OUTPUT_DIR}"/*.tsv | wc -l) data sets in ${OUTPUT_DIR}"

clickhouse-client/src/test/java/com/clickhouse/client/testdata/DataTypeGenerator.java

@@ -0,0 +1,80 @@
+package com.clickhouse.client.testdata;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+
+/**
+ * Describes a ClickHouse data type for test data generation: the type expression, the values
+ * of the type as Java objects and how each value is written as a SQL literal.
+ *
+ * <p>Generated values are client independent and use the following Java representation:
+ * <ul>
+ * <li>Int8 - {@code Byte}, UInt8 and Int16 - {@code Short}, UInt16 and Int32 - {@code Integer},
+ * UInt32 and Int64 - {@code Long}, UInt64 and wider integers - {@code BigInteger}</li>
+ * <li>Float32 - {@code Float}, Float64 - {@code Double}, Decimal - {@code BigDecimal} with the type scale</li>
+ * <li>Bool - {@code Boolean}, String and FixedString - {@code String} (binary strings - {@code byte[]}),
+ * FixedString values are always padded with {@code '\0'} to the full length</li>
+ * <li>Date and Date32 - {@code LocalDate}, Time and Time64 - signed {@code Duration},
+ * DateTime and DateTime64 - {@code ZonedDateTime} in the column timezone</li>
+ * <li>IPv4 - {@code Inet4Address}, IPv6 - {@code Inet6Address} (IPv4-mapped addresses stay {@code Inet6Address}),
+ * UUID - {@code UUID}, Enum8 and Enum16 - {@code String} name of the constant</li>
+ * <li>Interval - {@code Period} (day and longer units) or {@code Duration}</li>
+ * <li>Point - {@code double[]}, Ring and LineString - {@code double[][]}, Polygon and MultiLineString -
+ * {@code double[][][]}, MultiPolygon - {@code double[][][][]}</li>
+ * <li>Array, Tuple and Nested - {@code List}, Map - {@code LinkedHashMap}, NULL - {@code null}</li>
+ * </ul>
+ */
+public interface DataTypeGenerator {
+
+    /**
+     * @return ClickHouse type expression, for example {@code Nullable(Decimal(18, 4))}
+     */
+    String getType();
+
+    /**
+     * Generates values of the type. Implementations must only use the given random source so that
+     * the same seed produces the same values.
+     *
+     * @param random source of randomness
+     * @return non-empty list of values, may contain {@code null} for nullable types
+     */
+    List<Object> generate(Random random);
+
+    /**
+     * @param value one of the generated values
+     * @return SQL literal (or constant expression) usable in a {@code VALUES} clause
+     */
+    String toSqlLiteral(Object value);
+
+    /**
+     * Text values are byte strings: each char is one byte (ISO-8859-1), so strings are UTF-8 encoded and
+     * binary strings are written as is.
+     *
+     * @param value one of the generated values
+     * @return value as ClickHouse writes it in a {@code TabSeparated} column: escaped, {@code \N} for {@code null}
+     */
+    String toTsv(Object value);
+
+    /**
+     * @param value one of the generated values
+     * @return value as ClickHouse writes it inside composite types in text formats, for example as an array
+     *         element: strings are quoted, {@code NULL} for {@code null}; a byte string like {@link #toTsv}
+     */
+    String toQuotedText(Object value);
+
+    /**
+     * @return server settings required to create or insert into a column of this type
+     */
+    default Map<String, String> getRequiredSettings() {
+        return Collections.emptyMap();
+    }
+
+    /**
+     * @return {@code false} when the type cannot be used as a table column (Interval, Nothing)
+     */
+    default boolean isStorable() {
+        return true;
+    }
+}