Skip to content

Commit 02a99cf

Browse files
authored
Fix "Expression tree is too large" for strings with large number of NULL bytes (#274)
In SQLite, it is generally difficult to work with `NULL` bytes in strings, because SQLite uses C-style string syntax, which means it considers the first `NULL` byte as the string terminator in some scenarios. While it is possible to store and read `NULL` bytes within strings (with some limitations), there is no way to express the `NULL` byte in a string literal. It needs to be done in a different way: 1. Concatenation: `str_part1 || CHAR(0) || str_part2 || …` 2. Passing as a parameter to a prepared statement. 3. Using HEX notation and casting to string. More details: https://sqlite.org/nulinstr.html At the moment, the SQLite driver uses concatenation (option 1), which seems to fail for strings that contain many `NULL` bytes: ``` SQLSTATE[HY000]: General error: 1 Expression tree is too large (maximum depth 1000) ``` This PR fixes this by using HEX encoding.
1 parent 581d361 commit 02a99cf

File tree

2 files changed

+28
-11
lines changed

2 files changed

+28
-11
lines changed

tests/WP_SQLite_Driver_Tests.php

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3882,6 +3882,22 @@ public function testNullCharactersInStrings(): void {
38823882
);
38833883
}
38843884

3885+
public function testLargeNumberOfNullCharactersInStrings(): void {
3886+
$this->assertQuery( 'CREATE TABLE t (value TEXT)' );
3887+
3888+
$long_string_with_null_bytes = str_repeat( "abcdef\0xyz", 1000 );
3889+
3890+
$this->assertQuery(
3891+
sprintf(
3892+
"INSERT INTO t (value) VALUES ('%s')",
3893+
$long_string_with_null_bytes
3894+
)
3895+
);
3896+
3897+
$result = $this->assertQuery( 'SELECT value FROM t' );
3898+
$this->assertSame( $long_string_with_null_bytes, $result[0]->value );
3899+
}
3900+
38853901
public function testColumnDefaults(): void {
38863902
$this->assertQuery(
38873903
"

wp-includes/sqlite-ast/class-wp-sqlite-driver.php

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3339,7 +3339,7 @@ private function translate_string_literal( WP_Parser_Node $node ): string {
33393339
$value = $token->get_value();
33403340

33413341
/*
3342-
* 5. Translate datetime literals.
3342+
* Translate datetime literals.
33433343
*
33443344
* Process only strings that could possibly represent a datetime
33453345
* literal ("YYYY-MM-DDTHH:MM:SS", "YYYY-MM-DDTHH:MM:SSZ", etc.).
@@ -3349,7 +3349,7 @@ private function translate_string_literal( WP_Parser_Node $node ): string {
33493349
}
33503350

33513351
/*
3352-
* 6. Handle null characters.
3352+
* Handle null characters.
33533353
*
33543354
* SQLite doesn't fully support null characters (\u0000) in strings.
33553355
* However, it can store them and read them, with some limitations.
@@ -3358,25 +3358,26 @@ private function translate_string_literal( WP_Parser_Node $node ): string {
33583358
* Removing them would damage the serialized data.
33593359
*
33603360
* There is no way to store null bytes using a string literal, so we
3361-
* need to split the string and concatenate null bytes with its parts.
3361+
* need to pass the value as a HEX string and cast it back to TEXT.
33623362
* This will convert literals will null bytes to expressions.
33633363
*
33643364
* Alternatively, we could replace string literals with parameters and
33653365
* pass them using prepared statements. However, that's not universally
33663366
* applicable for all string literals (e.g., in default column values).
33673367
*
3368+
* We can't use the "part1 || CHAR(0) || part2 || ..." syntax, because
3369+
* with a large number of null bytes, SQLite throws the following error:
3370+
*
3371+
* SQLSTATE[HY000]:
3372+
* General error: 1 Expression tree is too large (maximum depth 1000)
3373+
*
33683374
* See:
33693375
* https://www.sqlite.org/nulinstr.html
33703376
*/
3371-
$parts = array();
3372-
foreach ( explode( "\0", $value ) as $segment ) {
3373-
// Escape and quote each segment.
3374-
$parts[] = "'" . str_replace( "'", "''", $segment ) . "'";
3375-
}
3376-
if ( count( $parts ) > 1 ) {
3377-
return '(' . implode( ' || CHAR(0) || ', $parts ) . ')';
3377+
if ( strpos( $value, "\0" ) !== false ) {
3378+
return sprintf( "CAST(x'%s' AS TEXT)", bin2hex( $value ) );
33783379
}
3379-
return $parts[0];
3380+
return $this->connection->quote( $value );
33803381
}
33813382

33823383
/**

0 commit comments

Comments
 (0)