MDEV-37662: Regression

The binary log could be corrupted when committing a large transaction
(i.e. one whose data exceeds the binlog_cache_size limit and spills
into a tmp file) in binlog_format=row if the server's --tmp-dir is
full. The corruption that happens is only the GTID of the errored
transaction would be written into the binary log, without any
body/finalizing events.  This would happen because the content of the
transaction wasn't flushed at the proper time, and the transaction's
binlog cache data was not durable while trying to copy the content
from the binlog cache file into the binary log itself. While switching
the tmp file from a WRITE_CACHE to a READ_CACHE, the server would see
there is still data to flush in the cache, and first try to flush it.
This is not a valid time to flush that data to the temporary file
though, as the GTID event has already been written directly to the
binary log.  So if this flushing fails, it leaves the binary log in a
corrupted state.

The flush itself is expected to happen in
THD::binlog_flush_pending_rows_event(). However, if there is no pending
event, the flush is skipped.

Author: bnestere

mysql-test/suite/rpl/t/rpl_row_binlog_tmp_file_flush_enospc.test

@@ -0,0 +1,62 @@
+#
+# This test ensures that a large transaction (i.e. one whose data exceeds the
+# binlog_cache_size limit and spills into a tmp file) that errors while
+# flushing the tmp file (e.g. due to no space on disk) does not corrupt the
+# binlog, and that the server/replication can continue working normally after
+# the error. To simulate the error, the test uses debug_dbug to inject a
+# failure in the my_write() function, which is called when flushing the tmp
+# file.
+#
+# References:
+#   MDEV-37662: Binlog Corruption When tmpdir is Full
+#
+--source include/have_debug.inc
+--source include/have_innodb.inc
+--source include/have_binlog_format_row.inc
+--source include/master-slave.inc
+
+--connection master
+set @old_binlog_cache_size= @@global.binlog_cache_size;
+set @@global.binlog_cache_size=4096;
+
+--echo #
+--echo # Initialize test data
+--connection master
+create table t1 (a int, b longtext default NULL) engine=innodb;
+
+--echo #
+--echo # Create transaction with cache data larger than the binlog_cache_size
+--echo # so it spills into a tmp file, then simulate ENOSPC while flushing
+--echo # the tmp file.
+--echo #
+set @@session.debug_dbug="+d,simulate_binlog_tmp_file_no_space_left_on_flush";
+--error 3
+insert into t1 values (2, repeat("y", 8192));
+set @@session.debug_dbug="";
+
+--echo #
+--echo # Create another transaction to make sure the server/replication can
+--echo # continue working normally after the error
+--echo #
+insert into t1 values (3, repeat("z", 8192));
+--source include/save_master_gtid.inc
+--connection slave
+--source include/sync_with_master_gtid.inc
+
+--let $diff_tables=master:test.t1,slave:test.t1
+--source include/diff_tables.inc
+
+--echo #
+--echo # Cleanup
+--connection master
+drop table t1;
+--source include/save_master_gtid.inc
+
+--connection slave
+--source include/sync_with_master_gtid.inc
+
+--connection master
+set @@global.binlog_cache_size= @old_binlog_cache_size;
+
+--source include/rpl_end.inc
+--echo # End of rpl_row_binlog_tmp_file_flush_enospc.test

mysys/my_write.c

@@ -57,6 +57,7 @@ size_t my_write(File Filedes, const uchar *Buffer, size_t Count, myf MyFlags)
                      if (!errors) {
                        errno= ENOSPC;
                        writtenbytes= (size_t) -1;
+                       MyFlags&= ~MY_WAIT_IF_FULL;
                      });
 
     if (writtenbytes == Count)

sql/log.cc

@@ -7708,8 +7708,12 @@ int MYSQL_BIN_LOG::write_cache(THD *thd, IO_CACHE *cache)
   DBUG_ENTER("MYSQL_BIN_LOG::write_cache");
 
   mysql_mutex_assert_owner(&LOCK_log);
+  DBUG_EXECUTE_IF("simulate_binlog_tmp_file_no_space_left_on_flush",
+                  { DBUG_SET("+d,simulate_file_write_error"); });
   if (reinit_io_cache(cache, READ_CACHE, 0, 0, 0))
     DBUG_RETURN(ER_ERROR_ON_WRITE);
+  DBUG_EXECUTE_IF("simulate_binlog_tmp_file_no_space_left_on_flush",
+                  { DBUG_SET("-d,simulate_file_write_error"); });
   size_t length= my_b_bytes_in_cache(cache), group, carry, hdr_offs;
   size_t val;
   size_t end_log_pos_inc= 0; // each event processed adds BINLOG_CHECKSUM_LEN 2 t