NullPointerException in populateRequestMetadata

[ICDI0906]

Upcoming End-of-Support

• I acknowledge the upcoming end-of-support for AWS SDK for Java v1 was announced, and migration to AWS SDK for Java v2 is recommended.

Describe the bug

i use spark engine to write s3 gateway. when each partition size is large, about 200M, it report
2024-10-31T19:22:34,559 WARN RenameOperation: While completing all active copies

org.apache.hadoop.fs.s3a.AWSClientIOException: copyFile(pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt, pressure_export_20/_temporary/0/task_202410311920521760879093253987703_0006_m_000008/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt) on pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt: com.amazonaws.AmazonClientException: Unable to complete transfer: null: Unable to complete transfer: null
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:208) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:117) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:320) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:412) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:316) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:291) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:3764) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1100(S3AFileSystem.java:225) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:1691) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySourceAndUpdateTracker(RenameOperation.java:636) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.lambda$initiateCopy$1(RenameOperation.java:600) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:62) ~[hadoop-aws-3.3.1.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:196) ~[hadoop-client-api-3.3.1-ldfs4.5.9.3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: com.amazonaws.AmazonClientException: Unable to complete transfer: null
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:286) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:265) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyImpl.waitForCopyResult(CopyImpl.java:67) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CopyOutcome.waitForCopy(CopyOutcome.java:72) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFile$19(S3AFileSystem.java:3780) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:115) ~[hadoop-aws-3.3.1.jar:?]
	... 15 more
Caused by: java.lang.NullPointerException
	at com.amazonaws.services.s3.AmazonS3Client.populateRequestMetadata(AmazonS3Client.java:4490) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.initiateMultipartUpload(AmazonS3Client.java:3800) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.initiateMultipartUpload(CopyCallable.java:250) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.copyInParts(CopyCallable.java:168) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.call(CopyCallable.java:130) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:132) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:43) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
	... 3 more
2024-10-31T19:22:34,568 ERROR SparkHadoopMapRedUtil: Error committing the output of task: attempt_202410311920521760879093253987703_0006_m_000008_32
org.apache.hadoop.fs.s3a.AWSClientIOException: copyFile(pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt, pressure_export_20/_temporary/0/task_202410311920521760879093253987703_0006_m_000008/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt) on pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt: com.amazonaws.AmazonClientException: Unable to complete transfer: null: Unable to complete transfer: null
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:208) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:117) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:320) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:412) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:316) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:291) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:3764) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1100(S3AFileSystem.java:225) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:1691) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySourceAndUpdateTracker(RenameOperation.java:636) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.lambda$initiateCopy$1(RenameOperation.java:600) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:62) ~[hadoop-aws-3.3.1.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:196) ~[hadoop-client-api-3.3.1-ldfs4.5.9.3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: com.amazonaws.AmazonClientException: Unable to complete transfer: null
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:286) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:265) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyImpl.waitForCopyResult(CopyImpl.java:67) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CopyOutcome.waitForCopy(CopyOutcome.java:72) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFile$19(S3AFileSystem.java:3780) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:115) ~[hadoop-aws-3.3.1.jar:?]
	... 15 more
Caused by: java.lang.NullPointerException
	at com.amazonaws.services.s3.AmazonS3Client.populateRequestMetadata(AmazonS3Client.java:4490) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.initiateMultipartUpload(AmazonS3Client.java:3800) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.initiateMultipartUpload(CopyCallable.java:250) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.copyInParts(CopyCallable.java:168) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.call(CopyCallable.java:130) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:132) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:43) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
	... 3 more
2024-10-31T19:22:34,626 ERROR Utils: Aborting task
org.apache.hadoop.fs.s3a.AWSClientIOException: copyFile(pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt, pressure_export_20/_temporary/0/task_202410311920521760879093253987703_0006_m_000008/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt) on pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt: com.amazonaws.AmazonClientException: Unable to complete transfer: null: Unable to complete transfer: null
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:208) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:117) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:320) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:412) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:316) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:291) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:3764) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1100(S3AFileSystem.java:225) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:1691) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySourceAndUpdateTracker(RenameOperation.java:636) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.lambda$initiateCopy$1(RenameOperation.java:600) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:62) ~[hadoop-aws-3.3.1.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:196) ~[hadoop-client-api-3.3.1-ldfs4.5.9.3.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: com.amazonaws.AmazonClientException: Unable to complete transfer: null
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:286) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:265) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyImpl.waitForCopyResult(CopyImpl.java:67) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CopyOutcome.waitForCopy(CopyOutcome.java:72) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFile$19(S3AFileSystem.java:3780) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:115) ~[hadoop-aws-3.3.1.jar:?]
	... 15 more
Caused by: java.lang.NullPointerException
	at com.amazonaws.services.s3.AmazonS3Client.populateRequestMetadata(AmazonS3Client.java:4490) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.initiateMultipartUpload(AmazonS3Client.java:3800) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.initiateMultipartUpload(CopyCallable.java:250) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.copyInParts(CopyCallable.java:168) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.call(CopyCallable.java:130) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:132) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:43) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
	... 3 more
2024-10-31T19:22:34,647 WARN FileOutputCommitter: Could not delete s3a://minimax-infra/pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32
2024-10-31T19:22:34,649 ERROR FileFormatWriter: Job job_202410311920521760879093253987703_0006 aborted.
2024-10-31T19:22:34,652 ERROR Executor: Exception in task 8.0 in stage 6.0 (TID 32)
org.apache.spark.SparkException: Task failed while writing rows.
	at org.apache.spark.sql.errors.QueryExecutionErrors$.taskFailedWhileWritingRowsError(QueryExecutionErrors.scala:655) ~[spark-catalyst_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:358) ~[spark-sql_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$22(FileFormatWriter.scala:266) ~[spark-sql_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) ~[spark-core_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.scheduler.Task.run(Task.scala:136) ~[spark-core_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) ~[spark-core_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1505) ~[spark-core_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) ~[spark-core_2.12-3.3.1-ldspark-1.0-SNAPSHOT.jar:3.3.1-ldspark-1.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: org.apache.hadoop.fs.s3a.AWSClientIOException: copyFile(pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt, pressure_export_20/_temporary/0/task_202410311920521760879093253987703_0006_m_000008/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt) on pressure_export_20/_temporary/0/_temporary/attempt_202410311920521760879093253987703_0006_m_000008_32/part-00008-387db23c-765c-46ca-a53b-3e5e78d5d96d-c000.txt: com.amazonaws.AmazonClientException: Unable to complete transfer: null: Unable to complete transfer: null
	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:208) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:117) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:320) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:412) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:316) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:291) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:3764) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1100(S3AFileSystem.java:225) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:1691) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySourceAndUpdateTracker(RenameOperation.java:636) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.lambda$initiateCopy$1(RenameOperation.java:600) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:62) ~[hadoop-aws-3.3.1.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:196) ~[hadoop-client-api-3.3.1-ldfs4.5.9.3.jar:?]
	... 3 more
Caused by: com.amazonaws.AmazonClientException: Unable to complete transfer: null
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:286) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:265) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyImpl.waitForCopyResult(CopyImpl.java:67) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CopyOutcome.waitForCopy(CopyOutcome.java:72) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$copyFile$19(S3AFileSystem.java:3780) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:115) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:320) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:412) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:316) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:291) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.copyFile(S3AFileSystem.java:3764) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem.access$1100(S3AFileSystem.java:225) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.S3AFileSystem$OperationCallbacksImpl.copyFile(S3AFileSystem.java:1691) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.copySourceAndUpdateTracker(RenameOperation.java:636) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.RenameOperation.lambda$initiateCopy$1(RenameOperation.java:600) ~[hadoop-aws-3.3.1.jar:?]
	at org.apache.hadoop.fs.s3a.impl.CallableSupplier.get(CallableSupplier.java:62) ~[hadoop-aws-3.3.1.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$RunnableWithPermitRelease.run(SemaphoredDelegatingExecutor.java:196) ~[hadoop-client-api-3.3.1-ldfs4.5.9.3.jar:?]
	... 3 more
Caused by: java.lang.NullPointerException
	at com.amazonaws.services.s3.AmazonS3Client.populateRequestMetadata(AmazonS3Client.java:4490) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.AmazonS3Client.initiateMultipartUpload(AmazonS3Client.java:3800) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.initiateMultipartUpload(CopyCallable.java:250) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.copyInParts(CopyCallable.java:168) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyCallable.call(CopyCallable.java:130) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:132) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at com.amazonaws.services.s3.transfer.internal.CopyMonitor.call(CopyMonitor.java:43) ~[aws-java-sdk-bundle-1.12.777.jar:?]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

write s3 success

Current Behavior

nullpointException

Reproduction Steps

use spark to write s3 gateway

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

1.12.777

JDK version used

not known

Operating System and version

not known

[debora-ito]

@ICDI0906 the NPE is coming from line 4490:

aws-sdk-java/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java

Lines 4486 to 4490 in df2ce8a

	protected static void populateRequestMetadata(Request<?> request, ObjectMetadata metadata) {
	Map<String, Object> rawMetadata = metadata.getRawMetadata();
	if (rawMetadata != null) {
	for (Entry<String, Object> entry : rawMetadata.entrySet()) {
	request.addHeader(entry.getKey(), entry.getValue().toString());

How's the request metadata being populated? Can you provide a code sample?
Have you ruled out any issues with hadoop fs?

[github-actions]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.