diff --git a/.doc_gen/metadata/s3_metadata.yaml b/.doc_gen/metadata/s3_metadata.yaml index 107bb09b428..b6ff0280b5a 100644 --- a/.doc_gen/metadata/s3_metadata.yaml +++ b/.doc_gen/metadata/s3_metadata.yaml @@ -294,6 +294,11 @@ s3_CopyObject: snippet_tags: - python.example_code.s3.helper.ObjectWrapper - python.example_code.s3.CopyObject + - description: Copy an object using a conditional request. + genai: some + snippet_tags: + - python.example_code.s3.helper.S3ConditionalRequests + - python.example_code.s3.CopyObjectConditional Ruby: versions: - sdk_version: 3 @@ -872,6 +877,11 @@ s3_GetObject: snippet_tags: - python.example_code.s3.helper.ObjectWrapper - python.example_code.s3.GetObject + - description: Get an object using a conditional request. + genai: some + snippet_tags: + - python.example_code.s3.helper.S3ConditionalRequests + - python.example_code.s3.GetObjectConditional JavaScript: versions: - sdk_version: 3 @@ -1510,6 +1520,11 @@ s3_PutObject: snippet_tags: - python.example_code.s3.helper.ObjectWrapper - python.example_code.s3.PutObject + - description: Upload an object using a conditional request. + genai: some + snippet_tags: + - python.example_code.s3.helper.S3ConditionalRequests + - python.example_code.s3.PutObjectConditional Rust: versions: - sdk_version: 1 @@ -3456,6 +3471,28 @@ s3_Scenario_DeleteAllObjects: - javascriptv3/example_code/s3/scenarios/delete-all-objects.js services: s3: {DeleteObjects, ListObjectsV2} +s3_Scenario_ConditionalRequests: + title: Make &S3; conditional requests using an &AWS; SDK. + title_abbrev: Make conditional requests + synopsis: add preconditions to &S3; requests. + category: Scenarios + languages: + Python: + versions: + - sdk_version: 3 + github: python/example_code/s3/scenarios/conditional_requests + sdkguide: + excerpts: + - description: Run an interactive scenario demonstrating &S3; conditional requests. + genai: some + snippet_tags: + - python.example_code.s3.S3ConditionalRequests.scenario + - description: A wrapper class that defines the conditional request operations. + genai: some + snippet_tags: + - python.example_code.s3.S3ConditionalRequests.wrapper + services: + s3: {GetObject, PutObject, CopyObject} s3_Scenario_ExpressBasics: title: Learn the basics of Amazon S3 Express One Zone with an &AWS; SDK title_abbrev: Learn the basics of S3 Express One Zone diff --git a/python/example_code/s3/README.md b/python/example_code/s3/README.md index 6fe9a747f63..b5568cded93 100644 --- a/python/example_code/s3/README.md +++ b/python/example_code/s3/README.md @@ -88,6 +88,7 @@ functions within the same service. - [Create an Amazon Textract explorer application](../../cross_service/textract_explorer) - [Detect entities in text extracted from an image](../../cross_service/textract_comprehend_notebook) - [Detect objects in images](../../cross_service/photo_analyzer) +- [Make conditional requests](scenarios/conditional_requests/scenario.py) - [Manage versioned objects in batches with a Lambda function](../../example_code/s3/s3_versioning) - [Upload or download large files](file_transfer/file_transfer.py) - [Work with versioned objects](s3_versioning/versioning.py) @@ -190,6 +191,24 @@ This example shows you how to build an app that uses Amazon Rekognition to detec +#### Make conditional requests + +This example shows you how to add preconditions to Amazon S3 requests. + + + + + +Start the example by running the following at a command prompt: + +``` +python scenarios/conditional_requests/scenario.py +``` + + + + + #### Manage versioned objects in batches with a Lambda function This example shows you how to manage versioned S3 objects in batches with a Lambda function. diff --git a/python/example_code/s3/requirements.txt b/python/example_code/s3/requirements.txt index a963ef0d1bb..29d321d5e40 100644 --- a/python/example_code/s3/requirements.txt +++ b/python/example_code/s3/requirements.txt @@ -1,3 +1,3 @@ -boto3>=1.34.4 +boto3>=1.35.49 pytest>=7.2.1 requests>=2.28.2 diff --git a/python/example_code/s3/scenarios/conditional_requests/README.md b/python/example_code/s3/scenarios/conditional_requests/README.md new file mode 100644 index 00000000000..f9bac8447c1 --- /dev/null +++ b/python/example_code/s3/scenarios/conditional_requests/README.md @@ -0,0 +1,53 @@ + +# Amazon S3 Conditional Requests Feature Scenario for the SDK for Python (boto3) + +## Overview + +This example demonstrates how to use the AWS SDK for Python (boto3) to work with Amazon Simple Storage Service (Amazon S3) conditional request features. The scenario demonstrates how to add preconditions to S3 operations, and how those operations will succeed or fail based on the conditional requests. + +[Amazon S3 Conditional Requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-requests.html) are used to add preconditions to S3 read, copy, or write requests. + +## ⚠ Important + +- Running this code might result in charges to your AWS account. For more details, see [AWS Pricing](https://aws.amazon.com/pricing/) and [Free Tier](https://aws.amazon.com/free/). +- Running the tests might result in charges to your AWS account. +- We recommend that you grant your code least privilege. At most, grant only the minimum permissions required to perform the task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege). +- This code is not tested in every AWS Region. For more information, see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services). + +## Code examples + +### Prerequisites + +To run these examples, you need: + +- Python 3.x installed. +- Run `python pip install -r requirements.txt` +- AWS credentials configured. For more information, see [Configuring the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html). + +### Scenario + +This example uses a feature scenario to demonstrate various aspects of S3 conditional requests. The scenario is divided into three stages: + +1. **Setup**: Create test buckets and objects. +2. **Conditional Reads and Writes**: Explore S3 conditional requests by listing objects, attempting to read or write with conditional requests, and viewing request results. +3. **Clean**: Delete all objects and buckets. + +#### Running the scenario +To run this feature scenario, run the command below from this directory: + +``` +python scenario.py +``` + + +## Additional resources + +- [Amazon S3 Developer Guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-requests.html) +- [Amazon S3 API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) +- [boto3 Amazon S3 reference](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html) + +--- + +© Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: Apache-2.0 diff --git a/python/example_code/s3/scenarios/conditional_requests/s3_conditional_requests.py b/python/example_code/s3/scenarios/conditional_requests/s3_conditional_requests.py new file mode 100644 index 00000000000..9f746429174 --- /dev/null +++ b/python/example_code/s3/scenarios/conditional_requests/s3_conditional_requests.py @@ -0,0 +1,142 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +# snippet-start:[python.example_code.s3.S3ConditionalRequests.wrapper] + +import boto3 +import logging + +from botocore.exceptions import ClientError + +# Configure logging +logger = logging.getLogger(__name__) + + +# snippet-start:[python.example_code.s3.helper.S3ConditionalRequests] +class S3ConditionalRequests: + """Encapsulates S3 conditional request operations.""" + + def __init__(self, s3_client): + self.s3 = s3_client + + @classmethod + def from_client(cls): + """ + Instantiates this class from a Boto3 client. + """ + s3_client = boto3.client("s3") + return cls(s3_client) + + # snippet-end:[python.example_code.s3.helper.S3ConditionalRequests] + + # snippet-start:[python.example_code.s3.GetObjectConditional] + + def get_object_conditional( + self, + object_key: str, + source_bucket: str, + condition_type: str, + condition_value: str, + ): + """ + Retrieves an object from Amazon S3 with a conditional request. + + :param object_key: The key of the object to retrieve. + :param source_bucket: The source bucket of the object. + :param condition_type: The type of condition: 'IfMatch', 'IfNoneMatch', 'IfModifiedSince', 'IfUnmodifiedSince'. + :param condition_value: The value to use for the condition. + """ + try: + response = self.s3.get_object( + Bucket=source_bucket, + Key=object_key, + **{condition_type: condition_value}, + ) + sample_bytes = response["Body"].read(20) + print( + f"\tConditional read successful. Here are the first 20 bytes of the object:\n" + ) + print(f"\t{sample_bytes}") + except ClientError as e: + error_code = e.response["Error"]["Code"] + if error_code == "PreconditionFailed": + print("\tConditional read failed: Precondition failed") + elif error_code == "304": # Not modified error code. + print("\tConditional read failed: Object not modified") + else: + logger.error(f"Unexpected error: {error_code}") + raise + + # snippet-end:[python.example_code.s3.GetObjectConditional] + + # snippet-start:[python.example_code.s3.PutObjectConditional] + + def put_object_conditional(self, object_key: str, source_bucket: str, data: bytes): + """ + Uploads an object to Amazon S3 with a conditional request. Prevents overwrite + using an IfNoneMatch condition for the object key. + + :param object_key: The key of the object to upload. + :param source_bucket: The source bucket of the object. + :param data: The data to upload. + """ + try: + self.s3.put_object( + Bucket=source_bucket, Key=object_key, Body=data, IfNoneMatch="*" + ) + print( + f"\tConditional write successful for key {object_key} in bucket {source_bucket}." + ) + except ClientError as e: + error_code = e.response["Error"]["Code"] + if error_code == "PreconditionFailed": + print("\tConditional write failed: Precondition failed") + else: + logger.error(f"Unexpected error: {error_code}") + raise + + # snippet-end:[python.example_code.s3.PutObjectConditional] + + # snippet-start:[python.example_code.s3.CopyObjectConditional] + def copy_object_conditional( + self, + source_key: str, + dest_key: str, + source_bucket: str, + dest_bucket: str, + condition_type: str, + condition_value: str, + ): + """ + Copies an object from one Amazon S3 bucket to another with a conditional request. + + :param source_key: The key of the source object to copy. + :param dest_key: The key of the destination object. + :param source_bucket: The source bucket of the object. + :param dest_bucket: The destination bucket of the object. + :param condition_type: The type of condition to apply, e.g. + 'CopySourceIfMatch', 'CopySourceIfNoneMatch', 'CopySourceIfModifiedSince', 'CopySourceIfUnmodifiedSince'. + :param condition_value: The value to use for the condition. + """ + try: + self.s3.copy_object( + Bucket=dest_bucket, + Key=dest_key, + CopySource={"Bucket": source_bucket, "Key": source_key}, + **{condition_type: condition_value}, + ) + print( + f"\tConditional copy successful for key {dest_key} in bucket {dest_bucket}." + ) + except ClientError as e: + error_code = e.response["Error"]["Code"] + if error_code == "PreconditionFailed": + print("\tConditional copy failed: Precondition failed") + elif error_code == "304": # Not modified error code. + print("\tConditional copy failed: Object not modified") + else: + logger.error(f"Unexpected error: {error_code}") + raise + + # snippet-end:[python.example_code.s3.CopyObjectConditional] +# snippet-end:[python.example_code.s3.S3ConditionalRequests.wrapper] diff --git a/python/example_code/s3/scenarios/conditional_requests/scenario.py b/python/example_code/s3/scenarios/conditional_requests/scenario.py new file mode 100644 index 00000000000..c96a9ac1f58 --- /dev/null +++ b/python/example_code/s3/scenarios/conditional_requests/scenario.py @@ -0,0 +1,299 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +# snippet-start:[python.example_code.s3.S3ConditionalRequests.scenario] +""" +Purpose + +Shows how to use AWS SDK for Python (Boto3) to get started using conditional requests for +Amazon Simple Storage Service (Amazon S3). + +""" + +import logging +import random +import sys +import datetime + +import boto3 +from botocore.exceptions import ClientError + +from s3_conditional_requests import S3ConditionalRequests + +# Add relative path to include demo_tools in this code example without need for setup. +sys.path.append("../../../..") +import demo_tools.question as q # noqa + +# Constants +FILE_CONTENT = "This is a test file for S3 conditional requests." +RANDOM_SUFFIX = str(random.randint(100, 999)) + +logger = logging.getLogger(__name__) + + +class ConditionalRequestsScenario: + """Runs a scenario that shows how to use S3 Conditional Requests.""" + + def __init__(self, conditional_requests, s3_client): + """ + :param conditional_requests: An object that wraps S3 conditional request actions. + :param s3_client: A Boto3 S3 client for setup and cleanup operations. + """ + self.conditional_requests = conditional_requests + self.s3_client = s3_client + + # snippet-start:[python.example_code.s3.SetupScenario] + def setup_scenario(self, source_bucket: str, dest_bucket: str, object_key: str): + """ + Sets up the scenario by creating a source and destination bucket. + Prompts the user to provide a bucket name prefix. + + :param source_bucket: The name of the source bucket. + :param dest_bucket: The name of the destination bucket. + :param object_key: The name of a test file to add to the source bucket. + """ + + # Create the buckets. + try: + self.s3_client.create_bucket(Bucket=source_bucket) + self.s3_client.create_bucket(Bucket=dest_bucket) + print( + f"Created source bucket: {source_bucket} and destination bucket: {dest_bucket}" + ) + except ClientError as e: + error_code = e.response["Error"]["Code"] + logger.error(f"Error creating buckets: {error_code}") + raise + + # Upload test file into the source bucket. + try: + print(f"Uploading file {object_key} to bucket {source_bucket}") + response = self.s3_client.put_object( + Bucket=source_bucket, Key=object_key, Body=FILE_CONTENT + ) + object_etag = response["ETag"] + return object_etag + + except Exception as e: + logger.error( + f"Failed to upload file {object_key} to bucket {source_bucket}: {e}" + ) + + # snippet-end:[python.example_code.s3.SetupScenario] + + # snippet-start:[python.example_code.s3.CleanupScenario] + def cleanup_scenario(self, source_bucket: str, dest_bucket: str): + """ + Cleans up the scenario by deleting the source and destination buckets. + + :param source_bucket: The name of the source bucket. + :param dest_bucket: The name of the destination bucket. + """ + self.cleanup_bucket(source_bucket) + self.cleanup_bucket(dest_bucket) + + def cleanup_bucket(self, bucket_name: str): + """ + Cleans up the bucket by deleting all objects and then the bucket itself. + + :param bucket_name: The name of the bucket. + """ + try: + # Get list of all objects in the bucket. + list_response = self.s3_client.list_objects_v2(Bucket=bucket_name) + objs = list_response.get("Contents", []) + for obj in objs: + key = obj["Key"] + self.s3_client.delete_object(Bucket=bucket_name, Key=key) + self.s3_client.delete_bucket(Bucket=bucket_name) + print(f"Cleaned up bucket: {bucket_name}.") + except ClientError as e: + error_code = e.response["Error"]["Code"] + if error_code == "NoSuchBucket": + logger.info(f"Bucket {bucket_name} does not exist, skipping cleanup.") + else: + logger.error(f"Error deleting bucket: {error_code}") + raise + + # snippet-end:[python.example_code.s3.CleanupScenario] + + def display_buckets(self, source_bucket: str, dest_bucket: str): + """ + Display a list of the objects in the test buckets. + + :param source_bucket: The name of the source bucket. + :param dest_bucket: The name of the destination bucket. + """ + self.list_bucket_contents(source_bucket) + self.list_bucket_contents(dest_bucket) + + def list_bucket_contents(self, bucket_name): + """ + Display a list of the objects in the bucket. + + :param bucket_name: The name of the bucket. + """ + try: + # Get list of all objects in the bucket. + print(f"\t Items in bucket {bucket_name}") + list_response = self.s3_client.list_objects_v2(Bucket=bucket_name) + objs = list_response.get("Contents", []) + if not objs: + print("\t\tNo objects found.") + for obj in objs: + key = obj["Key"] + print(f"\t\t object: {key} ETag {obj['ETag']}") + return objs + except ClientError as e: + error_code = e.response["Error"]["Code"] + if error_code == "NoSuchBucket": + logger.info(f"Bucket {bucket_name} does not exist.") + else: + logger.error(f"Error listing bucket and objects: {error_code}") + raise + + # snippet-start:[python.example_code.s3.DisplayMenu] + + def display_menu( + self, source_bucket: str, dest_bucket: str, object_key: str, etag: str + ): + """ + Displays the menu of conditional request options for the user. + + :param source_bucket: The name of the source bucket. + :param dest_bucket: The name of the destination bucket. + :param object_key: The key of the test object in the source bucket. + :param etag: The etag of the test object in the source bucket. + """ + + actions = [ + "Print list of bucket items.", + "Perform a conditional read.", + "Perform a conditional copy.", + "Perform a conditional write.", + "Clean up and exit.", + ] + + conditions = [ + "If-Match: using the object's ETag. This condition should succeed.", + "If-None-Match: using the object's ETag. This condition should fail.", + "If-Modified-Since: using yesterday's date. This condition should succeed.", + "If-Unmodified-Since: using yesterday's date. This condition should fail.", + ] + + condition_types = [ + "IfMatch", + "IfNoneMatch", + "IfModifiedSince", + "IfUnmodifiedSince", + ] + copy_condition_types = [ + "CopySourceIfMatch", + "CopySourceIfNoneMatch", + "CopySourceIfModifiedSince", + "CopySourceIfUnmodifiedSince", + ] + + yesterday_date = datetime.datetime.utcnow() - datetime.timedelta(days=1) + + choice = 0 + while choice != 4: + print("-" * 88) + print("Choose an action to explore some example conditional requests.") + choice = q.choose("Which action would you like to take? ", actions) + if choice == 0: + print("Listing the objects and buckets.") + self.display_buckets(source_bucket, dest_bucket) + elif choice == 1: + print("Perform a conditional read.") + condition_type = q.choose("Enter the condition type : ", conditions) + if condition_type == 0 or condition_type == 1: + self.conditional_requests.get_object_conditional( + object_key, source_bucket, condition_types[condition_type], etag + ) + elif condition_type == 2 or condition_type == 3: + self.conditional_requests.get_object_conditional( + object_key, + source_bucket, + condition_types[condition_type], + yesterday_date, + ) + elif choice == 2: + print("Perform a conditional copy.") + condition_type = q.choose("Enter the condition type : ", conditions) + dest_key = q.ask("Enter an object key: ", q.non_empty) + if condition_type == 0 or condition_type == 1: + self.conditional_requests.copy_object_conditional( + object_key, + dest_key, + source_bucket, + dest_bucket, + copy_condition_types[condition_type], + etag, + ) + elif condition_type == 2 or condition_type == 3: + self.conditional_requests.copy_object_conditional( + object_key, + dest_key, + copy_condition_types[condition_type], + yesterday_date, + ) + elif choice == 3: + print( + "Perform a conditional write using IfNoneMatch condition on the object key." + ) + print("If the key is a duplicate, the write will fail.") + object_key = q.ask("Enter an object key: ", q.non_empty) + self.conditional_requests.put_object_conditional( + object_key, source_bucket, b"Conditional write example data." + ) + elif choice == 4: + print("Proceeding to cleanup.") + + # snippet-end:[python.example_code.s3.DisplayMenu] + + def run_scenario(self): + """ + Runs the interactive scenario. + """ + print("-" * 88) + print("Welcome to the Amazon S3 conditional requests example.") + print("-" * 88) + + print( + f"""\ + This example demonstrates the use of conditional requests for S3 operations. + You can use conditional requests to add preconditions to S3 read requests to return or copy + an object based on its Entity tag (ETag), or last modified date. + You can use a conditional write requests to prevent overwrites by ensuring + there is no existing object with the same key. + + This example will allow you to perform conditional reads + and writes that will succeed or fail based on your selected options. + + Sample buckets and a sample object will be created as part of the example. + """ + ) + + bucket_prefix = q.ask("Enter a bucket name prefix: ", q.non_empty) + source_bucket_name = f"{bucket_prefix}-source-{RANDOM_SUFFIX}" + dest_bucket_name = f"{bucket_prefix}-dest-{RANDOM_SUFFIX}" + object_key = "test-upload-file.txt" + + try: + etag = self.setup_scenario(source_bucket_name, dest_bucket_name, object_key) + self.display_menu(source_bucket_name, dest_bucket_name, object_key, etag) + finally: + self.cleanup_scenario(source_bucket_name, dest_bucket_name) + + print("-" * 88) + print("Thanks for watching.") + print("-" * 88) + + +if __name__ == "__main__": + scenario = ConditionalRequestsScenario( + S3ConditionalRequests.from_client(), boto3.client("s3") + ) + scenario.run_scenario() +# snippet-end:[python.example_code.s3.S3ConditionalRequests.scenario] diff --git a/python/example_code/s3/scenarios/conditional_requests/test/conftest.py b/python/example_code/s3/scenarios/conditional_requests/test/conftest.py new file mode 100644 index 00000000000..9856180a92d --- /dev/null +++ b/python/example_code/s3/scenarios/conditional_requests/test/conftest.py @@ -0,0 +1,12 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +""" +Contains common test fixtures used to run unit tests. +""" + +import sys + +# This is needed so Python can find test_tools on the path. +sys.path.append("../../../..") +from test_tools.fixtures.common import * diff --git a/python/example_code/s3/scenarios/conditional_requests/test/test_conditional_requests.py b/python/example_code/s3/scenarios/conditional_requests/test/test_conditional_requests.py new file mode 100644 index 00000000000..759694c7262 --- /dev/null +++ b/python/example_code/s3/scenarios/conditional_requests/test/test_conditional_requests.py @@ -0,0 +1,24 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +import pytest + +import boto3 + +from s3_conditional_requests import S3ConditionalRequests +from scenario import ConditionalRequestsScenario + + +@pytest.mark.integ +def test_run_conditional_requests_scenario_integ(input_mocker, capsys): + conditional_requests = S3ConditionalRequests.from_client() + scenario = ConditionalRequestsScenario(conditional_requests, boto3.client("s3")) + + input_mocker.mock_answers( + ["cr-integ-tests", "1", "2", "1", "3", "1", "test-copy", "4", "test-write", "5"] + ) + + scenario.run_scenario() + + capt = capsys.readouterr() + assert "Thanks for watching." in capt.out diff --git a/workflows/s3_conditional_requests/README.md b/workflows/s3_conditional_requests/README.md new file mode 100644 index 00000000000..7eb52dcbe57 --- /dev/null +++ b/workflows/s3_conditional_requests/README.md @@ -0,0 +1,32 @@ +# Amazon S3 Conditional Requests Feature Scenario + +## Overview + +This example shows how to use AWS SDKs to work with Amazon Simple Storage Service (Amazon S3) conditional request features. The scenario demonstrates how to add preconditions to S3 operations, and how those operations will succeed or fail based on the conditional requests. + +[Amazon S3 Conditional Requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-requests.html) are used to add preconditions to S3 read, copy, or write requests. + +This scenario demonstrates the following steps and tasks: +1. Create test buckets and objects. +2. Use preconditions with S3 read and copy operations. +3. Use preconditions with S3 write operations to prevent overwrites. +4. Delete the objects and buckets. + +### Resources + +The scenario steps create the buckets and objects needed for the example. No additional resources are required. + +## Implementations + +This example is implemented in the following languages: + +- [Python](../../python/example_code/s3/scenarios/conditional_requests/README.md) + +## Additional reading + +- [S3 Conditional Reads](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-reads.html) +- [S3 Conditional Writes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-writes.html) + +--- + +Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: Apache-2.0 diff --git a/workflows/s3_conditional_requests/SPECIFICATION.md b/workflows/s3_conditional_requests/SPECIFICATION.md new file mode 100644 index 00000000000..4f38abb3998 --- /dev/null +++ b/workflows/s3_conditional_requests/SPECIFICATION.md @@ -0,0 +1,243 @@ +# Amazon S3 Conditional Requests Feature Scenario - Technical specification + +This document contains the technical specifications for _Amazon S3 Conditional Requests Feature Scenario_, +a feature scenario that showcases AWS services and SDKs. It is primarily intended for the AWS code +examples team to use while developing this example in additional languages. + +This document explains the following: + +- Architecture and features of the example scenario. +- Metadata information for the scenario. +- Sample reference output. + +For an introduction, see the [README.md](README.md). + +--- + +### Table of contents + +- [Resources and User Input](#resources-and-user-input) +- [Errors](#errors) +- [Metadata](#metadata) + +## Resources and User Input + +- Amazon Simple Storage Service (Amazon S3) Buckets (created in the scenario). + - One bucket as the source bucket. + - One bucket as the target bucket. + - One test file in the source bucket. +Bucket names will begin with a prefix provided by the user. + +Example: +``` +---------------------------------------------------------------------------------------- +Welcome to the Amazon S3 conditional requests example. +---------------------------------------------------------------------------------------- +This example demonstrates the use of conditional requests for S3 operations. +You can use conditional requests to add preconditions to S3 read requests to return or copy +an object based on its Entity tag (ETag), or last modified date. +You can use a conditional write requests to prevent overwrites by ensuring +there is no existing object with the same key. + +This example will allow you to perform conditional reads +and writes that will succeed or fail based on your selected options. + +Sample buckets and a sample object will be created as part of the example. + +Enter a bucket name prefix: test555 +Created source bucket: test555-source-279 and destination bucket: test555-dest-279 +Uploading file test-upload-file.txt to bucket test555-source-279 + +``` + +- Conditional Requests + - In order to cover all the example topics in the S3 guide section, the scenario covers write, copy, or read operations with a precondition using the SDK. + - The user can choose the type of precondition to add. + - This section should provide a menu of options to the user, so they can observe the results of their conditional request. + - Known exceptions due to the conditional operations are expected and should not end the scenario. + +- Menu options + - Print list of bucket items + - Iterate through buckets and objects, printing each one with their corresponding ETag. + - Conditional Read + - Provide the following options, and print the results of the operation. + - If-Match: using the object's ETag. This condition should succeed. + - If-None-Match: using the object's ETag. This condition should fail. + - If-Modified-Since: using yesterday's date. This condition should succeed. + - If-Unmodified-Since: using yesterday's date. This condition should fail. + - Conditional Copy + - Request a new key for the copied item. Copy it to the destination bucket. Print the results. + - Provide the following options, and print the results of the operation. + - If-Match: using the object's ETag. This condition should succeed. + - If-None-Match: using the object's ETag. This condition should fail. + - If-Modified-Since: using yesterday's date. This condition should succeed. + - If-Unmodified-Since: using yesterday's date. This condition should fail. + - Conditional Write + - Request a key for the new object. + - Attempt the write with an If-None-Match condition. + - If it is a duplicate, the operation will fail. Print the results. + +Example +``` +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? + +-------------------------------------------------------------------------------- + +Which action would you like to take? 1 +Listing the objects and buckets. + Items in bucket test555-source-279 + object: test-upload-file.txt ETag "3e3d5f53cec929a350af061a39a3a19d" + Items in bucket test555-dest-279 + No objects found. +-------------------------------------------------------------------------------- +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 2 +Perform a conditional read. +1. If-Match: using the object's ETag. This condition should succeed. +2. If-None-Match: using the object's ETag. This condition should fail. +3. If-Modified-Since: using yesterday's date. This condition should succeed. +4. If-Unmodified-Since: using yesterday's date. This condition should fail. +Enter the condition type : 1 + Conditional read successful. Here are the first 20 bytes of the object: + + b'This is a test file ' +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 2 +Perform a conditional read. +1. If-Match: using the object's ETag. This condition should succeed. +2. If-None-Match: using the object's ETag. This condition should fail. +3. If-Modified-Since: using yesterday's date. This condition should succeed. +4. If-Unmodified-Since: using yesterday's date. This condition should fail. +Enter the condition type : 2 + Conditional read failed: Object not modified +---------------------------------------------------------------------------------------- +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 3 +Perform a conditional copy. +1. If-Match: using the object's ETag. This condition should succeed. +2. If-None-Match: using the object's ETag. This condition should fail. +3. If-Modified-Since: using yesterday's date. This condition should succeed. +4. If-Unmodified-Since: using yesterday's date. This condition should fail. +Enter the condition type : 1 +Enter an object key: test44 + Conditional copy successful for key test44 in bucket test555-dest-279. +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 3 +Perform a conditional copy. +1. If-Match: using the object's ETag. This condition should succeed. +2. If-None-Match: using the object's ETag. This condition should fail. +3. If-Modified-Since: using yesterday's date. This condition should succeed. +4. If-Unmodified-Since: using yesterday's date. This condition should fail. +Enter the condition type : 2 +Enter an object key: test44 + Conditional copy failed: Precondition failed +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 4 +Perform a conditional write using IfNoneMatch condition on the object key. +If the key is a duplicate, the write will fail. +Enter an object key: test44 + Conditional write successful for key test44 in bucket test555-source-279. +---------------------------------------------------------------------------------------- +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 4 +Perform a conditional write using IfNoneMatch condition on the object key. +If the key is a duplicate, the write will fail. +Enter an object key: test44 + Conditional write failed: Precondition failed +---------------------------------------------------------------------------------------- + +``` +- Cleanup + - The scenario should get the full list of objects, and remove all objects before deleting the buckets. + - The user should be notified if the delete operation cannot occur. + - If any previous operation should fail unexpectedly, perform the cleanup operation. + +Example: + +``` +Choose an action to explore some example conditional requests. +1. Print list of bucket items. +2. Perform a conditional read. +3. Perform a conditional copy. +4. Perform a conditional write. +5. Clean up and exit. +Which action would you like to take? 5 +Cleaned up bucket: test555-source-279. +Cleaned up bucket: test555-dest-279. +---------------------------------------------------------------------------------------- +Thanks for watching. +---------------------------------------------------------------------------------------- + +Process finished with exit code 0 + +``` + +--- + +## Errors +The PreconditionFailed exceptions are part of the flow of this scenario. After a success or failure, +the user can print the contents of the buckets to see the result. + +| action | Error | Handling | +|--------------|-----------------------|--------------------------------------------| +| `GetObject` | PreconditionFailed | Notify the user and do not print contents. | +| `GetObject` | ObjectNotModified 304 | Notify the user and do not print contents. | +| `CopyObject` | PreconditionFailed | Notify the user of the failure. | +| `CopyObject` | ObjectNotModified 304 | Notify the user of the failure. | +| `PutObject` | PreconditionFailed | Notify the user of the failure. | + + +--- + +## Metadata +For languages which already have an entry for the action, add a description for the snippet describing the conditional request options. + +| action / scenario | metadata file | metadata key | +|------------------------------------|------------------|---------------------------------| +| `GetObject` | s3_metadata.yaml | s3_GetObject | +| `CopyObject` | s3_metadata.yaml | s3_CopyObject | +| `PutObject` | s3_metadata.yaml | s3_PutObject | +| `S3 Conditional Requests Scenario` | s3_metadata.yaml | s3_Scenario_ConditionalRequests | +