feat(storage): add Object Contexts support to GCS metadata and listing

[thiyaguk09]

Description

Please provide a detailed description for the change.
As much as possible, please try to keep changes separate by purpose. For example, try not to make a one-line bug fix in a feature request, or add an irrelevant README change to a bug fix.

Impact

What's the impact of this change?

Testing

Have you added unit and integration tests if necessary?
Were any tests changed? Are any breaking changes necessary?

Additional Information

Any additional details that we should be aware of?

Checklist

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease
• Appropriate docs were updated
• Appropriate comments were added, particularly in complex areas or places that require background
• No new warnings or issues will be generated from this change

Fixes #issue_number_goes_here 🦕

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates support for Google Cloud Storage Object Contexts into the client library. This feature allows for more granular, structured metadata to be associated with objects, which can then be used for advanced filtering during object listing. The changes encompass modifications to core file and bucket operations to handle contexts, alongside robust validation and extensive testing to ensure correct behavior and data integrity.

Highlights

• Object Contexts Support: Introduced comprehensive support for Google Cloud Storage Object Contexts, allowing users to attach custom key-value pairs to objects for enhanced metadata management.
• Metadata Operations: Enabled the creation, retrieval, update, and deletion of Object Contexts during file save, copy, combine, and setMetadata operations.
• Listing and Filtering: Added the ability to filter object listings based on Object Contexts, supporting queries for presence, absence, existence, and non-existence of specific context keys or key-value pairs.
• Input Validation: Implemented validation to prevent the use of forbidden characters (double quotes) in Object Context keys and values, ensuring compatibility with GCS filter string syntax.
• System and Unit Tests: Expanded test coverage with new system and unit tests to verify the functionality of Object Contexts across various scenarios, including creation, updates, copying, combining, and filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• src/bucket.ts
  • Imported validateContexts and ContextValue for context handling.
  • Extended GetFilesOptions with a filter property to support context-based listing.
  • Added contexts property to CombineOptions interface.
  • Integrated validateContexts call within the combine method to ensure valid context input.
  • Updated combine method to pass contexts to the destination object.
• src/file.ts
  • Imported validateContexts for context validation.
  • Extended CopyOptions and FileMetadata interfaces to include contexts properties.
  • Defined the ContextValue interface to represent individual context entries.
  • Added validateContexts calls to copy, save, and setMetadata methods to validate context data before API calls.
• src/util.ts
  • Imported FileMetadata for type definitions.
  • Introduced a new validateContexts utility function to check for forbidden double quotes in context keys and values.
• system-test/storage.ts
  • Added a new describe block for 'object contexts' system tests.
  • Included tests for creating, retrieving, and updating object contexts.
  • Verified that server-generated createTime and updateTime are returned for contexts.
  • Added tests for clearing all custom contexts from an object.
  • Implemented tests for copy/rewrite operations, ensuring contexts are inherited or overridden correctly.
  • Added tests for combine operations, verifying context inheritance and overriding for composed objects.
  • Introduced comprehensive tests for listing objects using context filters, covering key-value presence, absence, existence, non-existence, and handling of quoted keys.
• test/bucket.ts
  • Modified combine test to explicitly include contexts: undefined in the expected destination.
  • Added unit tests for getFiles to verify correct filter parameter construction for various context filtering scenarios.
  • Included a unit test to confirm that contexts are correctly parsed and included in the returned File metadata.
• test/file.ts
  • Added a new describe block for 'Object Contexts' unit tests.
  • Included tests for creating new objects with contexts, including Unicode characters and validation for invalid characters.
  • Added tests for updating existing objects, covering full replacement, partial updates, individual context removal, and clearing all contexts.
  • Implemented tests for copying objects with context overrides.
  • Added tests for composing objects and passing contexts to the destination.
  • Verified handling of empty string values within contexts.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[thiyaguk09]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for Object Contexts in GCS, a significant new feature. The changes are comprehensive, touching on file and bucket operations for creation, updates, copying, combining, and listing with context-based filters. The implementation is accompanied by a good set of unit and system tests. My review includes a few suggestions to improve TypeScript type definitions for correctness and to enhance documentation and code consistency. Overall, this is a well-executed feature addition.

_{Note: Security Review did not run due to the size of the PR.}

[gemini-code-assist]

Code Review

This pull request introduces support for Object Contexts in GCS metadata and listing, updating Bucket and File classes to handle contexts in combine, copy, save, and setMetadata operations, and adding a validateContexts utility function to ensure context keys and values do not contain forbidden characters. A critical security vulnerability has been identified: the context validation to prevent filter injection is not consistently applied across all relevant methods. Specifically, createWriteStream and createResumableUpload are missing this crucial validation, which could allow malicious contexts to be set and exploited for filter injection. Addressing this inconsistency is essential for the robustness of the new feature. Additionally, minor issues include an inconsistent coding style in error handling and an incorrect JSDoc comment in the new utility function.