Bucket eventing - Phase 2

[kaloyan-raev]

No description provided.

[mobyvb]

There is a third option that we can support, which is to use the entitlements service, defined here: https://github.com/storj/storj/tree/main/satellite/entitlements

Basically, it is backed by a generic DB table that has two main columns: scope and features (JSONB, very similar to your "approach A")

Currently, the only "scope" we are using looks like proj-id:<project ID>, and it maps to project-level features related to new bucket placements and pricing. However, when designing the entitlements service, we had possibilities of bucket-level features in mind, including bucket eventing.

Basically, we can add a new scope without modifying the DB schema - e.g. the scope could look like bucket:<project ID>:<bucket name> with feature json like {"eventing": {...}}, and extensible for other bucket-level features that are not yet planned or implemented.

It would require a caching layer to entitlements since this probably needs to be accessed much more frequently than the projects scope.

Anyway, I just want to make sure you are aware of this option, and that the entitlements table is (hopefully) designed to avoid needing a schema update every time we want to add features like this.

[mobyvb]

That was actually my plan here, before I realized that this phase 2 design doc existed 😅 https://github.com/storj/storj-private/issues/1666

[kaloyan-raev]

As you pointed out, it is similar to Approach A. So it has the same disadvantages. If the bucket eventing configuration were simpler, we could use Approach A and add a new column to the bucket_metainfo table, like we've done for versioning and eventing. What would be the entitlements benefit compared to Approach A?

[mobyvb]

What would be the entitlements benefit compared to Approach A?

In my opinion the primary benefits compared to Approach A:

• No DB schema update necessary
• Setting a precedent for enabling/configuring bucket-level features in entitlements table, so that future bucket-related features can also be added the same way without requiring a schema update

If there are not going to be a lot of features like this in the future, it is probably not a huge benefit. However, if we anticipate a lot of similar and unrelated features to be set at the bucket level, how many of these features is it reasonable to add a new column to bucket_metainfos? I might be overthinking it though.

[mobyvb]

Using entitlements is definitely more effort because it requires an additional caching layer and making sure things like bucket deletion are properly handled. So that approach should only be taken if it makes sense from a longer-term strategy perspective, which I tried to outline in my previous comment.

[jenlij]

^ This would be a nice to have in the admin @mobyvb

[mobyvb]

The admin behavior for "entitlements" in the back-office/NG UI is a proof-of-concept for what it could look like. These screenshots are all using the entitlements service+new admin UI

These screenshots represent the features for project-level entitlements:

{"compute_access_token":"dGVzdDI=","new_bucket_placements":[30,31,32],"placement_product_mappings":{"30":10,"31":11,"32":12}}

If the configuration for bucket eventing is also JSON, we can do something really similar, just at the bucket-level.

[kaloyan-raev]

The project-level gating is temporary: we’re limiting bucket eventing to a small group of initial customers. Once we’ve built confidence, we’ll remove the gate and open it to everyone. No need to overthink it.

[jenlij]

This is great, very thorough.