Merge branch 'refs/heads/support-gpp-usnat-v2' into functional-tests/support-gpp-usnat-v2

# Conflicts:
#	src/test/groovy/org/prebid/server/functional/tests/privacy/GppCookieSyncSpec.groovy
#	src/test/groovy/org/prebid/server/functional/tests/privacy/GppSyncUserActivitiesSpec.groovy
#	src/test/groovy/org/prebid/server/functional/tests/privacy/GppTransmitUfpdActivitiesSpec.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/GppConsent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/UsUtV1Consent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/UsVaV1Consent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/v1/UsCaV1Consent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/v1/UsCoV1Consent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/v1/UsCtV1Consent.groovy
#	src/test/groovy/org/prebid/server/functional/util/privacy/gpp/v1/UsNatV1Consent.groovy

Author: osulzhenko

.github/workflows/pr-java-ci.yml

@@ -23,7 +23,7 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: Set up JDK
-        uses: actions/setup-java@v3
+        uses: actions/setup-java@v4
         with:
           distribution: 'temurin'
           cache: 'maven'

.github/workflows/scripts/codepath-notification

@@ -17,3 +17,10 @@
 # The aim is to find a minimal set of regex patterns that matches any file in these paths
 
 /ix|Ix|ix.json|ix.yaml: [email protected]
+appnexus|Appnexus: [email protected]
+pubmatic|Pubmatic: [email protected]
+openx|OpenX: [email protected]
+medianet|Medianet: [email protected]
+thetradedesk|TheTradeDesk: [email protected]
+gumgum|GumGum: [email protected]
+kargo|Kargo: [email protected]

.mvn/wrapper/maven-wrapper.jar

@@ -6,13 +6,14 @@
 # "License"); you may not use this file except in compliance
 # with the License.  You may obtain a copy of the License at
 #
-#   https://www.apache.org/licenses/LICENSE-2.0
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing,
 # software distributed under the License is distributed on an
 # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.6/apache-maven-3.8.6-bin.zip
-wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.1/maven-wrapper-3.1.1.jar
+wrapperVersion=3.3.2
+distributionType=only-script
+distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.9/apache-maven-3.9.9-bin.zip

.mvn/wrapper/maven-wrapper.properties

@@ -1,4 +1,4 @@
-FROM amazoncorretto:21
+FROM amazoncorretto:21.0.8-al2023
 
 WORKDIR /app/prebid-server
 

Dockerfile

@@ -1,4 +1,4 @@
-FROM amazoncorretto:21
+FROM amazoncorretto:21.0.8-al2023
 
 WORKDIR /app/prebid-server
 

Dockerfile-modules

@@ -42,6 +42,7 @@ Follow next steps to create JAR file which can be deployed locally.
 
 - Install prerequsites
   - Java SDK: Oracle's or Corretto. Let us know if there's a distribution PBS-Java doesn't work with.
+  - Java SDK Version: 21
   - Maven
 
 - Clone the project:

README.md

@@ -12,6 +12,7 @@ There are two ways to configure application settings: database and file. This do
 - `auction.truncate-target-attr` - Maximum targeting attributes size. Values between 1 and 255.
 - `auction.default-integration` - Default integration to assume.
 - `auction.debug-allow` - enables debug output in the auction response. Default `true`.
+- `auction.impression-limit` - a max number of impressions allowed for the auction, impressions that exceed this limit will be dropped, 0 means no limit.
 - `auction.bid-validations.banner-creative-max-size` - Overrides creative max size validation for banners. Valid values
   are:
     - "skip": don't do anything about creative max size for this publisher
@@ -25,6 +26,11 @@ There are two ways to configure application settings: database and file. This do
 - `auction.bidadjustments.mediatype.*.*.*[].value` - value of the bid adjustment
 - `auction.bidadjustments.mediatype.*.*.*[].currency` - currency of the bid adjustment
 - `auction.events.enabled` - enables events for account if true
+- `auction.bid-rounding` - bid rounding options are:
+  - **down** - rounding down to the lower price bucket
+  - **up** - rounding up to the higher price bucket
+  - **timesplit** - 50% of the time rounding down to the lower PB and 50% of the time rounding up to the higher price bucket
+  - **true** - if the price >= 50% of the range, rounding up to the higher price bucket, otherwise rounding down
 - `auction.price-floors.enabled` - enables price floors for account if true. Defaults to true.
 - `auction.price-floors.fetch.enabled`- enables data fetch for price floors for account if true. Defaults to false.
 - `auction.price-floors.fetch.url` - url to fetch price floors data from.
@@ -50,6 +56,7 @@ Keep in mind following restrictions:
 - `auction.preferredmediatype.<bidder>.<media-type>` - <media-type> that will be left for <bidder> that doesn't support multi-format. Other media types will be removed. Acceptable values: `banner`, `video`, `audio`, `native`.
 - `auction.privacysandbox.cookiedeprecation.enabled` - boolean that turns on setting and reading of the Chrome Privacy Sandbox testing label header. Defaults to false.
 - `auction.privacysandbox.cookiedeprecation.ttlsec` - if the above setting is true, how long to set the receive-cookie-deprecation cookie's expiration
+- `auction.cache.enabled` - enables bids caching for account if true. Defaults to true.
 - `privacy.gdpr.enabled` - enables gdpr verifications if true. Has higher priority than configuration in
   application.yaml.
 - `privacy.gdpr.eea-countries` - overrides the host-level list of 2-letter country codes where TCF processing is applied
@@ -277,8 +284,8 @@ The general idea is that you'll place all the account-specific settings in a sep
 ```yaml
 settings:
   s3:
-      accessKeyId: <S3 access key>
-      secretAccessKey: <S3 access key>
+      accessKeyId: <S3 access key> # optional
+      secretAccessKey: <S3 access key> #optional
       endpoint: <endpoint> # http://s3.storage.com
       bucket: <bucket name> # prebid-application-settings
       region: <region name> # if not provided AWS_GLOBAL will be used. Example value: 'eu-central-1'
@@ -298,6 +305,14 @@ settings:
           timeout: 5000
 ```
 
+If `accessKeyId` and `secretAccessKey` are not specified in the Prebid Server configuration  then AWS credentials will be looked up in this order:
+- Java System Properties - `aws.accessKeyId` and `aws.secretAccessKey`
+- Environment Variables - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
+- Web Identity Token credentials from system properties or environment variables
+- Credential profiles file at the default location (`~/.aws/credentials`) shared by all AWS SDKs and the AWS CLI
+- Credentials delivered through the Amazon EC2 container service if "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable,
+- Instance profile credentials delivered through the Amazon EC2 metadata service
+
 ### File format
 
 We recommend using the `json` format for your account configuration. A minimal configuration may look like this.

docs/application-settings.md

@@ -14,6 +14,7 @@ This section can be extended against standard [Spring configuration](https://doc
 This parameter exists to allow to change the location of the directory Vert.x will create because it will and there is no way to make it not.
 - `vertx.init-timeout-ms` - time to wait for asynchronous initialization steps completion before considering them stuck. When exceeded - exception is thrown and Prebid Server stops.
 - `vertx.enable-per-client-endpoint-metrics` - enables HTTP client metrics per destination endpoint (`host:port`)
+- `vertx.round-robin-inet-address` - enables round-robin inet address selection of the ip address to use
 
 ## Server
 - `server.max-headers-size` - set the maximum length of all headers.
@@ -96,6 +97,7 @@ Removes and downloads file again if depending service cant process probably corr
 - `auction.cache.expected-request-time-ms` - approximate value in milliseconds for Cache Service interacting.
 - `auction.cache.only-winning-bids` - if equals to `true` only the winning bids would be cached. Has lower priority than request-specific flags.
 - `auction.generate-bid-id` - whether to generate seatbid[].bid[].ext.prebid.bidid in the OpenRTB response.
+- `auction.enforce-random-bid-id` - whether to enforce generating a robust random seatbid[].bid[].id in the OpenRTB response if the initial value is less than 17 characters.
 - `auction.validations.banner-creative-max-size` - enables creative max size validation for banners. Possible values: `skip`, `enforce`, `warn`. Default is `skip`.
 - `auction.validations.secure-markup` - enables secure markup validation. Possible values: `skip`, `enforce`, `warn`. Default is `skip`.
 - `auction.host-schain-node` - defines global schain node that will be appended to `request.source.ext.schain.nodes` passed to bidders
@@ -283,8 +285,12 @@ For `JVM` metrics
 - `cache.scheme` - set the external Cache Service protocol: `http`, `https`, etc.
 - `cache.host` - set the external Cache Service destination in format `host:port`.
 - `cache.path` - set the external Cache Service path, for example `/cache`.
+- `cache.internal.scheme` - set the internal Cache Service protocol: `http`, `https`, etc., the internal scheme get priority over the external one when provided.
+- `cache.internal.host` - set the internal Cache Service destination in format `host:port`, the internal port get priority over the external one when provided.
+- `cache.internal.path` - set the internal Cache Service path, for example `/cache`, the internal path get priority over the external one when provided.
 - `storage.pbc.enabled` - If set to true, this will allow storing modules’ data in third-party storage.
 - `storage.pbc.path` - set the external Cache Service path for module caching, for example `/pbc-storage`.
+- `cache.api-key-secured` - if set to `true`, will cause Prebid Server to add a special API key header to Prebid Cache requests.
 - `pbc.api.key` - set the external Cache Service api key for secured calls.
 - `cache.query` - appends to the cache path as query string params (used for legacy Auction requests).
 - `cache.banner-ttl-seconds` - how long (in seconds) banner will be available via the external Cache Service.
@@ -294,6 +300,7 @@ for particular publisher account. Overrides `cache.banner-ttl-seconds` property.
 - `cache.account.<ACCOUNT>.video-ttl-seconds` - how long (in seconds) video creative will be available in Cache Service 
 for particular publisher account. Overrides `cache.video-ttl-seconds` property.
 - `cache.default-ttl-seconds.{banner, video, audio, native}` - a default value how long (in seconds) a creative of the specific type will be available in Cache Service
+- `cache.append-trace-info-to-cache-id` - if set to `true`, causes the addition account ID and datacenter to cache UUID: _ACCOUNT-DATACENTER-remainderOfUUID_. Implies that cache UUID will be generated by the Prebid Server. 
 
 ## Application settings (account configuration, stored ad unit configurations, stored requests)
 Preconfigured application settings can be obtained from multiple data sources consequently: 
@@ -303,6 +310,10 @@ Preconfigured application settings can be obtained from multiple data sources co
 
 Warning! Application will not start in case of no one data source is defined and you'll get an exception in logs.
 
+For requests validation mode available next options:
+- `settings.fail-on-unknown-bidders` - fail with validation error or just make warning for unknown bidders.
+- `settings.fail-on-disabled-bidders` - fail with validation error or just make warning for disabled bidders.
+
 For filesystem data source available next options:
 - `settings.filesystem.settings-filename` - location of file settings.
 - `settings.filesystem.stored-requests-dir` - directory with stored requests.
@@ -335,9 +346,10 @@ For HTTP data source available next options:
 - `settings.http.amp-endpoint` - the url to fetch AMP stored requests.
 - `settings.http.video-endpoint` - the url to fetch video stored requests.
 - `settings.http.category-endpoint` - the url to fetch categories for long form video.
+- `settings.http.rfc3986-compatible` - if equals to `true` the url will be build according to RFC 3986, `false` by default
 
 For account processing rules available next options:
-- `settings.enforce-valid-account` - if equals to `true` then request without account id will be rejected with 401.
+- `settings.enforce-valid-account` - if equals to `true` then request without account id will be rejection with 401.
 - `settings.generate-storedrequest-bidrequest-id` - overrides `bidrequest.id` in amp or app stored request with generated UUID if true. Default value is false. This flag can be overridden by setting `bidrequest.id` as `{{UUID}}` placeholder directly in stored request.
 
 It is possible to specify default account configuration values that will be assumed if account config have them 
@@ -384,8 +396,8 @@ contain 'WHERE last_updated > ?' for MySQL and 'WHERE last_updated > $1' for Pos
 
 For S3 storage configuration
 - `settings.in-memory-cache.s3-update.refresh-rate` - refresh period in ms for stored request updates in S3
-- `settings.s3.access-key-id` - an access key
-- `settings.s3.secret-access-key` - a secret access key
+- `settings.s3.access-key-id` - an access key (optional)
+- `settings.s3.secret-access-key` - a secret access key (optional)
 - `settings.s3.region` - a region, AWS_GLOBAL by default
 - `settings.s3.endpoint` - an endpoint
 - `settings.s3.bucket` - a bucket name
@@ -395,6 +407,15 @@ For S3 storage configuration
 - `settings.s3.stored-requests-dir` - a directory with stored requests
 - `settings.s3.stored-responses-dir` - a directory with stored responses
 
+If `settings.s3.access-key-id` and `settings.s3.secret-access-key` are not specified in the Prebid Server configuration then AWS credentials will be looked up in this order:
+- Java System Properties - `aws.accessKeyId` and `aws.secretAccessKey`
+- Environment Variables - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
+- Web Identity Token credentials from system properties or environment variables 
+- Credential profiles file at the default location (`~/.aws/credentials`) shared by all AWS SDKs and the AWS CLI
+- Credentials delivered through the Amazon EC2 container service if "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable,
+- Instance profile credentials delivered through the Amazon EC2 metadata service
+
+
 For targeting available next options:
 - `settings.targeting.truncate-attr-chars` - set the max length for names of targeting keywords (0 means no truncation).
 

docs/config-app.md

@@ -0,0 +1,72 @@
+# Porting Guide
+
+## Overview
+
+First, thank you for taking on the migration of an adapter from Go to Java. But really the best way to think of it is not as straight port. Instead, we recommend treat this task as a re-implementation. It will take a few adapters before you fully get the hang of it, and that's okay—everyone goes through a learning curve.
+
+Keep in mind that the PBS-Go team is more lenient about what they allow in adapters compared to the PBS-Java team.
+
+## Pull Request Requirements
+
+We would appreciate it if your porting PR title follows these patterns:
+
+- `Port <adapter_name>: New Adapter` – For porting a completely new adapter to the project (e.g., `Port Kobler: New Adapter`).
+- `Port <adapter_name>: <update_description>` – For porting a specific update to an existing adapter (e.g., `Port OpenX: Native Support`).
+- `Port <alias_name>: New alias for <adapter_name>` – For porting an alias of an existing adapter to the project (e.g., `Port Artechnology: New alias of StartHub`).
+
+Additionally, we kindly ask that you:
+
+- Link any existing GitHub issues that your PR resolves. This ensures the issue will be automatically closed when your PR is merged.
+- Add the label `do not port` to your PR.
+
+## Porting Requirements
+
+1. **Feature Parity**: A Java adapter should have the same functionality as the Go adapter.
+2. **Java Adapter Code Should:**
+    - Follow the code style of the PBS-Java repository (see [the code style page](code-style.md)).
+    - Be well-written Java code: clear, readable, optimized, and following best practices.
+    - Maintain a structure similar to existing adapters (see below).
+3. **The adapter should be covered with tests:**
+    - Unit tests for implementation details.
+    - A simple integration test to ensure the adapter is reachable, can send requests to the bidder, and that its configuration works.
+
+### What does "having a similar structure to existing adapters" mean?
+
+The PBS-Java codebase has evolved over time. While existing adapters may not be perfect and could contain legacy issues (e.g., using outdated Java syntax), they still serve as a valuable reference for learning, inspiration, and even reuse.
+
+Each adapter is unique, but most share common patterns. For example, nearly every adapter includes:
+
+1. **A `makeHttpRequests(...)` method**
+    - Iterates over the `imps` in the bid request:
+        - Parses `imp[].ext.prebid.bidder` (i.e., bidder static parameters).
+        - Modifies the `imp`.
+        - Collects errors encountered during `imp` processing.
+    - Prepares outgoing request(s):
+        - Constructs headers.
+        - Builds the request URL.
+        - Modifies the incoming bid request based on the updated `imps`.
+
+2. **A `makeBids(...)` method**
+    - Parses the `BidResponse`.
+    - Iterates over `seatBids` and `bids`.
+    - Creates a list of `BidderBid` objects.
+
+### Ensuring Structural Consistency
+
+To maintain consistency across adapters:
+- Fit the Go adapter functionality into the Java adapter structure.
+- Use the same or similar method and variable names where applicable.
+- Reuse existing solutions for common functionality (e.g., use `BidderUtil`, `HttpUtil` classes).
+- Ensure unit tests follow a similar structure, with comparable test cases and code patterns.
+
+## Specific Rules and Tips for Porting
+
+1. Begin by determining how the Go adapter's functionality fits into the Java adapter structure.
+2. Go adapters deserialize JSON objects in-place, while Java adapters work with pre-deserialized objects. As a result, many errors thrown in the Go version do not apply in Java.
+3. **No hardcoded URLs.** If an adapter has a "test URL," it must be defined in the YAML file. See `org.prebid.server.spring.config.bidder.NextMillenniumConfiguration.NextMillenniumConfigurationProperties` for an example of how to handle special YAML entries.
+4. The structure of Go and Java bidder configuration files differs—do not copy and paste directly. Pay attention to details such as macros in the endpoint and redirect/iframe URLs.
+5. **Prohibited in bidder adapters:**
+    - Blocking code.
+    - Fully dynamic hostnames in URLs.
+    - Non-thread-safe code (bidder adapters should not store state internally).
+6. If an adapter has no special logic, consider using an alias to `Generic` instead. In this case, there will still need to be an integration test for this bidder. e.g. `src/test/java/org/prebid/server/it/BidderNameTest.java`