Skip to content

Align test definitions with the Testing Guidelines structure and the published API #34

Description

@hdamker

Problem description

The code/Test_definitions/ folder contains six .feature files that fall into two groups with different structure and different referenced API models, measured against the CAMARA API Testing Guidelines (r3.4) and the published code/API_definitions/multi-point-vpn.yaml.

Group A — multi-point-vpn.feature (aligned)

This file is consistent with the API definition and the Testing Guidelines:

  • Filename follows the single-feature-file convention (api-name), Testing Guidelines "Location of feature files".
  • Background section present, referencing a valid access token and the required scope.
  • Scenario tags follow the @<feature_identifier>_<number>_<detail> format (e.g. @createNetwork_01_success, Testing Guidelines "scenario identifier").
  • Scenarios reference the actual paths (/network, /network/{serviceId}, /assessment), operationIds, schemas (NetworkCreate, NetworkUpdate, AssessmentRequest), and CAMARA error codes (INVALID_ARGUMENT, UNAUTHENTICATED), with success, 400, 401 (and a 404 for GET) scenarios for all five operations.

Group B — creation.feature, modification.feature, query.feature, deletion.feature, assessment.feature

These five files do not follow the Testing Guidelines structure and reference an API model that is not present in multi-point-vpn.yaml:

  • Filenames use action words rather than the <api-name>-<operationId>.feature pattern recommended for per-operation splits (Testing Guidelines "Location of feature files").
  • Scenario tags are free-form without the numeric identifier component, e.g. @VPN_from_assessment_provisioning, @Scale_bandwidth_1000, @Get_VPN_ID, @Delete_non-existent_VPN.
  • Paths differ from the API: /vpns, /vpns/{vpnId}, /assessments (the API defines /network, /network/{serviceId}, /assessment).
  • Identifier differs: vpnId (the API uses serviceId).
  • They reference concepts not defined in the API: topology "hub-and-spoke", x-idempotency-key, lifecycle status values (provisioning, active, reconfiguring, terminating, deleted), assessmentId, availabilityLevel, estimatedFees, tenantId, and list/pagination queries (GET /vpns?status=active&tenantId=...).
  • Scopes used are vpn:read / vpn:write (not defined in the API and not in the CAMARA <api-name>:... scope convention).
  • Error references are lower-case (e.g. error.code "invalid_sla_tier"), where the API uses upper-case CAMARA codes.

Expected action

  • Decide the feature-file granularity (single file vs. one file per operation) and apply the Testing Guidelines filename and scenario-identifier conventions consistently.
  • Reconcile the two groups: the Group B scenarios reference an API model that does not match the published definition. Either rework them against the actual API (paths, identifiers, schemas, scopes, error codes) or remove them, keeping multi-point-vpn.feature as the aligned base.
  • Ensure each operation's mandatory scenarios (success and the documented error responses) are covered once, without conflicting duplicates.

Additional context

The scenarios themselves provide useful coverage of the intended operations; the issue is structural consistency and alignment with the published API definition. Line/section references are by filename within code/Test_definitions/.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions