diff --git a/CHANGELOG.md b/CHANGELOG.md index 0efaafc25..230770446 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,7 +54,7 @@ * Added support for internal $ref resolution in validation flows. * Fixed issue where parameter resolution was "schema" when "example" was specified. * Add supported formats for schema resolution (deref). -* Fix for [#7643](https://github.com/postmanlabs/postman-app-support/issues/7643), [#7914](https://github.com/postmanlabs/postman-app-support/issues/7914), [#9004](https://github.com/postmanlabs/postman-app-support/issues/9004) - Added support for Auth params in response/example. +* Fix for [#7643](https://github.com/postmanlabs/postman-app-support/issues/7643), [#7914](https://github.com/postmanlabs/postman-app-support/issues/7914), [#9004](https://github.com/postmanlabs/postman-app-support/issues/9004) - Added support for Auth params in response/example. * Bumped up multiple dependecies and dev-dependencies versions to keep them up-to-date. * Updated code coverage tool from deprecated istanbul to nyc. @@ -127,7 +127,7 @@ #### v1.1.13 (April 21, 2020) * Added support for detailed validation body mismatches with option detailedBlobValidation. -* Fix for [#8098](https://github.com/postmanlabs/postman-app-support/issues/8098) - Unable to validate schema with type array. +* Fix for [#8098](https://github.com/postmanlabs/postman-app-support/issues/8098) - Unable to validate schema with type array. * Fixed URIError for invalid URI in transaction. * Fix for [#152](https://github.com/postmanlabs/openapi-to-postman/issues/152) - Path references not resolved due to improver handling of special characters. * Fix for [#160](https://github.com/postmanlabs/openapi-to-postman/issues/160) - Added handling for variables in local servers not a part of a URL segment. All path servers to be added as collection variables. diff --git a/README.md b/README.md index 4c0431f73..afda85bab 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,17 @@ -![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) +# THIS BRANCH IS DEPRECATED, IN FAVOR OF A STANDALONE PACKAGE [Portman](https://github.com/apideck-libraries/portman) + --- -*Supercharge your API workflow.* +![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) + +*Supercharge your API workflow.* *Modern software is built on APIs. Postman helps you develop APIs faster.* # OpenAPI 3.0 to Postman Collection v2.1.0 Converter [![Build Status](https://travis-ci.org/postmanlabs/openapi-to-postman.svg?branch=master)](https://travis-ci.org/postmanlabs/openapi-to-postman) -#### Contents +#### Contents 1. [Getting Started](#getting-started) 2. [Using the converter as a NodeJS module](#using-the-converter-as-a-nodejs-module) @@ -21,6 +24,7 @@ 1. [Options](#options) 2. [Usage](#usage) 4. [Conversion Schema](#conversion-schema) +5. [Postman test suite generation options](/TESTGENERATION.md) --- @@ -99,7 +103,7 @@ Check out complete list of options and their usage at [OPTIONS.md](/OPTIONS.md) ### ConversionResult -- `result` - Flag responsible for providing a status whether the conversion was successful or not +- `result` - Flag responsible for providing a status whether the conversion was successful or not - `reason` - Provides the reason for an unsuccessful conversion, defined only if result: false @@ -158,28 +162,31 @@ The converter can be used as a CLI tool as well. The following [command line opt `openapi2postmanv2 [options]` ### Options -- `-v`, `--version` +- `-v`, `--version` Specifies the version of the converter -- `-s `, `--spec ` +- `-s `, `--spec ` Used to specify the OpenAPI specification (file path) which is to be converted -- `-o `, `--output ` +- `-o `, `--output ` Used to specify the destination file in which the collection is to be written -- `-t`, `--test` +- `-t`, `--test` Used to test the collection with an in-built sample specification -- `-p`, `--pretty` +- `-p`, `--pretty` Used to pretty print the collection object while writing to a file - `-O`, `--options` Used to supply options to the converter, for complete options details see [here](/OPTIONS.md) -- `-c`, `--options-config` +- `-c`, `--options-config` Used to supply options to the converter through config file, for complete options details see [here](/OPTIONS.md) -- `-h`, `--help` +- `-g `, `--generate ` + Used to generate postman tests given the JSON file with test options, for complete options details see [here](/TESTGENERATION.md) + +- `-h`, `--help` Specifies all the options along with a few usage examples on the terminal @@ -203,6 +210,11 @@ $ openapi2postmanv2 -s spec.yaml -o collection.json -p -c ./examples/cli-option $ openapi2postmanv2 --test ``` +- Generating additional postman tests for the OpenAPi specification +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json +``` + ## Conversion Schema | *postman* | *openapi* | *options* | *examples* | diff --git a/TESTGENERATION.md b/TESTGENERATION.md new file mode 100644 index 000000000..ef6260283 --- /dev/null +++ b/TESTGENERATION.md @@ -0,0 +1,565 @@ +## Postman test suite generation options + +This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. The goal is +to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used by the Newman +test runner. + +To generate the tests, define a JSON file like the example (postman-testsuite.json) below and run the CLI with the +--generate option. + +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json +``` + +Current postman-testsuite JSON properties + +```JSON +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" + ] + } + ], + "overwriteRequests": [ + { + "openApiOperationId": "post-accounts", + "overwriteRequestBody": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientId", + "value": "{{$guid}}", + "overwrite": true + } + ] + } + ] +} + +``` + +The JSON test suite format consists out of 5 parts: + +- **version** : which refers the JSON test suite version + (not relevant but might handy for future backward compatibility options). +- **generateTests** : which refers the default available generated postman tests. The default tests are grouped per + type (response, request) + - **responseChecks** : All response basic checks. (For now we have only included response checks). + - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test + will be generated for **all** operations). +- **extendTests**: which refers the custom additions of manual created postman tests. ( + see [Postman test suite extendTests](#postman-test-suite-extendtests)) +- **contentChecks**: which refers the additional Postman tests that check the content. ( + see [Postman test suite contentChecks](#postman-test-suite-contentchecks)) +- **assignPmVariables**: which refers to specific Postman environment variables for easier automation. ( + see [Postman test suite assignPmVariables](#postman-test-suite-assignpmvariables)) +- **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. ( + see [Postman test suite overwriteRequests](#postman-test-suite-overwriterequests)) + +See "postman-testsuite-advanced.json" file for an advanced example of the setting options. + +## Postman test suite properties + +Version 1.0 + +| name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | +|-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| +| Response status success (2xx) check | StatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | +| Response time check | responseTime | boolean | false | enabled | maxMs 300 | Adds the check if the response of the postman request is within a number of ms. | true | TEST GENERATION | +| Response content-type check | contentType | boolean | false | enabled | | Adds the check if the postman response header is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response Schema validation check | schemaValidation | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | +| Response Header presence check | headersPresent | boolean | false | enabled | | Adds the check if the postman response header has the header names present, like defined in the OpenApi spec. | true | TEST GENERATION | + +## Postman test suite extendTests + +The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. Anything added +in `tests` array, will be added to the postman test scripts. + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended +- **tests (Array)** : Array of additional postman test scripts. +- **responseChecks (array)** : Extends the generateTests `responseChecks` ( + see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. +- **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from + the `tests` array. Default: false + +## Postman test suite contentChecks + +Next to the generated tests, it is possible to define "content" checks where a property and the value of the response +body should exist and match a specific value or variable. + +The contentChecks are mapped based on the OpenApi operationId or the OpenApi Operation reference (method + path). +Anything added in `checkResponseBody` array, will be added as content check to the Postman tests. + +Properties explained: + +Target options: + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Response Body will be + tested. (example: `listPets`) +- **openApiOperation (String)** : Reference to the combination of the OpenApi method & path, for which the Postman Response + Body will be tested (example: `GET::/pets`) + +These target options are both supported for defining a target. In case both are set for the same target, only +the `openApiOperationId` will be used for content checks. + +Content check options: + +- **checkResponseBody (Array)** : Array of key/value pairs of properties & values in the Postman Response Body. + - **key (string)** : The key that will be targeted in the response body to check if it exists. + - **value (string)** : The value that will be used to check if the value in the response body matches. + +OpenAPI to Postman Testsuite Configuration: + +```json +{ + "version": 1.0, + "generateTests": { + "limitOperations": [], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": false, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [], + "contentChecks": [ + { + "openApiOperation": "GET::/contacts/{audienceId}", + "checkResponseBody": [ + { + "key": "value[0].name", + "value": "John" + }, + { + "key": "value[0].id", + "value": 1 + } + ] + } + ] +} +``` + +API Response: + +```json +{ + "value": [ + { + "id": 1, + "name": "John", + "createdDt": "2021-05-10T08:29:34.810Z" + }, + { + "id": 2, + "name": "Marco", + "createdDt": "2021-05-10T08:33:14.110Z" + } + ] +} +``` + +Part of Postman Testsuite checks: + +```javascript +// Set response object as internal variable +let jsonData = pm.response.json(); + +// Response body should have property "value[0].name" +pm.test("[GET] /contacts/{audienceId} - Content check if property 'value[0].name' exists", function () { + pm.expect((typeof jsonData.value[0].name !== "undefined")).to.be.true; +}); + +// Response body should have value "John" for "value[0].name" +if (typeof jsonData.value[0].name !== "undefined") { + pm.test("[GET] /contacts/{audienceId} - Content check if value for 'value[0].name' matches 'John'", function () { + pm.expect(jsonData.value[0].name).to.eql("John"); + }) +} +; + +// Response body should have property "value[0].id" +pm.test("[GET] /contacts/{audienceId} - Content check if property 'value[0].id' exists", function () { + pm.expect((typeof jsonData.value[0].id !== "undefined")).to.be.true; +}); + +// Response body should have value "1" for "value[0].id" +if (typeof jsonData.value[0].id !== "undefined") { + pm.test("[GET] //contacts/{audienceId} - Content check if value for 'value[0].id' matches '1'", function () { + pm.expect(jsonData.value[0].id).to.eql(1); + }) +} +; +``` + +### Postman test suite targeting for variables & overwrites + +It is possible to assign variables and overwrite query params, headers, request body data with values specifically for +the tests. + +To be able to do this very specifically, there are options to define the targets: + +- **openApiOperationId (String)** : References to the OpenApi operationId, example: `listPets` +- **openApiOperation (String)** : References to a combination of the OpenApi method & path, example: `GET::/pets` + +An `openApiOperationId` is an optional property. To offer support for OpenApi documents that don't have operationIds, we +have added the `openApiOperation` definition which is the unique combination of the OpenApi method & path, with a `::` +separator symbol. + +This will allow targeting for very specific OpenApi items. + +To facilitate managing the filtering, we have included wildcard options for the `openApiOperation` option, supporting +the methods & path definitions. + +REMARK: Be sure to put quotes around the target definition. + +Strict matching example: `"openApiOperation": "GET::/pets",` +This will target only the "GET" method and the specific path "/pets" + +Method wildcard matching example: `"openApiOperation": "*::/pets",` +This will target all methods ('get', 'put', 'post', 'delete', 'options', 'head', 'patch', 'trace') and the specific +path "/pets" + +Path wildcard matching example: `"openApiOperation": "GET::/pets/*"` +This will target only the "GET" method and any path matching any folder behind the "/pets", like "/pets/123" and +"/pets/123/buy". + +Method & Path wildcard matching example: `"openApiOperation": "*::/pets/*",` +A combination of wildcards for the method and path parts are even possible. + +## Postman test suite assignPmVariables + +To facilitate automation, we provide the option set "pm.environment" variables with values from the response. The +assigning of the pm.environment are mapped based on the OpenApi operationId. + +REMARK: By default the test suite will create a pm.environment variable for the ID property in the response object, if ID +is present in the response. + +Anything added in `assignPmVariables` array, will be used to generate specific pm.environment variables based on the +postman response body. + +Properties explained: + +Target options: + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman pm.environment variable + will be set. (example: `listPets`) +- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman + pm.environment variable will be set. (example: `GET::/pets`) + +These target options are both supported for defining a target. In case both are set for the same target, only +the `openApiOperationId` will be used for overwrites. + +EnvironmentVariables options: + +- **environmentVariables (Array)** : Array of key/value pairs to set the Postman variables. + - **responseBodyProp (string)** : The property for which the value will be taken in the response body and set the value as the pm.environment value. + - **responseHeaderProp (string)** : The property for which the value will be taken in the response header and set the value as the pm.environment value. + - **requestBodyProp (string)** : The property for which the value will be taken in the request body and set the value as the pm.environment value. + - **value (string)** : The defined value that will be set as the pm.environment value. + - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite the default generated variable name + +Example: + +```JSON +{ + "assignPmVariables": [ + { + "openApiOperationId": "post-accounts", + "environmentVariables": [ + { + "responseProp": "clientGuid", + "name": "client-ID" + } + ] + }, + { + "openApiOperationId": "get-accounts", + "environmentVariables": [ + { + "responseProp": "value[0].servers[0]", + "name": "server-address" + } + ] + } + ] +} +``` + +This will generate the following: + +- pm.environment for "post-accounts.id" - use {{post-accounts.id}} as variable for "reponse.id +- pm.environment for "get-accounts" - use {{server-address}} as variable for "reponse.value[0].servers[0]" + +This information on the assignment of the pm.environment will be published in the Postman Test script and during the +conversion via the CLI. + +This results in the following functions on the Postman Test pane: + +```javascript + +// Set response object as internal variable +let jsonData = pm.response.json(); + +// pm.environment - Set post-accounts.id as environment variable +if (jsonData.id) { + pm.environment.set("post-accounts.id", jsonData.id); + console.log("pm.environment - use {{post-accounts.id}} as variable for value", jsonData.id); +} + +// pm.environment - Set post-accounts.servers[0] as environment variable +if (jsonData.value[0].servers[0]) { + pm.environment.set("server-address", jsonData.value[0].servers[0]); + console.log("pm.environment - use {{server-address}} as variable for value", jsonData.value[0].servers[0]); +} +``` + +## Postman test suite overwriteRequests + +To facilitate automation, you might want to modify property values with "randomized" or specific values. The overwrites +are mapped based on the OpenApi operationId. Anything added in `overwriteRequestBody` array, will be used to modify to +the postman request body. + +Properties explained: + +Target options: + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be + extended. (example: `listPets`) +- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman Request + Body will be extended (example: `GET::/pets`) + +These target options are both supported for defining a target. In case both are set for the same target, only +the `openApiOperationId` will be used for overwrites. + +Overwrite options: + +- **overwriteRequestQueryParams (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. + - **key (string)** : The key that will be targeted in the request Query Param to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request Query Param OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request query param value OR attach the value to + the original request query param value. + - **disable (Boolean true/false | Default: false)** : Disables the request query param in Postman + - **remove (Boolean true/false | Default: false)** : Removes the request query param +- **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path + Variables. + - **key (string)** : The key that will be targeted in the request Path variables to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request path variable value OR attach the value + to the original request Path variable value. + - **remove (Boolean true/false | Default: false)** : Removes the request path variable +- **overwriteRequestHeaders (Array)** : Array of key/value pairs to overwrite in the Postman Request Headers. + - **key (string)** : The key that will be targeted in the request Headers to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request headers OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request header value OR attach the value to the + original request header value. + - **remove (Boolean true/false | Default: false)** : Removes the request headers +- **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. + - **key (string)** : The key that will be targeted in the request body to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the + original request body value. + - **remove (Boolean true/false | Default: false)** : Removes the request body property, including the value + +Postman request body before: + +```JSON +{ + "name": "account0-test", + "clientGuid": "ABC-123-DEF-456" +} +``` + +OpenAPI to Postman Testsuite Configuration: + +```JSON +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" + ] + } + ], + "assignPmVariables": [ + { + "openApiOperationId": "post-accounts", + "environmentVariables": [ + { + "responseProp": "clientGuid", + "name": "client-ID" + } + ] + }, + { + "openApiOperationId": "get-accounts", + "environmentVariables": [ + { + "responseProp": "value[0].servers[0]", + "name": "server-address" + } + ] + } + ], + "overwriteRequests": [ + { + "openApiOperationId": "get-accounts", + "overwriteRequestQueryParams": [ + { + "key": "$count", + "value": true, + "overwrite": true + }, + { + "key": "$filter", + "value": "{{$randomInt}}", + "overwrite": false + }, + { + "key": "$search", + "remove": true + }, + { + "key": "$select", + "disable": true + } + ], + "overwriteRequestHeaders": [ + { + "key": "team-id", + "value": "{{$randomInt}}", + "overwrite": true + } + ] + }, + { + "openApiOperationId": "post-accounts", + "overwriteRequestBody": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientGuid", + "value": "{{$guid}}", + "overwrite": true + } + ] + }, + { + "openApiOperationId": "delete-account", + "overwriteRequestPathVariables": [ + { + "key": "id", + "value": "99", + "overwrite": true + } + ] + } + ] +} + +``` + +The `overwriteRequestQueryParams` example will overwrite the "$count" query param value with the boolean "true", the +"$filter" with dynamic value for "$randomInt", for the "get-accounts" OpenAPI operationId. The "$search" query parameter +will be removed, so it will not exist in the Postman collection. The "$select" query parameter will be marked as " +disabled" in the Postman collection + +The `overwriteRequestHeaders` example will overwrite the "team-id" header value with a "{{$randomInt}}", for the +"get-accounts" OpenAPI operationId. + +The `overwriteRequestBody` example will extend the "name" value with the "--{{$randomInt}}" and overwrite the +"clientGuid" with the "{{$guid}}". This will only be applied on the OpenAPI operationId + +Postman request body after: + +```JSON +{ + "name": "account0-test--{{$randomInt}}", + "clientGuid": "{{$guid}}" +} +``` + +This is an example where we leverage the Postman Dynamic variables, but also static values can be used to +overwrite/extend. + +The `overwriteRequestPathVariables` example will overwrite the "id" path variable value with the "99", for the +"delete-account" OpenAPI operationId. diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 2ebb10891..0b26ea10e 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -11,6 +11,7 @@ var _ = require('lodash'), configFile, definedOptions, testFlag, + testsuiteFile, swaggerInput, swaggerData; @@ -51,7 +52,8 @@ program .option('-t, --test', 'Test the OPENAPI converter') .option('-p, --pretty', 'Pretty print the JSON file') .option('-c, --options-config ', 'JSON file containing Converter options') - .option('-O, --options ', 'comma separated list of options', parseOptions); + .option('-O, --options ', 'comma separated list of options', parseOptions) + .option('-g, --generate ', 'Generate postman tests given the JSON file with test options'); program.on('--help', function() { /* eslint-disable */ @@ -78,6 +80,7 @@ testFlag = program.test || false; prettyPrintFlag = program.pretty || false; configFile = program.optionsConfig || false; definedOptions = (!(program.options instanceof Array) ? program.options : {}); +testsuiteFile = program.generate || false; swaggerInput; swaggerData; @@ -126,6 +129,13 @@ function convert(swaggerData) { options = definedOptions; } + if (testsuiteFile) { + testsuiteFile = path.resolve(testsuiteFile); + console.log('Testsuite file: ', testsuiteFile); // eslint-disable-line no-console + options.testSuite = true; + options.testSuiteSettings = JSON.parse(fs.readFileSync(testsuiteFile, 'utf8')); + } + Converter.convert({ type: 'string', data: swaggerData diff --git a/examples/postman-testsuite-advanced.json b/examples/postman-testsuite-advanced.json new file mode 100644 index 000000000..c35cefd2c --- /dev/null +++ b/examples/postman-testsuite-advanced.json @@ -0,0 +1,45 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/examples/postman-testsuite-limited.json b/examples/postman-testsuite-limited.json new file mode 100644 index 000000000..38d23eac3 --- /dev/null +++ b/examples/postman-testsuite-limited.json @@ -0,0 +1,49 @@ +{ + "version": 1.0, + "generateTests": { + "limitOperations": ["get-lists"], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/examples/postman-testsuite-overwrite.json b/examples/postman-testsuite-overwrite.json new file mode 100644 index 000000000..e3aeedb7c --- /dev/null +++ b/examples/postman-testsuite-overwrite.json @@ -0,0 +1,107 @@ +{ + "version": 1.0, + "generateTests": { + "limitOperations": [ + "get-lists" + ], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" + ] + } + ], + "assignPmVariables": [ + { + "openApiOperationId": "post-accounts", + "environmentVariables": [ + { + "responseProp": "clientGuid", + "name": "client-ID" + } + ] + }, + { + "openApiOperationId": "get-accounts", + "environmentVariables": [ + { + "responseProp": "value[0].servers[0]", + "name": "server-address" + } + ] + } + ], + "overwriteRequests": [ + { + "openApiOperationId": "get-accounts", + "overwriteRequestQueryParams": [ + { + "key": "$count", + "value": true, + "overwrite": true + }, + { + "key": "$filter", + "value": "{{$randomInt}}", + "overwrite": false + } + ], + "overwriteRequestHeaders": [ + { + "key": "team-id", + "value": "{{$randomInt}}", + "overwrite": true + } + ] + }, + { + "openApiOperationId": "post-accounts", + "overwriteRequestBody": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientGuid", + "value": "{{$guid}}", + "overwrite": true + } + ] + }, + { + "openApiOperationId": "delete-account", + "overwriteRequestPathVariables": [ + { + "key": "id", + "value": "99", + "overwrite": true + } + ] + } + ] +} diff --git a/examples/postman-testsuite.json b/examples/postman-testsuite.json new file mode 100644 index 000000000..f8428d6f5 --- /dev/null +++ b/examples/postman-testsuite.json @@ -0,0 +1,32 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + } + ] +} diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js new file mode 100644 index 000000000..19f82ec03 --- /dev/null +++ b/lib/pmTestSuite.js @@ -0,0 +1,940 @@ +const _ = require('lodash'), + deref = require('./deref.js'), + defaultOptions = require('../lib/options.js').getOptions('use'), + APP_JSON = 'application/json', + // These are the methods supported in the PathItem schema + // https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#pathItemObject + METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch', 'trace']; + + +module.exports = { + /** + * function to convert an openapi reponse object to object of postman tests + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} components - components defined in the OAS spec. These are used to + * resolve references while generating params. + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @param {object} schemaCache - object storing schemaFaker and schemeResolution caches + * @returns {*} array of all tests + */ + convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { + options = _.merge({}, defaultOptions, options); + let testSuite = {}, + testSuiteSettings = {}, + testSuiteLimits = [], + testSuiteExtensions = [], + pmTests = [], + swagResponse = {}; + + // Check for test suite flag, abort early + if (!options.testSuite) { + return pmTests; + } + + // Check test suite for later usage, potentially convert object + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + if (testSuite.generateTests) { + testSuiteSettings = options.testSuiteSettings.generateTests; + } + + // Limit the test generation to the following operations + if (testSuiteSettings.limitOperations) { + testSuiteLimits = testSuiteSettings.limitOperations; + } + + // Extend the generated test with additional operations + if (testSuite.extendTests) { + testSuiteExtensions = options.testSuiteSettings.extendTests; + + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId && + testExtension && testExtension.responseChecks) { + // Extend testSuiteSettings with testExtension settings + testSuiteSettings.responseChecks = _.merge( + {}, testSuiteSettings.responseChecks, testExtension.responseChecks + ); + } + }); + } + + } + + _.forOwn(operation.responses, (response, code) => { + // Skip the operations, if there are limits defined + if (operation.operationId && testSuiteLimits.length > 0 && + testSuiteLimits.indexOf(operation.operationId) === -1) { + return; // skip this response + } + + // Only support 2xx response checks + // TODO Investigate how to support other response codes like validation response 4xx or 5xx + if (!_.inRange(code, 200, 299)) { + return; // skip this response + } + + // Format response + swagResponse = response; + if (response.$ref) { + swagResponse = this.getRefObject(response.$ref, components, options); + } + + // Add status code check + // TODO Validate other request codes + // if (operationItem.method.toUpperCase() === 'GET') { + // pmTests.push( + // '// Validate status code \n'+ + // 'pm.test("Status code should be '+code+'", function () {\n' + + // ' pm.response.to.have.status('+code+');\n' + + // '});\n' + // ) + // } + + // Add status success check + if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { + pmTests.push( + '// Validate status 2xx \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Status code is 2xx", function () {\n' + + ' pm.response.to.be.success;\n' + + '});\n' + ); + } + + // Add response timer check + if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { + let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); + pmTests.push( + '// Validate response time \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + + '});\n' + ); + } + + // Process the response content + _.forOwn(swagResponse.content, (content, contentType) => { + if (contentType) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { + pmTests.push( + '// Validate content-type \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content-Type is ' + contentType + '", function () {\n' + + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + + '});\n' + ); + } + + if (contentType === APP_JSON) { + let resolvedSchema, + pmVariablesCounter = 0; + + // Add JSON body check + if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { + pmTests.push( + '// Response should have JSON Body\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response has JSON Body", function () {\n' + + ' pm.response.to.have.jsonBody();\n' + + '});\n'); + } + + // Add JSON schema check + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && content.schema) { + let schemaContent = content.schema; + // if (content.schema.$ref) { + // schemaContent = this.getRefObject(content.schema.$ref, components, options); + // } + + try { + // When processing a reference, schema.type could also be undefined + resolvedSchema = deref.resolveRefs(_.cloneDeep(schemaContent), 'response', components, + schemaCache, 'VALIDATION', 'example', 0, {}, 999); + + // deletes nullable and adds "null" to type array if nullable is true + let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); + + pmTests.push( + '// Response Validation\n' + + 'const schema = ' + JSON.stringify(jsonSchema) + '\n' + + '\n' + + '// Test whether the response matches the schema\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Schema is valid", function() {\n' + + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + + '});\n' + ); + } + catch (e) { + console.warn('JSON schema check failed', e); + console.warn('invalid schemaValidation for ', content); + } + } + + // Add content checks for JSON response + if (testSuite.contentChecks) { + let testSuiteContentChecks = testSuite.contentChecks, + // Get the checkResponseBody settings for the operationId or openApiOperation + setContentChecksForOperation = testSuiteContentChecks.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (setContentChecksForOperation && setContentChecksForOperation.checkResponseBody) { + let responseBodyChecks = setContentChecksForOperation.checkResponseBody; + responseBodyChecks.forEach((check) => { + // Only set the jsonData once + if (pmVariablesCounter === 0) { + pmTests.push( + '// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + + if (check.key) { + pmTests.push( + '// Response body should have property "' + check.key + '"\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content check if property \'' + check.key + '\' exists", function() {\n' + + ' pm.expect((typeof jsonData.' + check.key + ' !== "undefined")).to.be.true;\n' + + '});\n' + ); + } + + if (check.value) { + let checkValue = check.value; + if (!(typeof check.value === 'number' || typeof check.value === 'boolean')) { + checkValue = '"' + check.value + '"'; + } + if (check.value.includes('{{') && check.value.includes('}}')) { + checkValue = 'pm.environment.get("' + check.value.replace(/{{|}}/g,'') + '")'; + } + + pmTests.push( + '// Response body should have value "' + check.value + '" for "' + check.key + '"\n' + + 'if (typeof jsonData.' + check.key + ' !== "undefined") {\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content check if value for \'' + check.key + '\' matches \'' + + check.value + '\'", function() {\n' + + ' pm.expect(jsonData.' + check.key + ').to.eql(' + checkValue + ');\n' + + '})};\n' + ); + } + pmVariablesCounter++; + }); + } + } + + // Automatic set the response.id as an environment variable for chaining option + if (operationItem.method.toUpperCase() === 'POST' && resolvedSchema !== undefined && + resolvedSchema.properties && resolvedSchema.properties.id) { + let opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + + // Only set the jsonData once + if (pmVariablesCounter === 0) { + pmTests.push( + '// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + pmTests.push( + '// pm.environment - Set ' + opsRef + '.id as environment variable \n' + + 'if (typeof jsonData.id !== "undefined") {\n' + + ' pm.environment.set("' + opsRef + '.id",jsonData.id);\n' + + ' console.log("pm.environment - use {{' + opsRef + '.id}} ' + + 'as variable for value", jsonData.id);\n' + + '};\n'); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '.id" - use {{' + + opsRef + '.id}} as variable for "reponse.id'); + pmVariablesCounter++; + } + + // Assign defined PmVariables for JSON response + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the operationId or openApiOperation + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + let i = 0; + // Set value as variable + if (environmentVariable.value) { + // Set variable name + let opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + let varName = opsRef + '.var-' + i; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + pmTests.push( + '// pm.environment - Set a value for ' + varName + '\n' + + 'pm.environment.set("' + varName + '",' + environmentVariable.value + ')\n' + ); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "' + environmentVariable.value + '"'); + i++; + } + + // Set response body property as variable + if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { + let responseProp = (environmentVariable.responseProp) ? + environmentVariable.responseProp : + environmentVariable.responseBodyProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + + // Set variable name + let varName = opsRef + '.' + responseProp; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + // Only set the jsonData once + if (pmVariablesCounter === 0) { + pmTests.push( + '// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + pmTests.push( + '// pm.environment - Set ' + varName + ' as variable for jsonData.' + responseProp + ' \n' + + 'if (typeof jsonData.' + responseProp + ' !== "undefined") {\n' + + ' pm.environment.set("' + varName + '",jsonData.' + responseProp + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + 'jsonData.' + responseProp + ');\n' + + '};\n'); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "reponse.' + responseProp + '"'); + pmVariablesCounter++; + } + }); + } + } + + } + } + + }); + + // Process the response header + _.forOwn(swagResponse.headers, (header, headerKey) => { + + if (headerKey && header.name) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { + pmTests.push( + '// Validate header \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response header ' + header.name + ' is present", function () {\n' + + ' pm.response.to.have.header("' + header.name + '");\n' + + '});\n' + ); + } + } + + }); + + // Assign defined PmVariables for response headers + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the operationId or openApiOperation + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set response header property as variable + if (environmentVariable.responseHeaderProp) { + let headerProp = environmentVariable.responseHeaderProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + + // Set variable name + let varName = opsRef + '.' + headerProp; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + + // Safe variable name + let safeVarName = varName.replace(/-/g, '') // eslint-disable-line one-var + .replace(/_/g, '').replace(/ /g, '') + .replace(/\./g, ''); + + pmTests.push( + String('// pm.environment - Set ' + varName + ' as environment variable \n' + + 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + + 'if (' + safeVarName + ' !== undefined) {\n' + + ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ') + + safeVarName + ');\n' + + '};\n'); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "header.' + headerProp + '"'); + } + + }); + } + } + + }); + + // Add test extensions that are defined in the test suite file + if (testSuiteExtensions.length > 0) { + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId) { + + if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { + // Reset generated tests + pmTests = []; + } + + // Add test extensions + if (testExtension.tests && testExtension.tests.length > 0) { + testExtension.tests.forEach((postmanTest) => { + try { + // Extend the generated tests, with the test extension scripts + pmTests.push(postmanTest); + } + catch (e) { + console.warn('invalid extendTests for ' + testExtension.openApiOperationI); + } + }); + } + } + }); + } + + return pmTests; + }, + + /** + * function to overwrite a request body with values defined by the postman testsuite + * @param {*} requestBody object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestBodyByTests: function (requestBody, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestBody = Object.assign({}, requestBody), // clone requestBody before manipulation + bodyString, + find, + replace, + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is RAW requestBody, abort early + if (!options.testSuite && !requestBody.raw) { + return requestBody; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId or openApiOperation + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestBody) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestBody; + + // Overwrite values for Keys + let bodyData = JSON.parse(testRequestBody.raw); + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && overwriteValue.hasOwnProperty('value')) { + let orgValue = _.get(bodyData, overwriteValue.key), + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + + bodyData = _.set(bodyData, overwriteValue.key, newValue); + + if (overwriteValue.remove === true) { + bodyData = _.omit(bodyData, overwriteValue.key); + } + } + }); + bodyString = JSON.stringify(bodyData, null, 4); + + // Handle {{$randomInt}},{{$randomCreditCardMask}} conversion from string to number + find = ['"{{$randomInt}}"', '"{{$randomCreditCardMask}}"', '"{{$randomBankAccount}}"']; + replace = ['{{$randomInt}}', '{{$randomCreditCardMask}}', '{{$randomBankAccount}}']; + find.forEach(function (item, index) { + let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, '\\$1'); + bodyString = bodyString.replace(new RegExp(escapedFind, 'g'), replace[index]); + }); + + // Set testRequestBody.raw with stringified body + testRequestBody.raw = bodyString; + } + } + + return testRequestBody; + }, + + /** + * function to overwrite a request path variables with values defined by the postman testsuite + * @param {*} requestPathVariables request Path Variables object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestPathVariables = JSON.parse(JSON.stringify(requestPathVariables)), // clone requestPathVariables + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestPathVariables, abort early + if (!options.testSuite && requestPathVariables.length > 0) { + return requestPathVariables; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId or openApiOperation + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestPathVariables) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestPathVariables; + + // Overwrite value for path variable name + testRequestPathVariables.forEach((pathVar, index) => { + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && pathVar.name && overwriteValue.key === pathVar.name && + overwriteValue.hasOwnProperty('value') && pathVar.schema) { + let orgValue = (pathVar.schema.example ? pathVar.schema.example : null), + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + pathVar.schema.type = 'string'; // Set schema as type string dynamic variable + pathVar.schema.example = newValue; + + if (overwriteValue.remove === true) { + testRequestPathVariables.splice(index, 1); + } + } + }); + }); + } + } + return testRequestPathVariables; + }, + + /** + * function to overwrite a request query params with values defined by the postman testsuite + * @param {*} requestQueryParam request Query Param object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestQueryParamByTests: function (requestQueryParam, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestQueryParam + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestQueryParam.key, abort early + if (!options.testSuite && !requestQueryParam.key) { + return requestQueryParam; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId or openApiOperation + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestQueryParams) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestQueryParams; + + // Overwrite value for query param key + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key) { + + // Test suite - Overwrite/extend query param value + if (overwriteValue.hasOwnProperty('value') && testRequestQueryParam.hasOwnProperty('value')) { + let orgValue = testRequestQueryParam.value, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestQueryParam.value = newValue; + } + + // Test suite - Disable query param + if (overwriteValue.disable === true) { + testRequestQueryParam.disabled = true; + } + + // Test suite - Remove query param + if (overwriteValue.remove === true) { + testRequestQueryParam = {}; + } + } + }); + } + } + return testRequestQueryParam; + }, + + /** + * function to overwrite a request header with values defined by the postman testsuite + * @param {*} requestHeader request Header object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestHeaderByTests: function (requestHeader, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestHeader = Object.assign({}, requestHeader), // clone requestHeader + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestHeader.schema, abort early + if (!options.testSuite && !requestHeader.schema) { + return requestHeader; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId or openApiOperation + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestHeaders) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestHeaders; + // Overwrite value for header name + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestHeader.name && overwriteValue.key === testRequestHeader.name && + overwriteValue.hasOwnProperty('value') && testRequestHeader.schema) { + let orgValue = (testRequestHeader.schema.example ? testRequestHeader.schema.example : null), + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestHeader.schema.example = newValue; + + if (overwriteValue.remove === true) { + testRequestHeader = {}; + } + } + }); + } + } + return testRequestHeader; + }, + + /** + * function to assign PM variables with values defined by the request body from OpenApi + * @param {*} requestBody request Body object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Postman Test scripts array + */ + assignVariablesByRequestBody: function (requestBody, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testSuite = {}, + pmTests = []; + + // Check for test suite flag and if there is RAW requestBody, abort early + if (!options.testSuite && !requestBody.raw) { + return requestBody; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the operationId or openApiOperation + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set request body property as variable + if (environmentVariable.requestBodyProp) { + let responseProp = environmentVariable.requestBodyProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + + // Set variable name + let varName = opsRef + '.' + responseProp; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + + // Set variable value + const requestBodyObj = JSON.parse(requestBody.raw); + if (typeof requestBodyObj[responseProp] !== 'undefined') { + let requestBodyValue = requestBodyObj[responseProp]; + if (!(typeof requestBodyValue === 'number' || typeof requestBodyValue === 'boolean')) { + requestBodyValue = '"' + requestBodyValue + '"'; + } + pmTests.push( + '// pm.environment - Set ' + varName + ' as environment variable from request body \n' + + 'pm.environment.set("' + varName + '",' + requestBodyValue + ');\n' + + 'console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + requestBodyValue + ');\n' + ); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "request.' + responseProp + '"'); + } + } + }); + } + } + + return pmTests; + }, + + /** + * function to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + * @param {*} oaSchema openAPI schema + * @returns {*} Modified openAPI schema object that is compatible with JSON schema validation + */ + convertUnsupportedJsonSchemaProperties: function (oaSchema) { + let jsonSchema = JSON.parse(JSON.stringify(oaSchema)); // Deep copy of the schema object + + // Convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + // let jsonSchemaNotSupported = ['nullable', 'discriminator', 'readOnly', 'writeOnly', 'xml', + // 'externalDocs', 'example', 'deprecated']; + + // Recurse through OpenAPI Schema + const traverse = (obj) => { + for (let k in obj) { + if (obj.hasOwnProperty(k) && obj[k] && typeof obj[k] === 'object') { + if (obj[k].nullable && obj[k].nullable === true) { + // deletes nullable and adds "null" to type array if nullable is true + let jsonTypes = []; + jsonTypes.push(obj[k].type); + jsonTypes.push('null'); + obj[k].type = jsonTypes; + delete obj[k].nullable; + } + if (obj[k].maxItems && obj[k].maxItems === 2 && obj[k].type === 'array') { + // deletes maxItems, which is added unwanted by resolveRefs() in combination with resolveFor CONVERSION + // TODO find another way to respect the maxItems that might be passed by OpenAPI + // Asked for assistance https://github.com/postmanlabs/openapi-to-postman/issues/367 + delete obj[k].maxItems; + } + traverse(obj[k]); + } + } + }; + traverse(jsonSchema); + return jsonSchema; + }, + + // TODO remove this function, use a reference to schemaUtils instead + /** + * @param {*} $ref reference object + * @param {*} components the components + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {Object} reference object from the saved components + * @no-unit-tests + */ + getRefObject: function ($ref, components, options) { + options = _.merge({}, defaultOptions, options); + var refObj, + savedSchema; + + savedSchema = $ref.split('/').slice(1).map((elem) => { + // https://swagger.io/docs/specification/using-ref#escape + // since / is the default delimiter, slashes are escaped with ~1 + return decodeURIComponent( + elem + .replace(/~1/g, '/') + .replace(/~0/g, '~') + ); + }); + // at this stage, savedSchema is [components, part1, parts] + // must have min. 2 segments after "#/components" + if (savedSchema.length < 3) { + console.warn(`ref ${$ref} not found.`); + return { value: `reference ${$ref} not found in the given specification` }; + } + + if (savedSchema[0] !== 'components' && savedSchema[0] !== 'paths') { + console.warn(`Error reading ${$ref}. Can only use references from components and paths`); + return { value: `Error reading ${$ref}. Can only use references from components and paths` }; + } + + // at this point, savedSchema is similar to ['components', 'schemas','Address'] + // components is actually components and paths (an object with components + paths as 1st-level-props) + refObj = _.get(components, savedSchema); + + if (!refObj) { + console.warn(`ref ${$ref} not found.`); + return { value: `reference ${$ref} not found in the given specification` }; + } + + if (refObj.$ref) { + return this.getRefObject(refObj.$ref, components, options); + } + + return refObj; + }, + + /** + * A check if the OpenApi operation item matches a target definition . + * @param {object} operationItem the OpenApi operation item to match + * @param {object} target the entered path definition that is a combination of the method & path, like GET::/lists + * @returns {boolean} matching information + */ + isMatchOperationItem: function (operationItem, target) { + if (target.openApiOperationId) { + return (operationItem.properties && operationItem.properties.operationId && + operationItem.properties.operationId === target.openApiOperationId); + } + + if (target.openApiOperation) { + const targetSplit = target.openApiOperation.split('::/'); + if (targetSplit[0] && targetSplit[1]) { + let targetMethod = [targetSplit[0].toLowerCase()]; + const targetPath = targetSplit[1].toLowerCase(); + // Wildcard support + if (targetMethod.includes('*')) { + targetMethod = METHODS; + } + return ((operationItem.method && targetMethod.includes(operationItem.method.toLowerCase())) && + (operationItem.path && this.matchPath(targetPath, operationItem.path.toLowerCase()))); + } + } + + return false; + }, + + /** + * Converts combination of the OpenApi method & path to a uniform OpenApiOperationRef. + * @param {object} operationItem the OpenApi operation item to match + * @param {string} format the format of OpenApi Operation reference (MATCH or PMVAR) + * @returns {string} Return a OpenApiOperation reference + * @no-unit-tests + */ + getOpenApiOperationRef: function (operationItem, format) { + const operationFormat = format || 'MATCH'; + if (operationItem.method && operationItem.path) { + if (operationFormat === 'MATCH') { + return operationItem.method.toUpperCase() + '::/' + operationItem.path; + } + if (operationFormat === 'PMVAR') { + return operationItem.method.toLowerCase() + '-' + operationItem.path.replace('{', ':') + .replace('}', '').replace(/#|\//g, '-'); + } + } + return ''; + }, + + /** + * Converts a string path to a Regular Expression. + * Transforms path parameters into named RegExp groups. + * @param {*} path the path pattern to match + * @returns {RegExp} Return a regex + * @no-unit-tests + */ + pathToRegExp: function (path) { + const pattern = path + // Escape literal dots + .replace(/\./g, '\\.') + // Escape literal slashes + .replace(/\//g, '/') + // Escape literal question marks + .replace(/\?/g, '\\?') + // Ignore trailing slashes + .replace(/\/+$/, '') + // Replace wildcard with any zero-to-any character sequence + .replace(/\*+/g, '.*') + // Replace parameters with named capturing groups + .replace(/:([^\d|^\/][a-zA-Z0-9_]*(?=(?:\/|\\.)|$))/g, (_, paramName) => { + return `(?<${paramName}>[^\/]+?)`; + }) + // Allow optional trailing slash + .concat('(\\/|$)'); + return new RegExp(pattern, 'gi'); + }, + + /** + * Matches a given url against a path, with Wildcard support (based on the node-match-path package) + * @param {*} path the path pattern to match + * @param {*} url the entered URL is being evaluated for matching + * @returns {Object} matching information + */ + matchPath: function (path, url) { + const expression = path instanceof RegExp ? path : this.pathToRegExp(path), + match = expression.exec(url) || false; + // Matches in strict mode: match string should equal to input (url) + // Otherwise loose matches will be considered truthy: + // match('/messages/:id', '/messages/123/users') // true + // eslint-disable-next-line one-var,no-implicit-coercion + const matches = path instanceof RegExp ? !!match : !!match && match[0] === match.input; + return matches; + // return { + // matches, + // params: match && matches ? match.groups || null : null + // }; + } + +}; diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index c825461cf..732a57e4d 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -13,6 +13,7 @@ const async = require('async'), openApiErr = require('./error.js'), ajvValidationError = require('./ajvValidationError'), utils = require('./utils.js'), + testSuite = require('./pmTestSuite.js'), defaultOptions = require('../lib/options.js').getOptions('use'), { Node, Trie } = require('./trie.js'), { validateSchema } = require('./ajvValidation'), @@ -2118,6 +2119,8 @@ module.exports = { displayUrl, reqUrl = '/' + operationItem.path, pmBody, + pmTests = [], + pmTestVars = [], authMeta, swagResponse, localServers = _.get(operationItem, 'properties.servers'), @@ -2273,6 +2276,8 @@ module.exports = { _.forEach(reqParams.query, (queryParam) => { this.convertToPmQueryParameters(queryParam, REQUEST_TYPE.ROOT, components, options, schemaCache) .forEach((pmParam) => { + // pmTestSuite - Modify the request query params for the test suites + pmParam = testSuite.overwriteRequestQueryParamByTests(pmParam, operation, operationItem, options); item.request.url.addQueryParams(pmParam); }); }); @@ -2292,6 +2297,8 @@ module.exports = { } }); item.request.url.variables.clear(); + // pmTestSuite - Modify the request paths variables for the test suites + reqParams.path = testSuite.overwriteRequestPathByTests(reqParams.path, operation, operationItem, options); item.request.url.variables.assimilate(this.convertPathVariables('param', pathVarArray, reqParams.path, components, options, schemaCache)); @@ -2308,6 +2315,8 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { + // pmTestSuite - Modify the request header for the test suite + header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); if (!_.includes(IMPLICIT_HEADERS, _.toLower(_.get(header, 'name')))) { item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); @@ -2320,6 +2329,11 @@ module.exports = { reqBody = this.getRefObject(reqBody.$ref, components, options); } pmBody = this.convertToPmBody(reqBody, REQUEST_TYPE.ROOT, components, options, schemaCache); + // pmTestSuite - Modify the request body for the test suite + pmBody.body = testSuite.overwriteRequestBodyByTests(pmBody.body, operation, operationItem, options); + // pmTestSuite - Assign PM vars based on the request body for the test suite + pmTestVars = testSuite.assignVariablesByRequestBody(pmBody.body, operation, operationItem, options); + item.request.body = pmBody.body; item.request.addHeader(pmBody.contentHeader); // extra form headers if encoding is present in request Body. @@ -2401,6 +2415,21 @@ module.exports = { }); } + // pmTestSuite - Generate tests for the response + pmTests = testSuite.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); + + // Merge pmTests array + const pmCombinedTests = [...pmTests, ...pmTestVars]; + + // Add tests to postman item + if (pmTests.length > 0) { + // Add test event + item.events.add({ + listen: 'test', + script: pmCombinedTests + }); + } + return item; }, diff --git a/lib/schemapack.js b/lib/schemapack.js index c97581868..6261b3f0a 100644 --- a/lib/schemapack.js +++ b/lib/schemapack.js @@ -46,6 +46,10 @@ class SchemaPack { let indentCharacter = this.computedOptions.indentCharacter; this.computedOptions.indentCharacter = indentCharacter === 'tab' ? '\t' : ' '; + // hardcoding the test suite options CLI mapping - not exposed to postman users yet + this.computedOptions.testSuite = options.testSuite; + this.computedOptions.testSuiteSettings = options.testSuiteSettings; + this.validate(); } diff --git a/test/data/validationData/implicitHeaderSpec.yaml b/test/data/validationData/implicitHeaderSpec.yaml index 4427bc63c..0b4a0aadd 100644 --- a/test/data/validationData/implicitHeaderSpec.yaml +++ b/test/data/validationData/implicitHeaderSpec.yaml @@ -74,4 +74,4 @@ components: type: integer format: int32 message: - type: string \ No newline at end of file + type: string diff --git a/test/data/validationData/urlencodedBodyCollection.json b/test/data/validationData/urlencodedBodyCollection.json index 9c77f5ab8..e541dff24 100644 --- a/test/data/validationData/urlencodedBodyCollection.json +++ b/test/data/validationData/urlencodedBodyCollection.json @@ -158,4 +158,4 @@ "type": "text/plain" } } -} \ No newline at end of file +}