From 53ee6b8ac5568df87be34718e9e6065ddd239cb7 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 1 Jun 2020 12:35:58 +0200 Subject: [PATCH 01/52] Added test generation for: - Success status check - Response time check - Content-type check - JSON schema validation - JSON body check Merged #218 for easy config file usage in the CLI tool. Added Postman test suite settings, which allow you to define which types of basic tests to be included or not, in the postman generation. Added Post test suite file loading (inspired by #218) option for the CLI Added postman-testsuite.json example Added test suite extension option, which offers the option to include manually defined postman tests based. The test extensions are mapped based on the OpenApi operationID. --- README.md | 95 ++++++++++++++++-- bin/openapi2postmanv2.js | 25 ++++- examples/postman-testsuite.json | 31 ++++++ lib/schemaUtils.js | 170 ++++++++++++++++++++++++++++++++ lib/schemapack.js | 4 + 5 files changed, 313 insertions(+), 12 deletions(-) create mode 100644 examples/postman-testsuite.json diff --git a/README.md b/README.md index d06b27e64..b45577b45 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ -![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) +![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) -*Supercharge your API workflow.* +*Supercharge your API workflow.* *Modern software is built on APIs. Postman helps you develop APIs faster.* # OpenAPI 3.0 to Postman Collection v2.1.0 Converter [![Build Status](https://travis-ci.org/postmanlabs/openapi-to-postman.svg?branch=master)](https://travis-ci.org/postmanlabs/openapi-to-postman) -#### Contents +#### Contents 1. [Getting Started](#getting-started) 2. [Using the converter as a NodeJS module](#using-the-converter-as-a-nodejs-module) @@ -95,7 +95,7 @@ function (err, result) { ### ConversionResult -- `result` - Flag responsible for providing a status whether the conversion was successful or not +- `result` - Flag responsible for providing a status whether the conversion was successful or not - `reason` - Provides the reason for an unsuccessful conversion, defined only if result: false @@ -154,22 +154,28 @@ The converter can be used as a CLI tool as well. The following [command line opt `openapi2postmanv2 [options]` ### Options -- `-V`, `--version` +- `-V`, `--version` Specifies the version of the converter -- `-s `, `--spec ` +- `-s `, `--spec ` Used to specify the OpenAPI specification (file path) which is to be converted -- `-o `, `--output ` +- `-o `, `--output ` Used to specify the destination file in which the collection is to be written -- `-t`, `--test` +- `-t`, `--test` Used to test the collection with an in-built sample specification -- `-p`, `--pretty` +- `-p`, `--pretty` Used to pretty print the collection object while writing to a file -- `-h`, `--help` +- `-c`, `--config ` + Used to supply options to the converter + +- `-g`, `--generate ` + Used to generate postman tests given the JSON file with test options + +- `-h`, `--help` Specifies all the options along with a few usage examples on the terminal @@ -188,6 +194,11 @@ $ openapi2postmanv2 -s spec.yaml -o collection.json -p $ openapi2postmanv2 --test ``` +- Generating additional postman tests for the OpenAPi specification +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-test.json +``` + ## Conversion Schema | *postman* | *openapi* | *options* | *examples* | @@ -204,3 +215,67 @@ $ openapi2postmanv2 --test | request.url.variables | parameter (`in = path`) | - | [link](#Header/Path-param-conversion-example) | | request.url.params | parameter (`in = query`) | - | {"key": param.name, "value": [link](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.1.md#style-examples)}| | api_key in (query or header) | components.securitySchemes.api_key | - || + +## Postman test generation + +This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. +The goal is to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used +by the Newman test runner. + +To generate the tests, define a JSON file like the example (postman-testsuite.json) below and run the CLI with the --generate option. + +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json +``` + +Current postman-testsuite JSON properties + +```JSON +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); postman.setEnvironmentVariable(\"userId\", responseBody);" + ] + } + ] +} +``` + +The JSON test suite format consists out of 3 parts: +- **version** : which refers the JSON test suite version (not relevant but might handy for future backward compatibility options). +- **generateTests** : which refers the default available generated postman tests. The default tests are grouped per type (response, request) + - **responseChecks** : All response basic checks. (For now we have only included response checks). +- **extendTests**: which refers the custom additions of manual created postman tests. The manual tests are added during +generation. The tests are mapped based on the OpenApi operationId. + +| name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | +|-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| +| Response status success (2xx) check | generaStatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | +| Response time check | responseTime | boolean | false | enabled | maxMs 300 | Adds the check if the response of the postman request is within a number of ms. | true | TEST GENERATION | +| Response content-type check | contentType | boolean | false | enabled | | Adds the check if the postman response header is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response Schema validation check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 11025d98a..9aee2fd17 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -6,7 +6,9 @@ var program = require('commander'), inputFile, outputFile, prettyPrintFlag, + configFile, testFlag, + testsuiteFile, swaggerInput, swaggerData; @@ -15,7 +17,9 @@ program .option('-s, --spec ', 'Convert given OPENAPI 3.0.0 spec to Postman Collection v2.0') .option('-o, --output ', 'Write the collection to an output file') .option('-t, --test', 'Test the OPENAPI converter') - .option('-p, --pretty', 'Pretty print the JSON file'); + .option('-p, --pretty', 'Pretty print the JSON file') + .option('-c, --config ', 'JSON file containing Converter options') + .option('-g, --generate ', 'Generate postman tests given the JSON file with test options'); program.on('--help', function() { @@ -41,6 +45,8 @@ inputFile = program.spec; outputFile = program.output || false; testFlag = program.test || false; prettyPrintFlag = program.pretty || false; +configFile = program.config || false; +testsuiteFile = program.generate || false; swaggerInput; swaggerData; @@ -73,10 +79,25 @@ function writetoFile(prettyPrintFlag, file, collection) { * @returns {void} */ function convert(swaggerData) { + let options = {}; + + if (configFile) { + configFile = path.resolve(configFile); + console.log('Config file: ', configFile); + options = JSON.parse(fs.readFileSync(configFile, 'utf8')); + } + + if (testsuiteFile) { + testsuiteFile = path.resolve(testsuiteFile); + console.log('Testsuite file: ', testsuiteFile); + options.testSuite = true; + options.testSuiteSettings = JSON.parse(fs.readFileSync(testsuiteFile, 'utf8')); + } + Converter.convert({ type: 'string', data: swaggerData - }, {}, (err, status) => { + }, options, (err, status) => { if (err) { return console.error(err); } diff --git a/examples/postman-testsuite.json b/examples/postman-testsuite.json new file mode 100644 index 000000000..b0f70b2df --- /dev/null +++ b/examples/postman-testsuite.json @@ -0,0 +1,31 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);}); pm.test('check userId after create', function(){Number.isInteger(responseBody);}); postman.setEnvironmentVariable(\"userId\", responseBody);" + ] + } + ] +} diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 6cad6234f..75ef51227 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -1804,6 +1804,7 @@ module.exports = { displayUrl, reqUrl = '/' + operationItem.path, pmBody, + pmTests, authMeta, swagResponse, localServers = _.get(operationItem, 'properties.servers'), @@ -2061,9 +2062,178 @@ module.exports = { }); } + // Generate tests for the response + pmTests = this.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); + + if (!_.isEmpty(pmTests)) { + // Add test event + item.events.add(pmTests); + } + return item; }, + /** + * function to convert an openapi reponse object to object of postman tests + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} components - components defined in the OAS spec. These are used to + * resolve references while generating params. + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @param {object} schemaCache - object storing schemaFaker and schmeResolution caches + * @returns {*} array of all query params + */ + convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { + console.log('options', options); + options = _.merge({}, defaultOptions, options); + let testEvent = {}, + testSuite = {}, + testSuiteSettings = {}, + testSuiteExtensions = [], + requestsTests = [], + swagResponse = {}; + + // Check for test suite flag, abort early + if (!options.testSuite) { + return testEvent; + } + + // Check test suite for later usage, potentially convert object + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + if (testSuite.generateTests) { + testSuiteSettings = options.testSuiteSettings.generateTests; + } + if (testSuite.extendTests) { + testSuiteExtensions = options.testSuiteSettings.extendTests; + } + console.log('optionsTestSuite', testSuiteSettings); + } + + _.forOwn(operation.responses, (response, code) => { + + // Only support 2xx response checks + // TODO Investigate how to support other response codes like validation response 4xx or 5xx + if (!_.inRange(code, 200, 299)) { + return; // skip this response + } + + // Format response + swagResponse = response; + if (response.$ref) { + swagResponse = this.getRefObject(response.$ref, components, options); + } + + // Add status code check + // TODO Validate other request codes + // if (operationItem.method.toUpperCase() === 'GET') { + // requestsTests.push( + // '// Validate status code \n'+ + // 'pm.test("Status code should be '+code+'", function () {\n' + + // ' pm.response.to.have.status('+code+');\n' + + // '});\n' + // ) + // } + + // Add status success check + if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { + requestsTests.push( + '// Validate status 2xx \n' + + 'pm.test("Status code is 2xx", function () {\n' + + ' pm.response.to.be.success;\n' + + '});\n' + ); + } + + // Add response timer check + if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { + let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); + requestsTests.push( + '// Validate response time \n' + + 'pm.test("Response time is less than ' + maxMs + 'ms", function () {\n' + + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + + '});\n' + ); + } + + // Process the response content + _.forOwn(swagResponse.content, (content, contentType) => { + if (contentType) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { + requestsTests.push( + '// Validate content-type \n' + + 'pm.test("Content-Type is ' + contentType + '", function () {\n' + + // ' pm.response.to.have.header("Content-Type");\n' + + // ' pm.response.to.have.header("Content-Type", "' + contentType + '");\n' + + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + + '});\n' + ); + } + + if (contentType === APP_JSON) { + // Add JSON body check + if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { + requestsTests.push( + '// Response should have JSON Body\n' + + 'pm.test("Response has JSON Body", function () {\n' + + ' pm.response.to.have.jsonBody();\n' + + '});\n'); + } + + // Add JSON schema check + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled')) { + let schemaContent = this.getRefObject(content.schema.$ref, components, options), + // When processing a reference, schema.type could also be undefined + resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + schemaCache, PROCESSING_TYPE.VALIDATION); + + requestsTests.push( + '// Response Validation\n' + + 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + + '\n' + + '// Test whether the response matches the schema\n' + + 'pm.test(\'Schema is valid\', function() {\n' + + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + + '});\n' + ); + } + } + + } + + }); + }); + + // Add test extensions that are defined in the test suite file + if (testSuiteExtensions.length > 0) { + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId) { + console.log('testExtension', testExtension); + if (testExtension.tests.length > 0) { + testExtension.tests.forEach((postmanTest) => { + try { + requestsTests.push(postmanTest); + } catch (e) { + console.log('invalid extendTests for ' + testExtension.openApiOperationI); + } + }); + } + } + }); + } + + // Add tests to postman item + if (requestsTests.length > 0) { + testEvent = { + listen: 'test', + script: requestsTests + }; + } + return testEvent; + }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object diff --git a/lib/schemapack.js b/lib/schemapack.js index 6fdf5b4f5..4b90c4ab1 100644 --- a/lib/schemapack.js +++ b/lib/schemapack.js @@ -44,6 +44,10 @@ class SchemaPack { let indentCharacter = this.computedOptions.indentCharacter; this.computedOptions.indentCharacter = indentCharacter === 'tab' ? '\t' : ' '; + // hardcoding the test suite options CLI mapping - not exposed to postman users yet + this.computedOptions.testSuite = options.testSuite; + this.computedOptions.testSuiteSettings = options.testSuiteSettings; + this.validate(); } From c6f881f088c8e1461d847328bcb5f9afc09f50a7 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 1 Jun 2020 12:41:47 +0200 Subject: [PATCH 02/52] Minor change in the sample --- examples/postman-testsuite.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/examples/postman-testsuite.json b/examples/postman-testsuite.json index b0f70b2df..f8428d6f5 100644 --- a/examples/postman-testsuite.json +++ b/examples/postman-testsuite.json @@ -24,7 +24,8 @@ { "openApiOperationId": "get-lists", "tests": [ - "pm.test('200 ok', function(){pm.response.to.have.status(200);}); pm.test('check userId after create', function(){Number.isInteger(responseBody);}); postman.setEnvironmentVariable(\"userId\", responseBody);" + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " ] } ] From 2d4d9e1bb213e6db9716f8257e7df8b419015c53 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 1 Jun 2020 19:41:48 +0200 Subject: [PATCH 03/52] Minor changes to the readme --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b45577b45..eea8ad6b5 100644 --- a/README.md +++ b/README.md @@ -258,7 +258,7 @@ Current postman-testsuite JSON properties "openApiOperationId": "get-lists", "tests": [ "pm.test('200 ok', function(){pm.response.to.have.status(200);});", - "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); postman.setEnvironmentVariable(\"userId\", responseBody);" + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" ] } ] @@ -270,8 +270,9 @@ The JSON test suite format consists out of 3 parts: - **generateTests** : which refers the default available generated postman tests. The default tests are grouped per type (response, request) - **responseChecks** : All response basic checks. (For now we have only included response checks). - **extendTests**: which refers the custom additions of manual created postman tests. The manual tests are added during -generation. The tests are mapped based on the OpenApi operationId. +generation. The tests are mapped based on the OpenApi operationId. Anything added in `tests` array, will be added to the postman test scripts. +## Postman test suite properties (V1) | name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | |-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| | Response status success (2xx) check | generaStatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | From 4203d908cc6122f6ecb5ae175238b51d10332aed Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 1 Jun 2020 21:40:44 +0200 Subject: [PATCH 04/52] Removed debug Added validation for schema.$ref --- lib/schemaUtils.js | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 75ef51227..56a4bb09f 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2084,7 +2084,6 @@ module.exports = { * @returns {*} array of all query params */ convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { - console.log('options', options); options = _.merge({}, defaultOptions, options); let testEvent = {}, testSuite = {}, @@ -2107,7 +2106,6 @@ module.exports = { if (testSuite.extendTests) { testSuiteExtensions = options.testSuiteSettings.extendTests; } - console.log('optionsTestSuite', testSuiteSettings); } _.forOwn(operation.responses, (response, code) => { @@ -2182,21 +2180,26 @@ module.exports = { } // Add JSON schema check - if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled')) { - let schemaContent = this.getRefObject(content.schema.$ref, components, options), - // When processing a reference, schema.type could also be undefined - resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, - schemaCache, PROCESSING_TYPE.VALIDATION); - - requestsTests.push( - '// Response Validation\n' + - 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + - '\n' + - '// Test whether the response matches the schema\n' + - 'pm.test(\'Schema is valid\', function() {\n' + - ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + - '});\n' - ); + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && + content.schema && content.schema.$ref) { + try { + let schemaContent = this.getRefObject(content.schema.$ref, components, options), + // When processing a reference, schema.type could also be undefined + resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + schemaCache, PROCESSING_TYPE.VALIDATION); + + requestsTests.push( + '// Response Validation\n' + + 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + + '\n' + + '// Test whether the response matches the schema\n' + + 'pm.test(\'Schema is valid\', function() {\n' + + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + + '});\n' + ); + } catch (e) { + console.log('invalid schemaValidation for ', content); + } } } @@ -2210,7 +2213,6 @@ module.exports = { testSuiteExtensions.forEach((testExtension) => { if (operation.operationId && testExtension.openApiOperationId && operation.operationId === testExtension.openApiOperationId) { - console.log('testExtension', testExtension); if (testExtension.tests.length > 0) { testExtension.tests.forEach((postmanTest) => { try { From 1892e5a9e391eee117ce57aaa32ea34a85aa385c Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 2 Jun 2020 07:59:28 +0200 Subject: [PATCH 05/52] Added additional configuration options for the `extendTests` section. It is now possible to `overwrite` all generateTests and extend them with specific settings per OpenApiOperationId. --- README.md | 18 +++++++--- examples/postman-testsuite-advanced.json | 45 ++++++++++++++++++++++++ lib/schemaUtils.js | 21 +++++++++++ 3 files changed, 80 insertions(+), 4 deletions(-) create mode 100644 examples/postman-testsuite-advanced.json diff --git a/README.md b/README.md index eea8ad6b5..73521e34f 100644 --- a/README.md +++ b/README.md @@ -266,11 +266,21 @@ Current postman-testsuite JSON properties ``` The JSON test suite format consists out of 3 parts: -- **version** : which refers the JSON test suite version (not relevant but might handy for future backward compatibility options). -- **generateTests** : which refers the default available generated postman tests. The default tests are grouped per type (response, request) +- **version** : which refers the JSON test suite version +(not relevant but might handy for future backward compatibility options). +- **generateTests** : which refers the default available generated postman tests. +The default tests are grouped per type (response, request) - **responseChecks** : All response basic checks. (For now we have only included response checks). -- **extendTests**: which refers the custom additions of manual created postman tests. The manual tests are added during -generation. The tests are mapped based on the OpenApi operationId. Anything added in `tests` array, will be added to the postman test scripts. +- **extendTests**: which refers the custom additions of manual created postman tests. +The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. +Anything added in `tests` array, will be added to the postman test scripts. + - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended + - **tests (Array)** : Array of additional postman test scripts. + - **responseChecks (array)** : Extends the generateTests `responseChecks` (see "Postman test suite properties") with specifics for the openApiOperationId. + - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. + Default: false + +See "postman-testsuite-advanced.json" for an advanced example of the setting options. ## Postman test suite properties (V1) | name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | diff --git a/examples/postman-testsuite-advanced.json b/examples/postman-testsuite-advanced.json new file mode 100644 index 000000000..c35cefd2c --- /dev/null +++ b/examples/postman-testsuite-advanced.json @@ -0,0 +1,45 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 56a4bb09f..1c28c9628 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2103,9 +2103,22 @@ module.exports = { if (testSuite.generateTests) { testSuiteSettings = options.testSuiteSettings.generateTests; } + if (testSuite.extendTests) { testSuiteExtensions = options.testSuiteSettings.extendTests; + + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId && + testExtension && testExtension.responseChecks) { + // Extend testSuiteSettings with testExtension settings + testSuiteSettings.responseChecks = _.merge( + {}, testSuiteSettings.responseChecks, testExtension.responseChecks + ); + } + }); } + } _.forOwn(operation.responses, (response, code) => { @@ -2213,9 +2226,17 @@ module.exports = { testSuiteExtensions.forEach((testExtension) => { if (operation.operationId && testExtension.openApiOperationId && operation.operationId === testExtension.openApiOperationId) { + + if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { + // Reset generated tests + requestsTests = []; + } + + // Add test extensions if (testExtension.tests.length > 0) { testExtension.tests.forEach((postmanTest) => { try { + // Extend the generated tests, with the test extension scripts requestsTests.push(postmanTest); } catch (e) { console.log('invalid extendTests for ' + testExtension.openApiOperationI); From 979eb805e2e06b29c62dde7b47ce988ff7da1dc7 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 2 Jun 2020 08:03:06 +0200 Subject: [PATCH 06/52] Fixed code to comply to the Travis build checks --- lib/schemaUtils.js | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 1c28c9628..4ab735ddb 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2210,8 +2210,9 @@ module.exports = { ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + '});\n' ); - } catch (e) { - console.log('invalid schemaValidation for ', content); + } + catch (e) { + console.warn('invalid schemaValidation for ', content); } } } @@ -2238,8 +2239,9 @@ module.exports = { try { // Extend the generated tests, with the test extension scripts requestsTests.push(postmanTest); - } catch (e) { - console.log('invalid extendTests for ' + testExtension.openApiOperationI); + } + catch (e) { + console.warn('invalid extendTests for ' + testExtension.openApiOperationI); } }); } From aca00be43a1c0358a36f448cc4497a1571ce793b Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 2 Jun 2020 08:05:56 +0200 Subject: [PATCH 07/52] Readme corrections --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 73521e34f..8d2a8f7f5 100644 --- a/README.md +++ b/README.md @@ -172,7 +172,7 @@ The converter can be used as a CLI tool as well. The following [command line opt - `-c`, `--config ` Used to supply options to the converter -- `-g`, `--generate ` +- `-g ``, `--generate ` Used to generate postman tests given the JSON file with test options - `-h`, `--help` @@ -196,7 +196,7 @@ $ openapi2postmanv2 --test - Generating additional postman tests for the OpenAPi specification ```terminal -$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-test.json +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json ``` ## Conversion Schema @@ -280,7 +280,7 @@ Anything added in `tests` array, will be added to the postman test scripts. - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. Default: false -See "postman-testsuite-advanced.json" for an advanced example of the setting options. +See "postman-testsuite-advanced.json" file for an advanced example of the setting options. ## Postman test suite properties (V1) | name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | From 3bd95844507caf881d5258f222485151d0a200f2 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 2 Jun 2020 08:07:31 +0200 Subject: [PATCH 08/52] Readme corrections --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8d2a8f7f5..8fe440579 100644 --- a/README.md +++ b/README.md @@ -285,8 +285,8 @@ See "postman-testsuite-advanced.json" file for an advanced example of the settin ## Postman test suite properties (V1) | name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | |-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| -| Response status success (2xx) check | generaStatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | +| Response status success (2xx) check | StatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | | Response time check | responseTime | boolean | false | enabled | maxMs 300 | Adds the check if the response of the postman request is within a number of ms. | true | TEST GENERATION | | Response content-type check | contentType | boolean | false | enabled | | Adds the check if the postman response header is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | | Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | -| Response Schema validation check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | +| Response Schema validation check | schemaValidation | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | From 7a6f487dbb762980b01de122b98f5686c07be8ca Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 2 Jun 2020 08:42:14 +0200 Subject: [PATCH 09/52] Improved readme --- README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 8fe440579..556b1682f 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ 1. [Options](#options) 2. [Usage](#usage) 4. [Conversion Schema](#conversion-schema) +5. [Postman test suite generation options](#postman-test-suite-generation-options) --- @@ -169,10 +170,10 @@ The converter can be used as a CLI tool as well. The following [command line opt - `-p`, `--pretty` Used to pretty print the collection object while writing to a file -- `-c`, `--config ` +- `-c `, `--config ` Used to supply options to the converter -- `-g ``, `--generate ` +- `-g `, `--generate ` Used to generate postman tests given the JSON file with test options - `-h`, `--help` @@ -216,7 +217,7 @@ $ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json | request.url.params | parameter (`in = query`) | - | {"key": param.name, "value": [link](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.1.md#style-examples)}| | api_key in (query or header) | components.securitySchemes.api_key | - || -## Postman test generation +## Postman test suite generation options This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. The goal is to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used @@ -276,13 +277,16 @@ The manual tests are added during generation. The tests are mapped based on the Anything added in `tests` array, will be added to the postman test scripts. - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended - **tests (Array)** : Array of additional postman test scripts. - - **responseChecks (array)** : Extends the generateTests `responseChecks` (see "Postman test suite properties") with specifics for the openApiOperationId. + - **responseChecks (array)** : Extends the generateTests `responseChecks` (see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. Default: false See "postman-testsuite-advanced.json" file for an advanced example of the setting options. -## Postman test suite properties (V1) +## Postman test suite properties + +Version 1.0 + | name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | |-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| | Response status success (2xx) check | StatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | From 6fda6013e102d628b613b614f07b19e8bfb81d03 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 5 Jun 2020 17:18:26 +0200 Subject: [PATCH 10/52] Added the Operation method & Path as part of the test description. This helps when running newman tests with reporting via CI (like Azure devops). The reporting will contain the references. --- lib/schemaUtils.js | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 4ab735ddb..ce95c1f1a 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2150,7 +2150,8 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { requestsTests.push( '// Validate status 2xx \n' + - 'pm.test("Status code is 2xx", function () {\n' + + 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + ' - Status code is 2xx", function () {\n' + ' pm.response.to.be.success;\n' + '});\n' ); @@ -2161,7 +2162,8 @@ module.exports = { let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); requestsTests.push( '// Validate response time \n' + - 'pm.test("Response time is less than ' + maxMs + 'ms", function () {\n' + + 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + '});\n' ); @@ -2174,9 +2176,8 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { requestsTests.push( '// Validate content-type \n' + - 'pm.test("Content-Type is ' + contentType + '", function () {\n' + - // ' pm.response.to.have.header("Content-Type");\n' + - // ' pm.response.to.have.header("Content-Type", "' + contentType + '");\n' + + 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + ' - Content-Type is ' + contentType + '", function () {\n' + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + '});\n' ); @@ -2187,7 +2188,8 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { requestsTests.push( '// Response should have JSON Body\n' + - 'pm.test("Response has JSON Body", function () {\n' + + 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + ' - Response has JSON Body", function () {\n' + ' pm.response.to.have.jsonBody();\n' + '});\n'); } @@ -2206,7 +2208,8 @@ module.exports = { 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + '\n' + '// Test whether the response matches the schema\n' + - 'pm.test(\'Schema is valid\', function() {\n' + + 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + ' - Schema is valid", function() {\n' + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + '});\n' ); From 79e13ec3ab3ba3b6ab52a0ed82f59e981fb24e8a Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 5 Jun 2020 17:26:05 +0200 Subject: [PATCH 11/52] Minor optimisation of the method & path naming in the pm.tests --- lib/schemaUtils.js | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index ce95c1f1a..e9a98dc10 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2150,7 +2150,7 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { requestsTests.push( '// Validate status 2xx \n' + - 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Status code is 2xx", function () {\n' + ' pm.response.to.be.success;\n' + '});\n' @@ -2162,7 +2162,7 @@ module.exports = { let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); requestsTests.push( '// Validate response time \n' + - 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + '});\n' @@ -2176,7 +2176,7 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { requestsTests.push( '// Validate content-type \n' + - 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Content-Type is ' + contentType + '", function () {\n' + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + '});\n' @@ -2188,7 +2188,7 @@ module.exports = { if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { requestsTests.push( '// Response should have JSON Body\n' + - 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Response has JSON Body", function () {\n' + ' pm.response.to.have.jsonBody();\n' + '});\n'); @@ -2208,7 +2208,7 @@ module.exports = { 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + '\n' + '// Test whether the response matches the schema\n' + - 'pm.test("[' + operationItem.method + '] ' + operationItem.path + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Schema is valid", function() {\n' + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + '});\n' From 53d2014ff77f794f5907ce11783a59ed424508be Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 13 Jul 2020 13:38:38 +0200 Subject: [PATCH 12/52] Added option to define test suite limits, based on the OperationID. This will only generate tests based on the defined list of operationId's. --- README.md | 1 + examples/postman-testsuite-limited.json | 46 +++++++++++++++++++++++++ lib/schemaUtils.js | 12 +++++++ 3 files changed, 59 insertions(+) create mode 100644 examples/postman-testsuite-limited.json diff --git a/README.md b/README.md index 556b1682f..6ccad7180 100644 --- a/README.md +++ b/README.md @@ -272,6 +272,7 @@ The JSON test suite format consists out of 3 parts: - **generateTests** : which refers the default available generated postman tests. The default tests are grouped per type (response, request) - **responseChecks** : All response basic checks. (For now we have only included response checks). + - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test will be generated for **all** operations). - **extendTests**: which refers the custom additions of manual created postman tests. The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. Anything added in `tests` array, will be added to the postman test scripts. diff --git a/examples/postman-testsuite-limited.json b/examples/postman-testsuite-limited.json new file mode 100644 index 000000000..e0da63299 --- /dev/null +++ b/examples/postman-testsuite-limited.json @@ -0,0 +1,46 @@ +{ + "version": 1.0, + "generateTests": { + "limitOperations": ["get-lists"], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 47c98e38f..f0d21b57b 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2097,6 +2097,7 @@ module.exports = { let testEvent = {}, testSuite = {}, testSuiteSettings = {}, + testSuiteLimits = [], testSuiteExtensions = [], requestsTests = [], swagResponse = {}; @@ -2113,6 +2114,12 @@ module.exports = { testSuiteSettings = options.testSuiteSettings.generateTests; } + // Limit the test generation to the following operations + if (testSuiteSettings.limitOperations) { + testSuiteLimits = testSuiteSettings.limitOperations; + } + + // Extend the generated test with additional operations if (testSuite.extendTests) { testSuiteExtensions = options.testSuiteSettings.extendTests; @@ -2131,6 +2138,11 @@ module.exports = { } _.forOwn(operation.responses, (response, code) => { + // Skip the operations, if there are limits defined + if (operation.operationId && testSuiteLimits.length > 0 && + testSuiteLimits.indexOf(operation.operationId) === -1) { + return; // skip this response + } // Only support 2xx response checks // TODO Investigate how to support other response codes like validation response 4xx or 5xx From 84a52b7afc9260747d3f1c21a1f101c1884f0d67 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 24 Jul 2020 10:53:19 +0200 Subject: [PATCH 13/52] Postman test generation - added support for anyOf, oneOf, allOf, $ref --- lib/schemaUtils.js | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index f0d21b57b..2b1e20001 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2216,13 +2216,16 @@ module.exports = { } // Add JSON schema check - if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && - content.schema && content.schema.$ref) { + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && content.schema) { + let schemaContent = content.schema; + // if (content.schema.$ref) { + // schemaContent = this.getRefObject(content.schema.$ref, components, options); + // } + try { - let schemaContent = this.getRefObject(content.schema.$ref, components, options), - // When processing a reference, schema.type could also be undefined - resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, - schemaCache, PROCESSING_TYPE.VALIDATION); + // When processing a reference, schema.type could also be undefined + let resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + schemaCache, PROCESSING_TYPE.VALIDATION); requestsTests.push( '// Response Validation\n' + @@ -2240,7 +2243,6 @@ module.exports = { } } } - } }); From ac44b63b23c7a19ab8b4eb50362e2cf3c4d86636 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 27 Jul 2020 08:03:12 +0200 Subject: [PATCH 14/52] TestExtension - added check if tests array exists --- lib/schemaUtils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 2b1e20001..fffa60bb6 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2260,7 +2260,7 @@ module.exports = { } // Add test extensions - if (testExtension.tests.length > 0) { + if (testExtension.tests && testExtension.tests.length > 0) { testExtension.tests.forEach((postmanTest) => { try { // Extend the generated tests, with the test extension scripts From c12b7ec2415ed4707e9de4549262d3e85fa2773f Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 21 Aug 2020 15:43:33 +0200 Subject: [PATCH 15/52] Postman test generation - added support for response header tests, where the header is checked to present in the response --- README.md | 1 + examples/postman-testsuite-limited.json | 3 +++ lib/schemaUtils.js | 18 ++++++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/README.md b/README.md index 6ccad7180..ec4b96085 100644 --- a/README.md +++ b/README.md @@ -295,3 +295,4 @@ Version 1.0 | Response content-type check | contentType | boolean | false | enabled | | Adds the check if the postman response header is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | | Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | | Response Schema validation check | schemaValidation | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | +| Response Header presence check | headersPresent | boolean | false | enabled | | Adds the check if the postman response header has the header names present, like defined in the OpenApi spec. | true | TEST GENERATION | diff --git a/examples/postman-testsuite-limited.json b/examples/postman-testsuite-limited.json index e0da63299..38d23eac3 100644 --- a/examples/postman-testsuite-limited.json +++ b/examples/postman-testsuite-limited.json @@ -10,6 +10,9 @@ "enabled": true, "maxMs": 300 }, + "headersPresent": { + "enabled": true + }, "contentType": { "enabled": true }, diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 92c292092..8a1417142 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2327,6 +2327,24 @@ module.exports = { } }); + + // Process the response header + _.forOwn(swagResponse.headers, (header, headerKey) => { + + if (headerKey && header.name) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { + requestsTests.push( + '// Validate header \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response header ' + header.name + ' is present", function () {\n' + + ' pm.response.to.have.header("' + header.name + '");\n' + + '});\n' + ); + } + } + + }); }); // Add test extensions that are defined in the test suite file From 39231dded78f47e0db32f8cad5150b4fa600e5b6 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Wed, 18 Nov 2020 15:30:16 +0100 Subject: [PATCH 16/52] Spelling corrections --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 309e0a25a..a0844bdfa 100644 --- a/README.md +++ b/README.md @@ -175,8 +175,8 @@ The converter can be used as a CLI tool as well. The following [command line opt - `-c`, `--options-config` Used to supply options to the converter through config file, for complete options details see [here](/OPTIONS.md) -`-g `, `--generate ` - Used to generate postman tests given the JSON file with test options, for complete options details see [here](/TESTGENERAION.md) +- `-g `, `--generate ` + Used to generate postman tests given the JSON file with test options, for complete options details see [here](/TESTGENERATION.md) - `-h`, `--help` Specifies all the options along with a few usage examples on the terminal From 7306501630b1af6734987134a8cb39db37b525ca Mon Sep 17 00:00:00 2001 From: Tim <> Date: Wed, 18 Nov 2020 16:36:43 +0100 Subject: [PATCH 17/52] Spelling corrections --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a0844bdfa..2f188ca90 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ 1. [Options](#options) 2. [Usage](#usage) 4. [Conversion Schema](#conversion-schema) -5. [Postman test suite generation options](/OPTIONS.md) +5. [Postman test suite generation options](/TESTGENERATION.md) --- From e9cfa48a111652d54fa1eeb098a52193c3296442 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 18 Dec 2020 15:44:57 +0100 Subject: [PATCH 18/52] Corrected mapping of program paramaters with ConfigFile --- bin/openapi2postmanv2.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index f1a480d7a..3b2cf026e 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -78,7 +78,7 @@ inputFile = program.spec; outputFile = program.output || false; testFlag = program.test || false; prettyPrintFlag = program.pretty || false; -configFile = program.config || false; +configFile = program.optionsConfig || false; definedOptions = program.options || {}; testsuiteFile = program.generate || false; swaggerInput; From 70f6906c2acb90eb4f48b17f519d8e5f9b31a7c9 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 18 Dec 2020 17:01:05 +0100 Subject: [PATCH 19/52] Provided support for the option-cli because of the https://github.com/postmanlabs/openapi-to-postman/issues/315 --- bin/openapi2postmanv2.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 3b2cf026e..3f15691bb 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -52,7 +52,7 @@ program .option('-t, --test', 'Test the OPENAPI converter') .option('-p, --pretty', 'Pretty print the JSON file') .option('-c, --options-config ', 'JSON file containing Converter options') - .option('-O, --options ', 'comma separated list of options', parseOptions) + .option('-O, --options-cli ', 'comma separated list of options', parseOptions) .option('-g, --generate ', 'Generate postman tests given the JSON file with test options'); program.on('--help', function() { @@ -79,7 +79,7 @@ outputFile = program.output || false; testFlag = program.test || false; prettyPrintFlag = program.pretty || false; configFile = program.optionsConfig || false; -definedOptions = program.options || {}; +definedOptions = program.optionsCli || {}; testsuiteFile = program.generate || false; swaggerInput; swaggerData; From 22f754e048868e70bf5056b3f4ef47905a43ad32 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 18 Dec 2020 18:25:36 +0100 Subject: [PATCH 20/52] Added an additional check if the definedOptions is not empty --- bin/openapi2postmanv2.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 3f15691bb..983c28526 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -125,7 +125,7 @@ function convert(swaggerData) { } // override options provided via cli - if (definedOptions) { + if (definedOptions && !_.isEmpty(definedOptions)) { options = definedOptions; } From 250e59eda54ba692d03c63a4234b4321c39bc915 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 21 Dec 2020 13:09:35 +0100 Subject: [PATCH 21/52] Corrected version parameters to resolve conflict --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2f188ca90..1f3d4d39e 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ The converter can be used as a CLI tool as well. The following [command line opt `openapi2postmanv2 [options]` ### Options -- `-V`, `--version` +- `-v`, `--version` Specifies the version of the converter - `-s `, `--spec ` From 13d54f6ba4309d0ef4d9946498285e55de18d067 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 21 Dec 2020 13:12:07 +0100 Subject: [PATCH 22/52] Corrected version parameters to resolve conflict --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1f3d4d39e..bed0d8c30 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ The converter can be used as a CLI tool as well. The following [command line opt `openapi2postmanv2 [options]` ### Options -- `-v`, `--version` +- `-v`, `--version` Specifies the version of the converter - `-s `, `--spec ` From 8759aaa58e74e26eb575a31cc3b397251f5d7587 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 28 Jan 2021 10:06:09 +0100 Subject: [PATCH 23/52] Bumped to openapi-to-postman 2.1.0 --- README.md | 31 +-- TESTGENERATION.md | 79 ++++++++ bin/openapi2postmanv2.js | 12 +- examples/postman-testsuite-advanced.json | 45 +++++ examples/postman-testsuite-limited.json | 49 +++++ examples/postman-testsuite.json | 32 ++++ lib/schemaUtils.js | 230 +++++++++++++++++++++++ lib/schemapack.js | 4 + 8 files changed, 470 insertions(+), 12 deletions(-) create mode 100644 TESTGENERATION.md create mode 100644 examples/postman-testsuite-advanced.json create mode 100644 examples/postman-testsuite-limited.json create mode 100644 examples/postman-testsuite.json diff --git a/README.md b/README.md index b9deb359b..9c39d28f0 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,14 @@ -![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) +![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) -*Supercharge your API workflow.* +*Supercharge your API workflow.* *Modern software is built on APIs. Postman helps you develop APIs faster.* # OpenAPI 3.0 to Postman Collection v2.1.0 Converter [![Build Status](https://travis-ci.org/postmanlabs/openapi-to-postman.svg?branch=master)](https://travis-ci.org/postmanlabs/openapi-to-postman) -#### Contents +#### Contents 1. [Getting Started](#getting-started) 2. [Using the converter as a NodeJS module](#using-the-converter-as-a-nodejs-module) @@ -21,6 +21,7 @@ 1. [Options](#options) 2. [Usage](#usage) 4. [Conversion Schema](#conversion-schema) +5. [Postman test suite generation options](/TESTGENERATION.md) --- @@ -94,7 +95,7 @@ Check out complete list of options and their usage at [OPTIONS.md](/OPTIONS.md) ### ConversionResult -- `result` - Flag responsible for providing a status whether the conversion was successful or not +- `result` - Flag responsible for providing a status whether the conversion was successful or not - `reason` - Provides the reason for an unsuccessful conversion, defined only if result: false @@ -153,28 +154,31 @@ The converter can be used as a CLI tool as well. The following [command line opt `openapi2postmanv2 [options]` ### Options -- `-v`, `--version` +- `-v`, `--version` Specifies the version of the converter -- `-s `, `--spec ` +- `-s `, `--spec ` Used to specify the OpenAPI specification (file path) which is to be converted -- `-o `, `--output ` +- `-o `, `--output ` Used to specify the destination file in which the collection is to be written -- `-t`, `--test` +- `-t`, `--test` Used to test the collection with an in-built sample specification -- `-p`, `--pretty` +- `-p`, `--pretty` Used to pretty print the collection object while writing to a file - `-O`, `--options` Used to supply options to the converter, for complete options details see [here](/OPTIONS.md) -- `-c`, `--options-config` +- `-c`, `--options-config` Used to supply options to the converter through config file, for complete options details see [here](/OPTIONS.md) -- `-h`, `--help` +- `-g `, `--generate ` + Used to generate postman tests given the JSON file with test options, for complete options details see [here](/TESTGENERATION.md) + +- `-h`, `--help` Specifies all the options along with a few usage examples on the terminal @@ -198,6 +202,11 @@ $ openapi2postmanv2 -s spec.yaml -o collection.json -p -c ./examples/cli-option $ openapi2postmanv2 --test ``` +- Generating additional postman tests for the OpenAPi specification +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json +``` + ## Conversion Schema | *postman* | *openapi* | *options* | *examples* | diff --git a/TESTGENERATION.md b/TESTGENERATION.md new file mode 100644 index 000000000..530dd387e --- /dev/null +++ b/TESTGENERATION.md @@ -0,0 +1,79 @@ +## Postman test suite generation options + +This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. +The goal is to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used +by the Newman test runner. + +To generate the tests, define a JSON file like the example (postman-testsuite.json) below and run the CLI with the --generate option. + +```terminal +$ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json +``` + +Current postman-testsuite JSON properties + +```JSON +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" + ] + } + ] +} +``` + +The JSON test suite format consists out of 3 parts: +- **version** : which refers the JSON test suite version +(not relevant but might handy for future backward compatibility options). +- **generateTests** : which refers the default available generated postman tests. +The default tests are grouped per type (response, request) + - **responseChecks** : All response basic checks. (For now we have only included response checks). + - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test will be generated for **all** operations). +- **extendTests**: which refers the custom additions of manual created postman tests. +The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. +Anything added in `tests` array, will be added to the postman test scripts. + - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended + - **tests (Array)** : Array of additional postman test scripts. + - **responseChecks (array)** : Extends the generateTests `responseChecks` (see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. + - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. + Default: false + +See "postman-testsuite-advanced.json" file for an advanced example of the setting options. + +## Postman test suite properties + +Version 1.0 + +| name | id | type | default/0 | availableOptions/0 | availableOptions/1 | description | external | usage/0 | +|-------------------------------------|---------------------|---------|-----------|--------------------|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------| +| Response status success (2xx) check | StatusSuccess | boolean | false | enabled | | Adds the check if the response of the postman request return a 2xx | true | TEST GENERATION | +| Response time check | responseTime | boolean | false | enabled | maxMs 300 | Adds the check if the response of the postman request is within a number of ms. | true | TEST GENERATION | +| Response content-type check | contentType | boolean | false | enabled | | Adds the check if the postman response header is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | +| Response Schema validation check | schemaValidation | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | +| Response Header presence check | headersPresent | boolean | false | enabled | | Adds the check if the postman response header has the header names present, like defined in the OpenApi spec. | true | TEST GENERATION | diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 6d7b319e1..000926ea3 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -11,6 +11,7 @@ var _ = require('lodash'), configFile, definedOptions, testFlag, + testsuiteFile, swaggerInput, swaggerData; @@ -51,7 +52,8 @@ program .option('-t, --test', 'Test the OPENAPI converter') .option('-p, --pretty', 'Pretty print the JSON file') .option('-c, --options-config ', 'JSON file containing Converter options') - .option('-O, --options ', 'comma separated list of options', parseOptions); + .option('-O, --options ', 'comma separated list of options', parseOptions) + .option('-g, --generate ', 'Generate postman tests given the JSON file with test options'); program.on('--help', function() { /* eslint-disable */ @@ -78,6 +80,7 @@ testFlag = program.test || false; prettyPrintFlag = program.pretty || false; configFile = program.optionsConfig || false; definedOptions = (!(program.options instanceof Array) ? program.options : {}); +testsuiteFile = program.generate || false; swaggerInput; swaggerData; @@ -126,6 +129,13 @@ function convert(swaggerData) { options = definedOptions; } + if (testsuiteFile) { + testsuiteFile = path.resolve(testsuiteFile); + console.log('Testsuite file: ', testsuiteFile); // eslint-disable-line no-console + options.testSuite = true; + options.testSuiteSettings = JSON.parse(fs.readFileSync(testsuiteFile, 'utf8')); + } + Converter.convert({ type: 'string', data: swaggerData diff --git a/examples/postman-testsuite-advanced.json b/examples/postman-testsuite-advanced.json new file mode 100644 index 000000000..c35cefd2c --- /dev/null +++ b/examples/postman-testsuite-advanced.json @@ -0,0 +1,45 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/examples/postman-testsuite-limited.json b/examples/postman-testsuite-limited.json new file mode 100644 index 000000000..38d23eac3 --- /dev/null +++ b/examples/postman-testsuite-limited.json @@ -0,0 +1,49 @@ +{ + "version": 1.0, + "generateTests": { + "limitOperations": ["get-lists"], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "overwrite": true, + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + }, + { + "openApiOperationId": "Lists_Get", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ], + "responseChecks": { + "responseTime": { + "maxMs": 1000 + } + } + } + ] +} diff --git a/examples/postman-testsuite.json b/examples/postman-testsuite.json new file mode 100644 index 000000000..f8428d6f5 --- /dev/null +++ b/examples/postman-testsuite.json @@ -0,0 +1,32 @@ +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);}); " + ] + } + ] +} diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 22921fc9d..7ba6e89e9 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2097,6 +2097,7 @@ module.exports = { displayUrl, reqUrl = '/' + operationItem.path, pmBody, + pmTests, authMeta, swagResponse, localServers = _.get(operationItem, 'properties.servers'), @@ -2379,9 +2380,238 @@ module.exports = { }); } + // Generate tests for the response + pmTests = this.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); + + if (!_.isEmpty(pmTests)) { + // Add test event + item.events.add(pmTests); + } + return item; }, + /** + * function to convert an openapi reponse object to object of postman tests + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} components - components defined in the OAS spec. These are used to + * resolve references while generating params. + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @param {object} schemaCache - object storing schemaFaker and schmeResolution caches + * @returns {*} array of all query params + */ + convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { + options = _.merge({}, defaultOptions, options); + let testEvent = {}, + testSuite = {}, + testSuiteSettings = {}, + testSuiteLimits = [], + testSuiteExtensions = [], + requestsTests = [], + swagResponse = {}; + + // Check for test suite flag, abort early + if (!options.testSuite) { + return testEvent; + } + + // Check test suite for later usage, potentially convert object + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + if (testSuite.generateTests) { + testSuiteSettings = options.testSuiteSettings.generateTests; + } + + // Limit the test generation to the following operations + if (testSuiteSettings.limitOperations) { + testSuiteLimits = testSuiteSettings.limitOperations; + } + + // Extend the generated test with additional operations + if (testSuite.extendTests) { + testSuiteExtensions = options.testSuiteSettings.extendTests; + + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId && + testExtension && testExtension.responseChecks) { + // Extend testSuiteSettings with testExtension settings + testSuiteSettings.responseChecks = _.merge( + {}, testSuiteSettings.responseChecks, testExtension.responseChecks + ); + } + }); + } + + } + + _.forOwn(operation.responses, (response, code) => { + // Skip the operations, if there are limits defined + if (operation.operationId && testSuiteLimits.length > 0 && + testSuiteLimits.indexOf(operation.operationId) === -1) { + return; // skip this response + } + + // Only support 2xx response checks + // TODO Investigate how to support other response codes like validation response 4xx or 5xx + if (!_.inRange(code, 200, 299)) { + return; // skip this response + } + + // Format response + swagResponse = response; + if (response.$ref) { + swagResponse = this.getRefObject(response.$ref, components, options); + } + + // Add status code check + // TODO Validate other request codes + // if (operationItem.method.toUpperCase() === 'GET') { + // requestsTests.push( + // '// Validate status code \n'+ + // 'pm.test("Status code should be '+code+'", function () {\n' + + // ' pm.response.to.have.status('+code+');\n' + + // '});\n' + // ) + // } + + // Add status success check + if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { + requestsTests.push( + '// Validate status 2xx \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Status code is 2xx", function () {\n' + + ' pm.response.to.be.success;\n' + + '});\n' + ); + } + + // Add response timer check + if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { + let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); + requestsTests.push( + '// Validate response time \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + + '});\n' + ); + } + + // Process the response content + _.forOwn(swagResponse.content, (content, contentType) => { + if (contentType) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { + requestsTests.push( + '// Validate content-type \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content-Type is ' + contentType + '", function () {\n' + + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + + '});\n' + ); + } + + if (contentType === APP_JSON) { + // Add JSON body check + if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { + requestsTests.push( + '// Response should have JSON Body\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response has JSON Body", function () {\n' + + ' pm.response.to.have.jsonBody();\n' + + '});\n'); + } + + // Add JSON schema check + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && content.schema) { + let schemaContent = content.schema; + // if (content.schema.$ref) { + // schemaContent = this.getRefObject(content.schema.$ref, components, options); + // } + + try { + // When processing a reference, schema.type could also be undefined + let resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + schemaCache, PROCESSING_TYPE.VALIDATION); + + requestsTests.push( + '// Response Validation\n' + + 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + + '\n' + + '// Test whether the response matches the schema\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Schema is valid", function() {\n' + + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + + '});\n' + ); + } + catch (e) { + console.warn('invalid schemaValidation for ', content); + } + } + } + } + + }); + + // Process the response header + _.forOwn(swagResponse.headers, (header, headerKey) => { + + if (headerKey && header.name) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { + requestsTests.push( + '// Validate header \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response header ' + header.name + ' is present", function () {\n' + + ' pm.response.to.have.header("' + header.name + '");\n' + + '});\n' + ); + } + } + + }); + }); + + // Add test extensions that are defined in the test suite file + if (testSuiteExtensions.length > 0) { + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId) { + + if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { + // Reset generated tests + requestsTests = []; + } + + // Add test extensions + if (testExtension.tests && testExtension.tests.length > 0) { + testExtension.tests.forEach((postmanTest) => { + try { + // Extend the generated tests, with the test extension scripts + requestsTests.push(postmanTest); + } + catch (e) { + console.warn('invalid extendTests for ' + testExtension.openApiOperationI); + } + }); + } + } + }); + } + + // Add tests to postman item + if (requestsTests.length > 0) { + testEvent = { + listen: 'test', + script: requestsTests + }; + } + return testEvent; + }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object diff --git a/lib/schemapack.js b/lib/schemapack.js index 46da3dce5..557c52bf1 100644 --- a/lib/schemapack.js +++ b/lib/schemapack.js @@ -45,6 +45,10 @@ class SchemaPack { let indentCharacter = this.computedOptions.indentCharacter; this.computedOptions.indentCharacter = indentCharacter === 'tab' ? '\t' : ' '; + // hardcoding the test suite options CLI mapping - not exposed to postman users yet + this.computedOptions.testSuite = options.testSuite; + this.computedOptions.testSuiteSettings = options.testSuiteSettings; + this.validate(); } From b7403c09bf1e19ccc183705d2781e095687a08ee Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Feb 2021 19:03:31 +0100 Subject: [PATCH 24/52] Added option to overwrite/extend the Request Body with dynamic values --- TESTGENERATION.md | 52 ++++++++++++++++++---- examples/postman-testsuite-overwrite.json | 47 ++++++++++++++++++++ lib/schemaUtils.js | 54 +++++++++++++++++++++++ 3 files changed, 145 insertions(+), 8 deletions(-) create mode 100644 examples/postman-testsuite-overwrite.json diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 530dd387e..b8947622a 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -43,8 +43,26 @@ Current postman-testsuite JSON properties "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" ] } + ], + "overwriteRequests": [ + { + "openApiOperationId": "post-accounts", + "overwriteKeyValues": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientId", + "value": "{{$guid}}", + "overwrite": true + } + ] + } ] } + ``` The JSON test suite format consists out of 3 parts: @@ -54,14 +72,8 @@ The JSON test suite format consists out of 3 parts: The default tests are grouped per type (response, request) - **responseChecks** : All response basic checks. (For now we have only included response checks). - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test will be generated for **all** operations). -- **extendTests**: which refers the custom additions of manual created postman tests. -The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. -Anything added in `tests` array, will be added to the postman test scripts. - - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended - - **tests (Array)** : Array of additional postman test scripts. - - **responseChecks (array)** : Extends the generateTests `responseChecks` (see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. - - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. - Default: false +- **extendTests**: which refers the custom additions of manual created postman tests. (see [Postman test suite extendTests](#postman-test-suite-extendtests)) +- **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. (see [Postman test suite overwriteRequests](#postman-test-suite-overwriterequests)) See "postman-testsuite-advanced.json" file for an advanced example of the setting options. @@ -77,3 +89,27 @@ Version 1.0 | Response JSON body format check | jsonBody | boolean | false | enabled | | Adds the check if the postman response body is matching the expected content-type defined in the OpenApi spec. | true | TEST GENERATION | | Response Schema validation check | schemaValidation | boolean | false | enabled | | Adds the check if the postman response body is matching the JSON schema defined in the OpenApi spec. The JSON schema is inserted inline in the postman test. | true | TEST GENERATION | | Response Header presence check | headersPresent | boolean | false | enabled | | Adds the check if the postman response header has the header names present, like defined in the OpenApi spec. | true | TEST GENERATION | + +## Postman test suite extendTests + +The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. +Anything added in `tests` array, will be added to the postman test scripts. +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended +- **tests (Array)** : Array of additional postman test scripts. +- **responseChecks (array)** : Extends the generateTests `responseChecks` (see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. +- **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. + Default: false + +## Postman test suite overwriteRequests + +To facilitate automation, you might want to modify property values with "randomized" or specific values. +The overwrites are mapped based on the OpenApi operationId. +Anything added in `overwriteKeyValues` array, will be used to modify to the postman request body. + +Properties explained: +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended +- **overwriteKeyValues (Array)** : Array of additional postman test scripts. + +- **key (string)** : The key that will be targetted in the request body to overwrite/extend. +- **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. +- **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request body value. diff --git a/examples/postman-testsuite-overwrite.json b/examples/postman-testsuite-overwrite.json new file mode 100644 index 000000000..3597cba4a --- /dev/null +++ b/examples/postman-testsuite-overwrite.json @@ -0,0 +1,47 @@ +{ + "version": 1.0, + "generateTests": { + "limitOperations": [ + "get-lists" + ], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [], + "overwriteRequests": [ + { + "openApiOperationId": "post-accounts", + "overwriteKeyValues": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientGuid", + "value": "{{$guid}}", + "overwrite": true + } + ] + } + ] +} diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 7ba6e89e9..c8a8cc38b 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2299,6 +2299,8 @@ module.exports = { reqBody = this.getRefObject(reqBody.$ref, components, options); } pmBody = this.convertToPmBody(reqBody, REQUEST_TYPE.ROOT, components, options, schemaCache); + // Modify the request body for the test suites + pmBody.body = this.overwriteRequestBodyByTests(pmBody.body, operation, operationItem, options); item.request.body = pmBody.body; item.request.addHeader(pmBody.contentHeader); // extra form headers if encoding is present in request Body. @@ -2612,6 +2614,58 @@ module.exports = { return testEvent; }, + /** + * function to overwrite a request body with values defined by the postman testsuite + * @param {*} postman requestBody object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestBodyByTests: function (requestBody, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestBody = Object.assign({}, requestBody), // clone requestBody before manipulation + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is RAW requestBody, abort early + if (!options.testSuite && !requestBody.raw) { + return requestBody; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + // Add test extensions that are defined in the test suite file + // Extend the generated test with additional operations + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find(or => or.openApiOperationId === operation.operationId); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteKeyValues) { + overwriteValues = testSuiteOverwriteRequest.overwriteKeyValues; + + // Overwrite values for Keys + let bodyData = JSON.parse(testRequestBody.raw); + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && overwriteValue.value) { + let orgValue = _.get(bodyData, overwriteValue.key); + let newValue = overwriteValue.value; + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + bodyData = _.set(bodyData, overwriteValue.key, newValue); + } + }) + testRequestBody.raw = JSON.stringify(bodyData, null, 4) + } + } + + return testRequestBody; + }, /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From 192364748b76b0b01a12497daa6402893610a586 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Feb 2021 19:16:21 +0100 Subject: [PATCH 25/52] Added option to overwrite/extend the Request Body with dynamic values example to the documentation --- TESTGENERATION.md | 76 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index b8947622a..1bca32cec 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -113,3 +113,79 @@ Properties explained: - **key (string)** : The key that will be targetted in the request body to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request body value. + +Postman request body before: + +```JSON +{ + "name": "account0-test", + "clientGuid": "ABC-123-DEF-456" +} +``` + +OpenAPI to Postman Testsuite Configuration: + +```JSON +{ + "version": 1.0, + "generateTests": { + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": true, + "maxMs": 300 + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [ + { + "openApiOperationId": "get-lists", + "tests": [ + "pm.test('200 ok', function(){pm.response.to.have.status(200);});", + "pm.test('check userId after create', function(){Number.isInteger(responseBody);});" + ] + } + ], + "overwriteRequests": [ + { + "openApiOperationId": "post-accounts", + "overwriteKeyValues": [ + { + "key": "name", + "value": "--{{$randomInt}}", + "overwrite": false + }, + { + "key": "clientId", + "value": "{{$guid}}", + "overwrite": true + } + ] + } + ] +} + +``` + +This will extend the "name" value with the "--{{$randomInt}}" and overwrite the "clientGuid" with the "{{$guid}}". + +Postman request body after: +```JSON +{ + "name": "account0-test--{{$randomInt}}", + "clientGuid": "{{$guid}}" +} +``` + +This is an example where we leverage the Postman Dynamic variables, but also static values can be used to overwrite/extend. From a187c7c8866c901b4019bad76eff61a0278b7a40 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Feb 2021 19:30:57 +0100 Subject: [PATCH 26/52] Added option to overwrite/extend the Request Body with dynamic values example to the documentation --- TESTGENERATION.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 1bca32cec..d95e43a91 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -167,7 +167,7 @@ OpenAPI to Postman Testsuite Configuration: "overwrite": false }, { - "key": "clientId", + "key": "clientGuid", "value": "{{$guid}}", "overwrite": true } From 723237de2de5b83b3500b6683b7d3a1477f9be00 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Feb 2021 19:33:18 +0100 Subject: [PATCH 27/52] Added option to overwrite/extend the Request Body with dynamic values example to the documentation --- TESTGENERATION.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index d95e43a91..8d3d6cdc1 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -107,8 +107,8 @@ The overwrites are mapped based on the OpenApi operationId. Anything added in `overwriteKeyValues` array, will be used to modify to the postman request body. Properties explained: -- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended -- **overwriteKeyValues (Array)** : Array of additional postman test scripts. +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be extended +- **overwriteKeyValues (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - **key (string)** : The key that will be targetted in the request body to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. From 3dd051cfe5ba2e35c68379a47d016a16bbb07d2a Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Feb 2021 21:18:48 +0100 Subject: [PATCH 28/52] Added option to overwrite/extend the Request Body with dynamic values example to the documentation --- TESTGENERATION.md | 8 ++++---- examples/postman-testsuite-overwrite.json | 2 +- lib/schemaUtils.js | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 8d3d6cdc1..4a57bf9c3 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -47,7 +47,7 @@ Current postman-testsuite JSON properties "overwriteRequests": [ { "openApiOperationId": "post-accounts", - "overwriteKeyValues": [ + "overwriteRequestBody": [ { "key": "name", "value": "--{{$randomInt}}", @@ -104,11 +104,11 @@ Anything added in `tests` array, will be added to the postman test scripts. To facilitate automation, you might want to modify property values with "randomized" or specific values. The overwrites are mapped based on the OpenApi operationId. -Anything added in `overwriteKeyValues` array, will be used to modify to the postman request body. +Anything added in `overwriteRequestBody` array, will be used to modify to the postman request body. Properties explained: - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be extended -- **overwriteKeyValues (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. +- **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - **key (string)** : The key that will be targetted in the request body to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. @@ -160,7 +160,7 @@ OpenAPI to Postman Testsuite Configuration: "overwriteRequests": [ { "openApiOperationId": "post-accounts", - "overwriteKeyValues": [ + "overwriteRequestBody": [ { "key": "name", "value": "--{{$randomInt}}", diff --git a/examples/postman-testsuite-overwrite.json b/examples/postman-testsuite-overwrite.json index 3597cba4a..a4701f2e7 100644 --- a/examples/postman-testsuite-overwrite.json +++ b/examples/postman-testsuite-overwrite.json @@ -30,7 +30,7 @@ "overwriteRequests": [ { "openApiOperationId": "post-accounts", - "overwriteKeyValues": [ + "overwriteRequestBody": [ { "key": "name", "value": "--{{$randomInt}}", diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index c8a8cc38b..afe6a922b 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2645,8 +2645,8 @@ module.exports = { testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; testSuiteOverwriteRequest = testSuiteOverwriteRequests.find(or => or.openApiOperationId === operation.operationId); - if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteKeyValues) { - overwriteValues = testSuiteOverwriteRequest.overwriteKeyValues; + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestBody) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestBody; // Overwrite values for Keys let bodyData = JSON.parse(testRequestBody.raw); From 3909bb0d8cd4d7d8169c819ce0e50a5c18e4e6f4 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 25 Feb 2021 15:40:28 +0100 Subject: [PATCH 29/52] Added support for the Postman dynamic values $randomInt}}, {{$randomCreditCardMask}}, {{$randomBankAccount}} number values to overwrite/extend the Request Body --- lib/schemaUtils.js | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index afe6a922b..ec9640157 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2652,20 +2652,34 @@ module.exports = { let bodyData = JSON.parse(testRequestBody.raw); overwriteValues.forEach((overwriteValue) => { if (overwriteValue.key && overwriteValue.value) { - let orgValue = _.get(bodyData, overwriteValue.key); - let newValue = overwriteValue.value; + let orgValue = _.get(bodyData, overwriteValue.key), + newValue = overwriteValue.value; + if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; + newValue = orgValue + newValue; } + bodyData = _.set(bodyData, overwriteValue.key, newValue); } - }) - testRequestBody.raw = JSON.stringify(bodyData, null, 4) + }); + let bodyString = JSON.stringify(bodyData, null, 4); + + // Handle {{$randomInt}},{{$randomCreditCardMask}} conversion from string to number + let find = ['"{{$randomInt}}"','"{{$randomCreditCardMask}}"','"{{$randomBankAccount}}"']; + let replace = ['{{$randomInt}}','{{$randomCreditCardMask}}','{{$randomBankAccount}}']; + find.forEach(function (item, index) { + let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, "\\$1"); + bodyString = bodyString.replace(new RegExp(escapedFind, 'g'), replace[index]); + }); + + // Set testRequestBody.raw with stringified body + testRequestBody.raw = bodyString; } } return testRequestBody; }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From 6189b07c041e324034cf814dfb67e56a8da9db0a Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 00:32:19 +0100 Subject: [PATCH 30/52] Added support to overwrite request path variables Initial support to automatic get response ID's for POST operations and set them as pm.environment variables --- lib/schemaUtils.js | 88 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 80 insertions(+), 8 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index ec9640157..fa821b1ab 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2273,6 +2273,7 @@ module.exports = { } }); item.request.url.variables.clear(); + reqParams.path = this.overwriteRequestPathByTests(reqParams.path, operation, operationItem, options); item.request.url.variables.assimilate(this.convertPathVariables('param', pathVarArray, reqParams.path, components, options, schemaCache)); @@ -2553,6 +2554,18 @@ module.exports = { console.warn('invalid schemaValidation for ', content); } } + + // Set the ID of the response for chaining options + if (operationItem.method.toUpperCase() === 'POST') { + requestsTests.push( + '// Chaining - Set ' + operation.operationId + ' ID as environment variable \n' + + 'let jsonData = pm.response.json();\n' + + 'if (jsonData.id) {\n' + + ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + + ' console.log("' + operation.operationId + '.id",jsonData.id);\n' + + '};\n'); + } + } } @@ -2616,7 +2629,7 @@ module.exports = { /** * function to overwrite a request body with values defined by the postman testsuite - * @param {*} postman requestBody object + * @param {*} requestBody object * @param {*} operation openapi operation object * @param {*} operationItem - The operation item to get relevant information for the test generation * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. @@ -2625,6 +2638,9 @@ module.exports = { overwriteRequestBodyByTests: function (requestBody, operation, operationItem, options) { options = _.merge({}, defaultOptions, options); let testRequestBody = Object.assign({}, requestBody), // clone requestBody before manipulation + bodyString, + find, + replace, testSuite = {}, overwriteValues = [], testSuiteOverwriteRequest = {}, @@ -2639,11 +2655,12 @@ module.exports = { testSuite = options.testSuiteSettings; } - // Add test extensions that are defined in the test suite file - // Extend the generated test with additional operations if (testSuite.overwriteRequests) { testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; - testSuiteOverwriteRequest = testSuiteOverwriteRequests.find(or => or.openApiOperationId === operation.operationId); + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestBody) { overwriteValues = testSuiteOverwriteRequest.overwriteRequestBody; @@ -2662,13 +2679,13 @@ module.exports = { bodyData = _.set(bodyData, overwriteValue.key, newValue); } }); - let bodyString = JSON.stringify(bodyData, null, 4); + bodyString = JSON.stringify(bodyData, null, 4); // Handle {{$randomInt}},{{$randomCreditCardMask}} conversion from string to number - let find = ['"{{$randomInt}}"','"{{$randomCreditCardMask}}"','"{{$randomBankAccount}}"']; - let replace = ['{{$randomInt}}','{{$randomCreditCardMask}}','{{$randomBankAccount}}']; + find = ['"{{$randomInt}}"', '"{{$randomCreditCardMask}}"', '"{{$randomBankAccount}}"']; + replace = ['{{$randomInt}}', '{{$randomCreditCardMask}}', '{{$randomBankAccount}}']; find.forEach(function (item, index) { - let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, "\\$1"); + let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, '\\$1'); bodyString = bodyString.replace(new RegExp(escapedFind, 'g'), replace[index]); }); @@ -2680,6 +2697,61 @@ module.exports = { return testRequestBody; }, + /** + * function to overwrite a request path variables with values defined by the postman testsuite + * @param {*} requestPathVariables request Params Path Variables object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestPathVariables = Object.assign([], requestPathVariables), // clone requestBody before manipulation + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestPathVariables, abort early + if (!options.testSuite && requestPathVariables.length > 0) { + return requestPathVariables; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestPathVariables) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestPathVariables; + + // Overwrite value for path variable name + testRequestPathVariables.forEach((pathVar) => { + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && pathVar.name && overwriteValue.key === pathVar.name && + overwriteValue.value && pathVar.schema && pathVar.schema.example) { + let orgValue = pathVar.schema.example, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + pathVar.schema.example = newValue; + } + }); + }); + } + } + return testRequestPathVariables; + }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From 8c0e104321f74c8c0738e901b2d7ee7369148f28 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 00:52:11 +0100 Subject: [PATCH 31/52] Added support to overwrite request path variables Initial support to automatic get response ID's for POST operations and set them as pm.environment variables --- TESTGENERATION.md | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 4a57bf9c3..d901eaac7 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -108,11 +108,14 @@ Anything added in `overwriteRequestBody` array, will be used to modify to the po Properties explained: - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be extended +- **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path Variables. + - **key (string)** : The key that will be targeted in the request Path variables to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. + - **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request Path variable value. - **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - -- **key (string)** : The key that will be targetted in the request body to overwrite/extend. -- **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. -- **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request body value. + - **key (string)** : The key that will be targeted in the request body to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. + - **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request body value. Postman request body before: @@ -172,13 +175,24 @@ OpenAPI to Postman Testsuite Configuration: "overwrite": true } ] + }, + { + "openApiOperationId": "delete-account", + "overwriteRequestPathVariables": [ + { + "key": "id", + "value": "99", + "overwrite": true + } + ] } ] } ``` -This will extend the "name" value with the "--{{$randomInt}}" and overwrite the "clientGuid" with the "{{$guid}}". +The `overwriteRequestBody` example will extend the "name" value with the "--{{$randomInt}}" and overwrite the "clientGuid" with the "{{$guid}}". +This will only be applied on the OpenAPI operationId Postman request body after: ```JSON @@ -189,3 +203,5 @@ Postman request body after: ``` This is an example where we leverage the Postman Dynamic variables, but also static values can be used to overwrite/extend. + +The `overwriteRequestPathVariables` example will overwrite the "id" path variable value with the "99", for the "delete-account" OpenAPI operationId. From a4842851942f44dc7574c4c1b25768892f9ca5c2 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 11:12:01 +0100 Subject: [PATCH 32/52] Added support to overwrite request query params --- TESTGENERATION.md | 31 ++++++++++++++++++++++---- lib/schemaUtils.js | 55 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index d901eaac7..77e162523 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -108,14 +108,18 @@ Anything added in `overwriteRequestBody` array, will be used to modify to the po Properties explained: - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be extended +- **overwriteRequestQueryParams (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. + - **key (string)** : The key that will be targeted in the request Query Param to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request Query Param OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request Path variable value. - **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path Variables. - **key (string)** : The key that will be targeted in the request Path variables to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - - **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request Path variable value. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request Path variable value. - **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - **key (string)** : The key that will be targeted in the request body to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - - **overwrite (Boolean true/false)** : Overwrites the request body value OR attach the value to the original request body value. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request body value. Postman request body before: @@ -161,6 +165,21 @@ OpenAPI to Postman Testsuite Configuration: } ], "overwriteRequests": [ + { + "openApiOperationId": "get-accounts", + "overwriteRequestQueryParams": [ + { + "key": "$count", + "value": true, + "overwrite": true + }, + { + "key": "$filter", + "value": "{{$randomInt}}", + "overwrite": false + } + ] + }, { "openApiOperationId": "post-accounts", "overwriteRequestBody": [ @@ -190,8 +209,11 @@ OpenAPI to Postman Testsuite Configuration: } ``` +The `overwriteRequestQueryParams` example will overwrite the "$count" query param value with the boolean "true" and the +"$filter" with dynamic value for "$randomInt", for the "get-accounts" OpenAPI operationId. -The `overwriteRequestBody` example will extend the "name" value with the "--{{$randomInt}}" and overwrite the "clientGuid" with the "{{$guid}}". +The `overwriteRequestBody` example will extend the "name" value with the "--{{$randomInt}}" and overwrite the +"clientGuid" with the "{{$guid}}". This will only be applied on the OpenAPI operationId Postman request body after: @@ -204,4 +226,5 @@ Postman request body after: This is an example where we leverage the Postman Dynamic variables, but also static values can be used to overwrite/extend. -The `overwriteRequestPathVariables` example will overwrite the "id" path variable value with the "99", for the "delete-account" OpenAPI operationId. +The `overwriteRequestPathVariables` example will overwrite the "id" path variable value with the "99", for the +"delete-account" OpenAPI operationId. diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index fa821b1ab..e72197407 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2253,6 +2253,7 @@ module.exports = { _.forEach(reqParams.query, (queryParam) => { this.convertToPmQueryParameters(queryParam, REQUEST_TYPE.ROOT, components, options, schemaCache) .forEach((pmParam) => { + pmParam = this.overwriteRequestQueryParamByTests(pmParam, operation, operationItem, options); item.request.url.addQueryParams(pmParam); }); }); @@ -2562,7 +2563,7 @@ module.exports = { 'let jsonData = pm.response.json();\n' + 'if (jsonData.id) {\n' + ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + - ' console.log("' + operation.operationId + '.id",jsonData.id);\n' + + ' console.log("Chaining - use {{' + operation.operationId + '.id}} as variable", jsonData.id);\n' + '};\n'); } @@ -2752,6 +2753,58 @@ module.exports = { return testRequestPathVariables; }, + /** + * function to overwrite a request path variables with values defined by the postman testsuite + * @param {*} requestQueryParam request Query Param object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestQueryParamByTests: function (requestQueryParam, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestBody before manipulation + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestPathVariables, abort early + if (!options.testSuite && !requestQueryParam.key) { + return requestPathVariables; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestQueryParams) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestQueryParams; + + // Overwrite value for query param key + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key && + overwriteValue.value && testRequestQueryParam.value) { + let orgValue = testRequestQueryParam.value, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestQueryParam.value = newValue; + } + }); + } + } + return testRequestQueryParam; + }, /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From 465ccc92ce6e928e1a252fe88adab75c288f2e38 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 11:50:13 +0100 Subject: [PATCH 33/52] Added support to overwrite request headers --- TESTGENERATION.md | 98 ++++++++++++++++++++++++++++++++-------------- lib/schemaUtils.js | 60 ++++++++++++++++++++++++++-- 2 files changed, 125 insertions(+), 33 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 77e162523..7e50d2e4c 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -1,10 +1,11 @@ ## Postman test suite generation options -This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. -The goal is to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used -by the Newman test runner. +This CLI option will provide the option to add basic Postman test and/or add manual defined Postman tests. The goal is +to allow easy usage of OpenApi based generated Postman collections with integrated tests, ready to be used by the Newman +test runner. -To generate the tests, define a JSON file like the example (postman-testsuite.json) below and run the CLI with the --generate option. +To generate the tests, define a JSON file like the example (postman-testsuite.json) below and run the CLI with the +--generate option. ```terminal $ openapi2postmanv2 -s spec.yaml -o collection.json -p -g postman-testsuite.json @@ -66,14 +67,18 @@ Current postman-testsuite JSON properties ``` The JSON test suite format consists out of 3 parts: + - **version** : which refers the JSON test suite version -(not relevant but might handy for future backward compatibility options). -- **generateTests** : which refers the default available generated postman tests. -The default tests are grouped per type (response, request) + (not relevant but might handy for future backward compatibility options). +- **generateTests** : which refers the default available generated postman tests. The default tests are grouped per + type (response, request) - **responseChecks** : All response basic checks. (For now we have only included response checks). - - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test will be generated for **all** operations). -- **extendTests**: which refers the custom additions of manual created postman tests. (see [Postman test suite extendTests](#postman-test-suite-extendtests)) -- **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. (see [Postman test suite overwriteRequests](#postman-test-suite-overwriterequests)) + - **limitOperations**: refers to a list of operation IDs for which tests will be generated. (Default not set, so test + will be generated for **all** operations). +- **extendTests**: which refers the custom additions of manual created postman tests. ( + see [Postman test suite extendTests](#postman-test-suite-extendtests)) +- **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. ( + see [Postman test suite overwriteRequests](#postman-test-suite-overwriterequests)) See "postman-testsuite-advanced.json" file for an advanced example of the setting options. @@ -92,34 +97,55 @@ Version 1.0 ## Postman test suite extendTests -The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. -Anything added in `tests` array, will be added to the postman test scripts. +The manual tests are added during generation. The tests are mapped based on the OpenApi operationId. Anything added +in `tests` array, will be added to the postman test scripts. + - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the tests will be extended - **tests (Array)** : Array of additional postman test scripts. -- **responseChecks (array)** : Extends the generateTests `responseChecks` (see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. -- **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. - Default: false +- **responseChecks (array)** : Extends the generateTests `responseChecks` ( + see [Postman test suite properties](#postman-test-suite-properties)) with specifics for the openApiOperationId. +- **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from + the `tests` array. Default: false ## Postman test suite overwriteRequests -To facilitate automation, you might want to modify property values with "randomized" or specific values. -The overwrites are mapped based on the OpenApi operationId. -Anything added in `overwriteRequestBody` array, will be used to modify to the postman request body. +To facilitate automation, you might want to modify property values with "randomized" or specific values. The overwrites +are mapped based on the OpenApi operationId. Anything added in `overwriteRequestBody` array, will be used to modify to +the postman request body. Properties explained: -- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be extended + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be + extended - **overwriteRequestQueryParams (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. - **key (string)** : The key that will be targeted in the request Query Param to overwrite/extend. - - **value (string)** : The value that will be used to overwrite/extend the value in the request Query Param OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request Path variable value. -- **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path Variables. + - **value (string)** : The value that will be used to overwrite/extend the value in the request Query Param OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request query param value OR attach the value + to the original request query param value. +- **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path + Variables. - **key (string)** : The key that will be targeted in the request Path variables to overwrite/extend. - - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request Path variable value. + - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request path variable value OR attach the value to the + original request Path variable value. +- **overwriteRequestHeaders (Array)** : Array of key/value pairs to overwrite in the Postman Request Headers. + - **key (string)** : The key that will be targeted in the request Headers to overwrite/extend. + - **value (string)** : The value that will be used to overwrite/extend the value in the request headers OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request header value OR attach the value to the + original request header value. - **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - **key (string)** : The key that will be targeted in the request body to overwrite/extend. - - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{randomInt}}`. - - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request body value. + - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use + the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) + to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the + original request body value. Postman request body before: @@ -178,6 +204,13 @@ OpenAPI to Postman Testsuite Configuration: "value": "{{$randomInt}}", "overwrite": false } + ], + "overwriteRequestHeaders": [ + { + "key": "team-id", + "value": "{{$randomInt}}", + "overwrite": true + } ] }, { @@ -209,14 +242,18 @@ OpenAPI to Postman Testsuite Configuration: } ``` -The `overwriteRequestQueryParams` example will overwrite the "$count" query param value with the boolean "true" and the + +The `overwriteRequestQueryParams` example will overwrite the "$count" query param value with the boolean "true", the "$filter" with dynamic value for "$randomInt", for the "get-accounts" OpenAPI operationId. +The `overwriteRequestHeaders` example will overwrite the "team-id" header value with a "{{$randomInt}}", for the +"get-accounts" OpenAPI operationId. + The `overwriteRequestBody` example will extend the "name" value with the "--{{$randomInt}}" and overwrite the -"clientGuid" with the "{{$guid}}". -This will only be applied on the OpenAPI operationId +"clientGuid" with the "{{$guid}}". This will only be applied on the OpenAPI operationId Postman request body after: + ```JSON { "name": "account0-test--{{$randomInt}}", @@ -224,7 +261,8 @@ Postman request body after: } ``` -This is an example where we leverage the Postman Dynamic variables, but also static values can be used to overwrite/extend. +This is an example where we leverage the Postman Dynamic variables, but also static values can be used to +overwrite/extend. The `overwriteRequestPathVariables` example will overwrite the "id" path variable value with the "99", for the "delete-account" OpenAPI operationId. diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index e72197407..893119267 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2291,6 +2291,7 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { + header = this.overwriteRequestHeaderByTests(header, operation, operationItem, options); item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); }); @@ -2708,7 +2709,7 @@ module.exports = { */ overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { options = _.merge({}, defaultOptions, options); - let testRequestPathVariables = Object.assign([], requestPathVariables), // clone requestBody before manipulation + let testRequestPathVariables = Object.assign([], requestPathVariables), // clone requestPathVariables testSuite = {}, overwriteValues = [], testSuiteOverwriteRequest = {}, @@ -2763,13 +2764,13 @@ module.exports = { */ overwriteRequestQueryParamByTests: function (requestQueryParam, operation, operationItem, options) { options = _.merge({}, defaultOptions, options); - let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestBody before manipulation + let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestQueryParam testSuite = {}, overwriteValues = [], testSuiteOverwriteRequest = {}, testSuiteOverwriteRequests = []; - // Check for test suite flag and if there is no requestPathVariables, abort early + // Check for test suite flag and if there is no requestQueryParam.key, abort early if (!options.testSuite && !requestQueryParam.key) { return requestPathVariables; } @@ -2805,6 +2806,59 @@ module.exports = { } return testRequestQueryParam; }, + + /** + * function to overwrite a request header with values defined by the postman testsuite + * @param {*} requestHeader request Header object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestHeaderByTests: function (requestHeader, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestHeader = Object.assign({}, requestHeader), // clone requestHeader + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestHeader.schema, abort early + if (!options.testSuite && !requestHeader.schema) { + return requestHeader; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestHeaders) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestHeaders; + // Overwrite value for header name + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestHeader.name && overwriteValue.key === testRequestHeader.name && + overwriteValue.value && testRequestHeader.schema && testRequestHeader.schema.example) { + let orgValue = testRequestHeader.schema.example, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestHeader.schema.example = newValue; + } + }); + } + } + return testRequestHeader; + }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From dd19f811a67534babaf9f3af0310acc86e039b34 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 11:54:07 +0100 Subject: [PATCH 34/52] Added support to overwrite request headers --- lib/schemaUtils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 893119267..a041a9d36 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2755,7 +2755,7 @@ module.exports = { }, /** - * function to overwrite a request path variables with values defined by the postman testsuite + * function to overwrite a request query params with values defined by the postman testsuite * @param {*} requestQueryParam request Query Param object * @param {*} operation openapi operation object * @param {*} operationItem - The operation item to get relevant information for the test generation From 2cbc98bb54f04c262e9941532e9b6dc743b5b0e1 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 12:30:25 +0100 Subject: [PATCH 35/52] Bumped to openapi-to-postman 2.3.0 + included the requestOverwrite option for Request body, Query Params, Path variables & Headers --- CHANGELOG.md | 13 +- OPTIONS.md | 2 +- lib/options.js | 2 +- lib/parse.js | 3 +- lib/schemaUtils.js | 263 ++++++++++++++---- lib/schemapack.js | 7 +- package-lock.json | 45 +-- package.json | 4 +- .../valid_openapi/required_in_parameters.json | 16 +- .../validationData/implicitHeaderSpec.yaml | 8 +- .../urlencodedBodyCollection.json | 161 +++++++++++ .../validationData/urlencodedBodySpec.yaml | 68 +++++ test/system/structure.test.js | 6 + test/unit/validator.test.js | 42 +++ 14 files changed, 547 insertions(+), 93 deletions(-) create mode 100644 test/data/validationData/urlencodedBodyCollection.json create mode 100644 test/data/validationData/urlencodedBodySpec.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index c1d428aa8..b5f94b72e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # OpenAPI-Postman Changelog +#### v2.3.0 (February 19, 2021) +* Fixed [issue](https://community.postman.com/t/user-feedback-updating-api-elements/13308/13) where content type header was reported missing in validation even if present. +* Feature request [#9046](https://github.com/postmanlabs/postman-app-support/issues/9046) - Added support for validation of request body of type urlencoded. + +#### v2.2.0 (January 28, 2021) +* Fixed [issue](https://community.postman.com/t/openapi-import-with-multi-file-support/9439/8) with import folder flow on respective platform. +* Fixed issue where collection did not contain name for specification with empty string as title. +* Updated description of option to disable optional parameters. + #### v2.1.0 (January 27, 2021) * Fix for [#9404](https://github.com/postmanlabs/postman-app-support/issues/9404) - Fixed issue where incorrect mismatches were reported for valid path variables. * Fixed issue where validation result contained incorrect path variable indices. @@ -22,7 +31,7 @@ * Added support for internal $ref resolution in validation flows. * Fixed issue where parameter resolution was "schema" when "example" was specified. * Add supported formats for schema resolution (deref). -* Fix for [#7643](https://github.com/postmanlabs/postman-app-support/issues/7643), [#7914](https://github.com/postmanlabs/postman-app-support/issues/7914), [#9004](https://github.com/postmanlabs/postman-app-support/issues/9004) - Added support for Auth params in response/example. +* Fix for [#7643](https://github.com/postmanlabs/postman-app-support/issues/7643), [#7914](https://github.com/postmanlabs/postman-app-support/issues/7914), [#9004](https://github.com/postmanlabs/postman-app-support/issues/9004) - Added support for Auth params in response/example. * Bumped up multiple dependecies and dev-dependencies versions to keep them up-to-date. * Updated code coverage tool from deprecated istanbul to nyc. @@ -95,7 +104,7 @@ #### v1.1.13 (April 21, 2020) * Added support for detailed validation body mismatches with option detailedBlobValidation. -* Fix for [#8098](https://github.com/postmanlabs/postman-app-support/issues/8098) - Unable to validate schema with type array. +* Fix for [#8098](https://github.com/postmanlabs/postman-app-support/issues/8098) - Unable to validate schema with type array. * Fixed URIError for invalid URI in transaction. * Fix for [#152](https://github.com/postmanlabs/openapi-to-postman/issues/152) - Path references not resolved due to improver handling of special characters. * Fix for [#160](https://github.com/postmanlabs/openapi-to-postman/issues/160) - Added handling for variables in local servers not a part of a URL segment. All path servers to be added as collection variables. diff --git a/OPTIONS.md b/OPTIONS.md index 9fb3e9431..b2c4c9439 100644 --- a/OPTIONS.md +++ b/OPTIONS.md @@ -16,4 +16,4 @@ suggestAvailableFixes|boolean|-|false|Whether to provide fixes for patching corr validateMetadata|boolean|-|false|Whether to show mismatches for incorrect name and description of request|VALIDATION ignoreUnresolvedVariables|boolean|-|false|Whether to ignore mismatches resulting from unresolved variables in the Postman request|VALIDATION strictRequestMatching|boolean|-|false|Whether requests should be strictly matched with schema operations. Setting to true will not include any matches where the URL path segments don't match exactly.|VALIDATION -disableOptionalParameters|boolean|-|false|Whether to set optional parameters (not required) as disabled|CONVERSION +disableOptionalParameters|boolean|-|false|Whether to set optional parameters as disabled|CONVERSION diff --git a/lib/options.js b/lib/options.js index 3c0cba6f6..c0cc1b6f1 100644 --- a/lib/options.js +++ b/lib/options.js @@ -207,7 +207,7 @@ module.exports = { id: 'disableOptionalParameters', type: 'boolean', default: false, - description: 'Whether to set optional parameters (not required) as disabled', + description: 'Whether to set optional parameters as disabled', external: true, usage: ['CONVERSION'] } diff --git a/lib/parse.js b/lib/parse.js index 9372aba51..16fbadc34 100644 --- a/lib/parse.js +++ b/lib/parse.js @@ -1,7 +1,8 @@ var yaml = require('js-yaml'), fs = require('fs'), _ = require('lodash'), - path = require('path-browserify'), + // use path based on platform it's running on (web or node) + path = typeof process === 'object' ? require('path') : require('path-browserify'), resolver = require('oas-resolver-browser'), yamlParse = require('yaml'); diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index a041a9d36..661cba982 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -1105,6 +1105,35 @@ module.exports = { return helper; }, + /** + * Generates appropriate collection element based on parameter location + * + * @param {Object} param - Parameter object habing key, value and description (optional) + * @param {String} location - Parameter location ("in" property of OAS defined parameter object) + * @returns {Object} - SDK element + */ + generateSdkParam: function (param, location) { + const sdkElementMap = { + 'query': sdk.QueryParam, + 'header': sdk.Header, + 'path': sdk.Variable + }; + + let generatedParam = { + key: param.key, + value: param.value + }; + + _.has(param, 'disabled') && (generatedParam.disabled = param.disabled); + + // use appropriate sdk element based on location parmaeter is in for param generation + if (sdkElementMap[location]) { + generatedParam = new sdkElementMap[location](generatedParam); + } + param.description && (generatedParam.description = param.description); + return generatedParam; + }, + /** * Generates Auth helper for response, params (query, headers) in helper object is added in * request (originalRequest) part of example. @@ -1510,12 +1539,12 @@ module.exports = { case 'form': if (explode && _.isObject(paramValue)) { _.forEach(paramValue, (value, key) => { - pmParams.push({ + pmParams.push(this.generateSdkParam({ key: _.isArray(paramValue) ? paramName : key, value: (value === undefined ? '' : value), description, disabled - }); + }, _.get(param, 'in'))); }); return pmParams; } @@ -1528,12 +1557,12 @@ module.exports = { case 'deepObject': if (_.isObject(paramValue)) { _.forOwn(paramValue, (value, key) => { - pmParams.push({ + pmParams.push(this.generateSdkParam({ key: param.name + '[' + key + ']', value: (value === undefined ? '' : value), description, disabled - }); + }, _.get(param, 'in'))); }); } return pmParams; @@ -1559,12 +1588,12 @@ module.exports = { // prepend starting value to serialised value (valid for empty value also) serialisedValue = startValue + serialisedValue; - pmParams.push({ + pmParams.push(this.generateSdkParam({ key: paramName, value: serialisedValue, description, disabled - }); + }, _.get(param, 'in'))); return pmParams; }, @@ -1672,55 +1701,36 @@ module.exports = { } description = (required ? '(Required) ' : '') + description + (enumValue ? ' (This can only be one of ' + enumValue + ')' : ''); - if (encoding.hasOwnProperty(key)) { - encoding[key].name = key; - encoding[key].schema = { - type: typeof value - }; - encoding[key].description = description; - params = this.convertParamsWithStyle(encoding[key], value, PARAMETER_SOURCE.REQUEST, components, - schemaCache, options); - // TODO: Show warning for incorrect schema if !params - params && params.forEach((element) => { - // Collection v2.1 schema allows urlencoded param value to be only string - if (typeof element.value !== 'string') { - try { - // convert other datatype to string (i.e. number, boolean etc) - element.value = JSON.stringify(element.value); - } - catch (e) { - // JSON.stringify can fail in few cases, suggest invalid type for such case - // eslint-disable-next-line max-len - // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#Exceptions - element.value = 'INVALID_URLENCODED_PARAM_TYPE'; - } - } - delete element.description; - }); - paramArray.push(...params); - } - else { + + !encoding[key] && (encoding[key] = {}); + encoding[key].name = key; + encoding[key].schema = { + type: typeof value + }; + // for urlencoded body serialisation is treated similar to query param + // reference https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#fixed-fields-13 + encoding[key].in = 'query'; + encoding[key].description = description; + + params = this.convertParamsWithStyle(encoding[key], value, PARAMETER_SOURCE.REQUEST, components, + schemaCache, options); + // TODO: Show warning for incorrect schema if !params + params && params.forEach((element) => { // Collection v2.1 schema allows urlencoded param value to be only string - if (typeof value !== 'string') { + if (typeof element.value !== 'string') { try { // convert other datatype to string (i.e. number, boolean etc) - value = JSON.stringify(value); + element.value = JSON.stringify(element.value); } catch (e) { // JSON.stringify can fail in few cases, suggest invalid type for such case // eslint-disable-next-line max-len // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#Exceptions - value = 'INVALID_URLENCODED_PARAM_TYPE'; + element.value = 'INVALID_URLENCODED_PARAM_TYPE'; } } - - param = new sdk.QueryParam({ - key: key, - value: value - }); - param.description = description; - paramArray.push(param); - } + }); + paramArray.push(...params); }); updateOptions = { mode: rDataMode, @@ -1865,7 +1875,7 @@ module.exports = { return null; } _.forOwn(response.headers, (value, key) => { - if (key !== 'Content-Type') { + if (_.toLower(key) !== 'content-type') { if (value.$ref) { // the convert to PmHeader function handles the // schema-faking @@ -2253,12 +2263,12 @@ module.exports = { _.forEach(reqParams.query, (queryParam) => { this.convertToPmQueryParameters(queryParam, REQUEST_TYPE.ROOT, components, options, schemaCache) .forEach((pmParam) => { + // Modify the request query params for the test suites pmParam = this.overwriteRequestQueryParamByTests(pmParam, operation, operationItem, options); item.request.url.addQueryParams(pmParam); }); }); item.request.url.query.members.forEach((query) => { - query.description = _.get(query, 'description.content', ''); // Collection v2.1 schema allows query param value to be string/null if (typeof query.value !== 'string') { try { @@ -2274,6 +2284,7 @@ module.exports = { } }); item.request.url.variables.clear(); + // Modify the request paths variables for the test suites reqParams.path = this.overwriteRequestPathByTests(reqParams.path, operation, operationItem, options); item.request.url.variables.assimilate(this.convertPathVariables('param', pathVarArray, reqParams.path, components, options, schemaCache)); @@ -2291,6 +2302,7 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { + // Modify the request header for the test suites header = this.overwriteRequestHeaderByTests(header, operation, operationItem, options); item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); @@ -4203,7 +4215,8 @@ module.exports = { return false; } h.name = hName; - return h.required; + // exclude non-required and implicit header from further validation + return h.required && !_.includes(IMPLICIT_HEADERS, _.toLower(hName)); }), (header) => { if (!_.find(resHeaders, (param) => { return param.key === header.name; })) { @@ -4275,6 +4288,160 @@ module.exports = { ); }, 0); } + else if (requestBody && requestBody.mode === 'urlencoded') { + let urlencodedBodySchema = _.get(schemaPath, ['requestBody', 'content', URLENCODED, 'schema']), + resolvedSchemaParams = [], + pathPrefix = `${schemaPathPrefix}.requestBody.content[${URLENCODED}].schema`; + + urlencodedBodySchema = deref.resolveRefs(urlencodedBodySchema, PARAMETER_SOURCE.REQUEST, components, + schemaCache.schemaResolutionCache, PROCESSING_TYPE.VALIDATION, 'example', 0, {}, options.stackLimit); + + // resolve each property as separate param similar to query parmas + _.forEach(_.get(urlencodedBodySchema, 'properties'), (propSchema, propName) => { + let resolvedProp = { + name: propName, + schema: propSchema, + in: 'query' // serialization follows same behaviour as query params + }, + encodingValue = _.get(schemaPath, ['requestBody', 'content', URLENCODED, 'encoding', propName]), + pSerialisationInfo, + isPropSeparable; + + if (_.isObject(encodingValue)) { + _.has(encodingValue, 'style') && (resolvedProp.style = encodingValue.style); + _.has(encodingValue, 'explode') && (resolvedProp.explode = encodingValue.explode); + } + + if (_.includes(_.get(urlencodedBodySchema, 'required'), propName)) { + resolvedProp.required = true; + } + + pSerialisationInfo = this.getParamSerialisationInfo(resolvedProp, PARAMETER_SOURCE.REQUEST, + components, schemaCache); + isPropSeparable = _.includes(['form', 'deepObject'], pSerialisationInfo.style); + + if (isPropSeparable && propSchema.type === 'array' && pSerialisationInfo.explode) { + // add schema of items and instead array + resolvedSchemaParams.push(_.assign({}, resolvedProp, { + schema: _.get(propSchema, 'items'), + isResolvedParam: true + })); + } + else if (isPropSeparable && propSchema.type === 'object' && pSerialisationInfo.explode) { + // add schema of all properties instead entire object + _.forEach(_.get(propSchema, 'properties', {}), (value, key) => { + resolvedSchemaParams.push({ + name: key, + schema: value, + isResolvedParam: true + }); + }); + } + else { + resolvedSchemaParams.push(resolvedProp); + } + }); + + return async.map(requestBody.urlencoded, (uParam, cb) => { + let mismatches = [], + index = _.findIndex(requestBody.urlencoded, uParam), + resolvedParamValue = uParam.value; + + const schemaParam = _.find(resolvedSchemaParams, (param) => { return param.name === uParam.key; }); + + if (!schemaParam) { + // no schema param found + if (options.showMissingInSchemaErrors) { + mismatches.push({ + property: mismatchProperty, + transactionJsonPath: transactionPathPrefix + `.urlencoded[${index}]`, + schemaJsonPath: null, + reasonCode: 'MISSING_IN_SCHEMA', + reason: `The Url Encoded body param "${uParam.key}" was not found in the schema` + }); + } + return cb(null, mismatches); + } + + if (!schemaParam.isResolvedParam) { + resolvedParamValue = this.deserialiseParamValue(schemaParam, uParam.value, PARAMETER_SOURCE.REQUEST, + components, schemaCache); + } + // store value of transaction to use in mismatch object + schemaParam.actualValue = uParam.value; + + // param found in spec. check param's schema + setTimeout(() => { + if (!schemaParam.schema) { + // no errors to show if there's no schema present in the spec + return cb(null, []); + } + this.checkValueAgainstSchema(mismatchProperty, + transactionPathPrefix + `.urlencoded[${index}].value`, + uParam.key, + resolvedParamValue, + pathPrefix + '.properties[' + schemaParam.name + ']', + schemaParam.schema, + PARAMETER_SOURCE.REQUEST, + components, options, schemaCache, cb + ); + }, 0); + }, (err, res) => { + let mismatches = [], + mismatchObj, + // fetches property name from schem path + getPropNameFromSchemPath = (schemaPath) => { + let regex = /\.properties\[(.+)\]/gm; + return _.last(regex.exec(schemaPath)); + }; + + // update actual value and suggested value from JSON to serialized strings + _.forEach(_.flatten(res), (mismatchObj) => { + if (!_.isEmpty(mismatchObj)) { + let propertyName = getPropNameFromSchemPath(mismatchObj.schemaJsonPath), + schemaParam = _.find(resolvedSchemaParams, (param) => { return param.name === propertyName; }), + serializedParamValue; + + if (schemaParam) { + // serialize param value (to be used in suggested value) + serializedParamValue = _.get(this.convertParamsWithStyle(schemaParam, _.get(mismatchObj, + 'suggestedFix.suggestedValue'), PARAMETER_SOURCE.REQUEST, components, schemaCache, options), + '[0].value'); + _.set(mismatchObj, 'suggestedFix.actualValue', schemaParam.actualValue); + _.set(mismatchObj, 'suggestedFix.suggestedValue', serializedParamValue); + } + } + }); + + _.each(resolvedSchemaParams, (uParam) => { + // report mismatches only for reuired properties + if (!_.find(requestBody.urlencoded, (param) => { return param.key === uParam.name; }) && uParam.required) { + mismatchObj = { + property: mismatchProperty, + transactionJsonPath: transactionPathPrefix + '.urlencoded', + schemaJsonPath: pathPrefix + '.properties[' + uParam.name + ']', + reasonCode: 'MISSING_IN_REQUEST', + reason: `The Url Encoded body param "${uParam.name}" was not found in the transaction` + }; + + if (options.suggestAvailableFixes) { + mismatchObj.suggestedFix = { + key: uParam.name, + actualValue: null, + suggestedValue: { + key: uParam.name, + value: safeSchemaFaker(uParam.schema || {}, 'example', PROCESSING_TYPE.VALIDATION, + PARAMETER_SOURCE.REQUEST, components, SCHEMA_FORMATS.DEFAULT, options.indentCharacter, schemaCache, + options.stackLimit) + } + }; + } + mismatches.push(mismatchObj); + } + }); + return callback(null, _.concat(_.flatten(res), mismatches)); + }); + } else { return callback(null, []); } diff --git a/lib/schemapack.js b/lib/schemapack.js index 557c52bf1..6261b3f0a 100644 --- a/lib/schemapack.js +++ b/lib/schemapack.js @@ -1,7 +1,7 @@ 'use strict'; // This is the default collection name if one can't be inferred from the OpenAPI spec -const COLLECTION_NAME = 'Converted from OpenAPI', +const COLLECTION_NAME = 'Imported from OpenAPI 3.0', Ajv = require('ajv'), async = require('async'), sdk = require('postman-collection'), @@ -16,7 +16,8 @@ const COLLECTION_NAME = 'Converted from OpenAPI', utils = require('./utils.js'), _ = require('lodash'), fs = require('fs'), - path = require('path-browserify'), + // use path based on platform it's running on (web or node) + path = typeof process === 'object' ? require('path') : require('path-browserify'), // options for oas-resolver // This provides the base class for @@ -263,7 +264,7 @@ class SchemaPack { // All generated folders and requests will go inside this generatedStore.collection = new sdk.Collection({ info: { - name: _.get(openapi, 'info.title', COLLECTION_NAME) + name: _.isEmpty(_.get(openapi, 'info.title')) ? COLLECTION_NAME : _.get(openapi, 'info.title') } }); diff --git a/package-lock.json b/package-lock.json index 75c682b4c..c33aefff4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,6 +1,6 @@ { "name": "openapi-to-postmanv2", - "version": "2.1.0", + "version": "2.3.0", "lockfileVersion": 1, "requires": true, "dependencies": { @@ -175,11 +175,6 @@ "to-fast-properties": "^2.0.0" } }, - "@types/color-name": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/@types/color-name/-/color-name-1.1.1.tgz", - "integrity": "sha512-rr+OQyAjxze7GgWrSaJwydHStIhHq2lvY3BOC2Mj7KnzI7XK0Uw1TOOdI9lDoajEbSWLiYgoo4f1R51erQfhPQ==" - }, "acorn": { "version": "6.4.1", "resolved": "https://registry.npmjs.org/acorn/-/acorn-6.4.1.tgz", @@ -1838,16 +1833,23 @@ } }, "oas-resolver-browser": { - "version": "2.3.3", - "resolved": "https://registry.npmjs.org/oas-resolver-browser/-/oas-resolver-browser-2.3.3.tgz", - "integrity": "sha512-KvggQ6xU7WlUWRYZKEktR90zJtNCHi1wbTAZuUX6oSfmBSdZo/b26rzfg3w2AdPVwQPRXMga6tqLW3OhbUF0Qg==", + "version": "2.5.1", + "resolved": "https://registry.npmjs.org/oas-resolver-browser/-/oas-resolver-browser-2.5.1.tgz", + "integrity": "sha512-F/UpzUvloe+8F0f9G7/WTcERAGma4TTpLGMddHRJFmocpnfx1jihCwidezZvSV3fZNi0ZaB2bc26xMIK6ltPeg==", "requires": { "node-fetch-h2": "^2.3.0", "oas-kit-common": "^1.0.8", "path-browserify": "^1.0.1", - "reftools": "^1.1.1", - "yaml": "^1.8.3", + "reftools": "^1.1.6", + "yaml": "^1.10.0", "yargs": "^15.3.1" + }, + "dependencies": { + "yaml": { + "version": "1.10.0", + "resolved": "https://registry.npmjs.org/yaml/-/yaml-1.10.0.tgz", + "integrity": "sha512-yr2icI4glYaNG+KWONODapy2/jDdMSDnrONSjblABjD9B4Z5LgiircSt8m8sRZFNi08kG9Sm0uSHtEmP3zaEGg==" + } } }, "once": { @@ -2232,9 +2234,9 @@ } }, "reftools": { - "version": "1.1.1", - "resolved": "https://registry.npmjs.org/reftools/-/reftools-1.1.1.tgz", - "integrity": "sha512-7ySkzK7YpUeJP16rzJqEXTZ7IrAq/AL/p+wWejD9wdKQOe+mYYVAOB3w5ZTs2eoHfmAidwr/6PcC+q+LzPF/DQ==" + "version": "1.1.8", + "resolved": "https://registry.npmjs.org/reftools/-/reftools-1.1.8.tgz", + "integrity": "sha512-Yvz9NH8uFHzD/AXX82Li1GdAP6FzDBxEZw+njerNBBQv/XHihqsWAjNfXtaq4QD2l4TEZVnp4UbktdYSegAM3g==" }, "regenerator-runtime": { "version": "0.13.5", @@ -2759,11 +2761,10 @@ "integrity": "sha512-bY6fj56OUQ0hU1KjFNDQuJFezqKdrAyFdIevADiqrWHwSlbmBNMHp5ak2f40Pm8JTFyM2mqxkG6ngkHO11f/lg==" }, "ansi-styles": { - "version": "4.2.1", - "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.2.1.tgz", - "integrity": "sha512-9VGjrMsG1vePxcSweQsN20KY/c4zN0h9fLjqAbwbPfahM3t+NL+M9HC8xeXG2I8pX5NoamTGNuomEUFI7fcUjA==", + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", + "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==", "requires": { - "@types/color-name": "^1.1.1", "color-convert": "^2.0.1" } }, @@ -2841,9 +2842,9 @@ } }, "yargs": { - "version": "15.3.1", - "resolved": "https://registry.npmjs.org/yargs/-/yargs-15.3.1.tgz", - "integrity": "sha512-92O1HWEjw27sBfgmXiixJWT5hRBp2eobqXicLtPBIDBhYB+1HpwZlXmbW2luivBJHBzki+7VyCLRtAkScbTBQA==", + "version": "15.4.1", + "resolved": "https://registry.npmjs.org/yargs/-/yargs-15.4.1.tgz", + "integrity": "sha512-aePbxDmcYW++PaqBsJ+HYUFwCdv4LVvdnhBy78E57PIor8/OVvhMrADFFEDh8DHDFRv/O9i3lPhsENjO7QX0+A==", "requires": { "cliui": "^6.0.0", "decamelize": "^1.2.0", @@ -2855,7 +2856,7 @@ "string-width": "^4.2.0", "which-module": "^2.0.0", "y18n": "^4.0.0", - "yargs-parser": "^18.1.1" + "yargs-parser": "^18.1.2" }, "dependencies": { "ansi-regex": { diff --git a/package.json b/package.json index cd3742224..ab2a5b60d 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "openapi-to-postmanv2", - "version": "2.1.0", + "version": "2.3.0", "description": "Convert a given OpenAPI specification to Postman Collection v2.0", "homepage": "https://github.com/postmanlabs/openapi-to-postman", "bugs": "https://github.com/postmanlabs/openapi-to-postman/issues", @@ -122,7 +122,7 @@ "js-yaml": "3.13.1", "json-schema-merge-allof": "0.7.0", "lodash": "4.17.20", - "oas-resolver-browser": "2.3.3", + "oas-resolver-browser": "2.5.1", "path-browserify": "1.0.1", "postman-collection": "3.6.6", "yaml": "1.8.3" diff --git a/test/data/valid_openapi/required_in_parameters.json b/test/data/valid_openapi/required_in_parameters.json index 5f85b7d42..74f3214c4 100644 --- a/test/data/valid_openapi/required_in_parameters.json +++ b/test/data/valid_openapi/required_in_parameters.json @@ -91,15 +91,11 @@ "formParam1": { "description": "Description of formParam1", "required": true, - "schema": { - "type": "string" - } + "type": "string" }, "formParam2": { "description": "Description of formParam2", - "schema": { - "type": "string" - } + "type": "string" } } } @@ -120,15 +116,11 @@ "urlencodedParam1": { "description": "Description of urlencodedParam1", "required": true, - "schema": { - "type": "string" - } + "type": "string" }, "urlencodedParam2": { "description": "Description of urlencodedParam2", - "schema": { - "type": "string" - } + "type": "string" } } } diff --git a/test/data/validationData/implicitHeaderSpec.yaml b/test/data/validationData/implicitHeaderSpec.yaml index 028daafdf..0b4a0aadd 100644 --- a/test/data/validationData/implicitHeaderSpec.yaml +++ b/test/data/validationData/implicitHeaderSpec.yaml @@ -35,6 +35,12 @@ paths: description: Null response default: description: unexpected error + headers: + content-type: + description: content-type of response body + required: true + schema: + type: string content: application/json: schema: @@ -68,4 +74,4 @@ components: type: integer format: int32 message: - type: string \ No newline at end of file + type: string diff --git a/test/data/validationData/urlencodedBodyCollection.json b/test/data/validationData/urlencodedBodyCollection.json new file mode 100644 index 000000000..e541dff24 --- /dev/null +++ b/test/data/validationData/urlencodedBodyCollection.json @@ -0,0 +1,161 @@ +{ + "item": [ + { + "id": "f5983708-1a61-43a4-919b-ca40a34fe2a3", + "name": "pet", + "description": { + "content": "", + "type": "text/plain" + }, + "item": [ + { + "id": "9c4d4bf3-c8f6-47f5-83d6-6e25f89c1314", + "name": "Updates a pet in the store with form data", + "request": { + "name": "Updates a pet in the store with form data", + "description": {}, + "url": { + "path": [ + "pets", + ":petId" + ], + "host": [ + "{{baseUrl}}" + ], + "query": [], + "variable": [ + { + "description": "(Required) ID of pet that needs to be updated", + "type": "any", + "value": "elit nulla", + "key": "petId" + } + ] + }, + "header": [ + { + "key": "Content-Type", + "value": "application/x-www-form-urlencoded" + } + ], + "method": "POST", + "auth": null, + "body": { + "mode": "urlencoded", + "urlencoded": [ + { + "key": "prop1", + "value": "hello" + }, + { + "key": "prop2", + "value": "false" + }, + { + "key": "propObjectNonExplodable", + "value": "prop3,hello,prop4,true" + }, + { + "key": "propArray", + "value": "str1" + }, + { + "key": "propArray", + "value": "999" + }, + { + "key": "propSimple", + "value": "123" + } + ] + } + }, + "response": [ + { + "id": "613fc0d2-9836-48a6-9518-8f239ba9427f", + "name": "Pet updated.", + "originalRequest": { + "url": { + "path": [ + "pets", + ":petId" + ], + "host": [ + "{{baseUrl}}" + ], + "query": [], + "variable": [ + { + "type": "any", + "key": "petId" + } + ] + }, + "method": "POST", + "body": { + "mode": "urlencoded", + "urlencoded": [ + { + "key": "prop1", + "value": "hello" + }, + { + "key": "prop2", + "value": "world" + }, + { + "key": "propObjectNonExplodable", + "value": "prop1,hello,prop2,world" + }, + { + "key": "propArray", + "value": "str1" + }, + { + "key": "propArray", + "value": "str2" + }, + { + "key": "propSimple", + "value": "123" + } + ] + } + }, + "status": "OK", + "code": 200, + "header": [ + { + "key": "Content-Type", + "value": "text/plain" + } + ], + "body": "", + "cookie": [], + "_postman_previewlanguage": "text" + } + ], + "event": [] + } + ], + "event": [] + } + ], + "event": [], + "variable": [ + { + "id": "baseUrl", + "type": "string", + "value": "http://petstore.swagger.io/v1" + } + ], + "info": { + "_postman_id": "acd16ff0-eda5-48b4-b0de-4d35138da21d", + "name": "Swagger Petstore", + "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json", + "description": { + "content": "", + "type": "text/plain" + } + } +} diff --git a/test/data/validationData/urlencodedBodySpec.yaml b/test/data/validationData/urlencodedBodySpec.yaml new file mode 100644 index 000000000..da507719c --- /dev/null +++ b/test/data/validationData/urlencodedBodySpec.yaml @@ -0,0 +1,68 @@ +openapi: "3.0.0" +info: + version: 1.0.0 + title: Swagger Petstore + license: + name: MIT +servers: + - url: http://petstore.swagger.io/v1 +paths: + /pets/{petId}: + post: + tags: + - pet + summary: Updates a pet in the store with form data + operationId: updatePetWithForm + parameters: + - name: petId + in: path + description: ID of pet that needs to be updated + required: true + schema: + type: string + requestBody: + content: + 'application/x-www-form-urlencoded': + schema: + properties: + propObjectExplodable: + type: object + properties: + prop1: + type: string + example: hello + prop2: + type: string + example: world + propObjectNonExplodable: + type: object + properties: + prop3: + type: string + example: hello + prop4: + type: string + example: world + propArray: + type: array + items: + type: string + example: exampleString + example: + - str1 + - str2 + propSimple: + type: integer + example: 123 + required: + - status + encoding: + propObjectExplodable: + style: form + explode: true + propObjectNonExplodable: + style: form + explode: false + responses: + '200': + description: Pet updated. diff --git a/test/system/structure.test.js b/test/system/structure.test.js index 2d39824c7..f798d08d1 100644 --- a/test/system/structure.test.js +++ b/test/system/structure.test.js @@ -140,6 +140,12 @@ const optionIds = [ default: false, description: 'Whether requests should be strictly matched with schema operations. Setting to true will not ' + 'include any matches where the URL path segments don\'t match exactly.' + }, + disableOptionalParameters: { + name: 'Disable optional parameters', + type: 'boolean', + default: false, + description: 'Whether to set optional parameters as disabled' } }; diff --git a/test/unit/validator.test.js b/test/unit/validator.test.js index 7afb7616b..535b1e8a3 100644 --- a/test/unit/validator.test.js +++ b/test/unit/validator.test.js @@ -415,6 +415,10 @@ describe('VALIDATE FUNCTION TESTS ', function () { the mismatch for header-1 should contain correct index as in request. */ expect(_.endsWith(resultObj.mismatches[0].transactionJsonPath, '[2].value')).to.eql(true); + _.forEach(resultObj.responses, (response) => { + expect(response.matched).to.be.true; + expect(response.mismatches).to.have.lengthOf(0); + }); done(); }); }); @@ -627,4 +631,42 @@ describe('VALIDATE FUNCTION TESTS ', function () { done(); }); }); + + it('Should be able to validate schema with request body of content type "application/x-www-form-urlencoded" ' + + 'against transaction with valid UrlEncoded body correctly', function (done) { + let urlencodedBodySpec = fs.readFileSync(path.join(__dirname, VALIDATION_DATA_FOLDER_PATH + + '/urlencodedBodySpec.yaml'), 'utf-8'), + urlencodedBodyCollection = fs.readFileSync(path.join(__dirname, VALIDATION_DATA_FOLDER_PATH + + '/urlencodedBodyCollection.json'), 'utf-8'), + resultObj, + historyRequest = [], + schemaPack = new Converter.SchemaPack({ type: 'string', data: urlencodedBodySpec }, + { suggestAvailableFixes: true }); + + getAllTransactions(JSON.parse(urlencodedBodyCollection), historyRequest); + + schemaPack.validateTransaction(historyRequest, (err, result) => { + expect(err).to.be.null; + expect(result).to.be.an('object'); + resultObj = result.requests[historyRequest[0].id].endpoints[0]; + expect(resultObj.mismatches).to.have.lengthOf(3); + + // for explodable property of type object named "propObjectExplodable", + // second property named "prop2" is incorrect, while property "prop1" is correct + expect(resultObj.mismatches[0].transactionJsonPath).to.eql('$.request.body.urlencoded[1].value'); + expect(resultObj.mismatches[0].suggestedFix.actualValue).to.eql('false'); + expect(resultObj.mismatches[0].suggestedFix.suggestedValue).to.eql('world'); + + // for non explodable property of type object, entire property with updated value should be suggested + expect(resultObj.mismatches[1].transactionJsonPath).to.eql('$.request.body.urlencoded[2].value'); + expect(resultObj.mismatches[1].suggestedFix.actualValue).to.eql('prop3,hello,prop4,true'); + expect(resultObj.mismatches[1].suggestedFix.suggestedValue).to.eql('prop3,hello,prop4,world'); + + // for type array property named "propArray" second element is incorrect + expect(resultObj.mismatches[2].transactionJsonPath).to.eql('$.request.body.urlencoded[4].value'); + expect(resultObj.mismatches[2].suggestedFix.actualValue).to.eql('999'); + expect(resultObj.mismatches[2].suggestedFix.suggestedValue).to.eql('exampleString'); + done(); + }); + }); }); From 1e276320db16c356dde208fd896592254609d1bc Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 28 Feb 2021 17:53:57 +0100 Subject: [PATCH 36/52] Added to automatic set the response.id as an environment variable for chaining option Added the option to assign pm.environment variables from the response object --- TESTGENERATION.md | 97 ++++++++++++++++++++++++++++++++++++++++++++++ lib/schemaUtils.js | 61 +++++++++++++++++++++++++---- 2 files changed, 151 insertions(+), 7 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 7e50d2e4c..6845983f9 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -77,6 +77,8 @@ The JSON test suite format consists out of 3 parts: will be generated for **all** operations). - **extendTests**: which refers the custom additions of manual created postman tests. ( see [Postman test suite extendTests](#postman-test-suite-extendtests)) +- **assignPmVariables**: which refers to specific Postman environment variables for easier automation. ( + see [Postman test suite assignPmVariables](#postman-test-suite-assignpmvariables)) - **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. ( see [Postman test suite overwriteRequests](#postman-test-suite-overwriterequests)) @@ -107,6 +109,81 @@ in `tests` array, will be added to the postman test scripts. - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. Default: false +## Postman test suite assignPmVariables + +To facilitate automation, we provide the option set "pm.environment" variables with values from the response. +The assigning of the pm.environment are mapped based on the OpenApi operationId. + +REMARK: By default the test suite will create pm.environment variable for the ID property in the response object, if +ID is present in the reponse. + +Anything added in `assignPmVariables` array, will be used to generate specific pm.environment variables based on the +postman response body. + +Properties explained: + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman pm.environment variable + will be set. +- **environmentVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. + - **responseProp (string)** : The property for which the value will be taken in the response body and set as the + pm.environment value. + - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite + the default generated variable name + +Example: + +```JSON +{ + "assignPmVariables": [ + { + "openApiOperationId": "post-accounts", + "environmentVariables": [ + { + "responseProp": "clientGuid", + "name": "client-ID" + } + ] + }, + { + "openApiOperationId": "get-accounts", + "environmentVariables": [ + { + "responseProp": "value[0].servers[0]", + "name": "server-address" + } + ] + } + ] +} +``` + +This will generate the following: +- pm.environment for "post-accounts.id" - use {{post-accounts.id}} as variable for "reponse.id +- pm.environment for "get-accounts" - use {{server-address}} as variable for "reponse.value[0].servers[0]" + +This information on the assignment of the pm.environment will be published in the Postman Test script and during +the conversion via the CLI + +This results in the following functions on the Postman Test pane: + +```javascript + +// Set response object as internal variable +let jsonData = pm.response.json(); + +// pm.environment - Set post-accounts.id as environment variable +if (jsonData.id) { + pm.environment.set("post-accounts.id",jsonData.id); + console.log("pm.environment - use {{post-accounts.id}} as variable for value", jsonData.id); +}; + +// pm.environment - Set post-accounts.servers[0] as environment variable +if (jsonData.value[0].servers[0]) { + pm.environment.set("server-address",jsonData.value[0].servers[0]); + console.log("pm.environment - use {{server-address}} as variable for value", jsonData.value[0].servers[0]); +}; +``` + ## Postman test suite overwriteRequests To facilitate automation, you might want to modify property values with "randomized" or specific values. The overwrites @@ -190,6 +267,26 @@ OpenAPI to Postman Testsuite Configuration: ] } ], + "assignPmVariables": [ + { + "openApiOperationId": "post-accounts", + "environmentVariables": [ + { + "responseProp": "clientGuid", + "name": "client-ID" + } + ] + }, + { + "openApiOperationId": "get-accounts", + "environmentVariables": [ + { + "responseProp": "value[0].servers[0]", + "name": "server-address" + } + ] + } + ], "overwriteRequests": [ { "openApiOperationId": "get-accounts", diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 661cba982..99bf2cbd1 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2531,6 +2531,8 @@ module.exports = { } if (contentType === APP_JSON) { + let resolvedSchema, assignPmVariablesCounter = 0; + // Add JSON body check if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { requestsTests.push( @@ -2550,7 +2552,7 @@ module.exports = { try { // When processing a reference, schema.type could also be undefined - let resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, schemaCache, PROCESSING_TYPE.VALIDATION); requestsTests.push( @@ -2569,15 +2571,60 @@ module.exports = { } } - // Set the ID of the response for chaining options - if (operationItem.method.toUpperCase() === 'POST') { + // Automatic set the response.id as an environment variable for chaining option + if (operationItem.method.toUpperCase() === 'POST' && resolvedSchema !== undefined && + resolvedSchema.properties && resolvedSchema.properties.id) { + // Only set the jsonData once + if (assignPmVariablesCounter === 0) { + requestsTests.push('// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } requestsTests.push( - '// Chaining - Set ' + operation.operationId + ' ID as environment variable \n' + - 'let jsonData = pm.response.json();\n' + + '// pm.environment - Set ' + operation.operationId + '.id as environment variable \n' + 'if (jsonData.id) {\n' + ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + - ' console.log("Chaining - use {{' + operation.operationId + '.id}} as variable", jsonData.id);\n' + + ' console.log("pm.environment - use {{' + operation.operationId + '.id}} as variable for value", jsonData.id);\n' + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + + operation.operationId + '.id}} as variable for "reponse.id'); + assignPmVariablesCounter++; + } + + // Assign defined PmVariables + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables; + // Get the environmentVariables settings for the current operationId + let assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + assignEnvironmentVariables.forEach((environmentVariable) => { + if (environmentVariable.responseProp) { + // Set variable name + let varName = operation.operationId + '.' + environmentVariable.responseProp; + if (environmentVariable.name) { + varName = environmentVariable.name; + } + // Only set the jsonData once + if (assignPmVariablesCounter === 0) { + requestsTests.push('// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + requestsTests.push( + '// pm.environment - Set ' + varName + ' as environment variable \n' + + 'if (jsonData.' + environmentVariable.responseProp + ') {\n' + + ' pm.environment.set("' + varName + '",jsonData.' + environmentVariable.responseProp + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + 'jsonData.' + environmentVariable.responseProp + ');\n' + + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + 'as variable for "reponse.' + environmentVariable.responseProp + '"'); + assignPmVariablesCounter++; + } + }); + } } } @@ -2713,7 +2760,7 @@ module.exports = { /** * function to overwrite a request path variables with values defined by the postman testsuite - * @param {*} requestPathVariables request Params Path Variables object + * @param {*} requestPathVariables request Path Variables object * @param {*} operation openapi operation object * @param {*} operationItem - The operation item to get relevant information for the test generation * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. From b190b6109f651a8ba6a40caf714cd827603e3be4 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 9 Mar 2021 00:04:27 +0100 Subject: [PATCH 37/52] Bug fix for path variable conversions when using a dynamic variable to replace an integer example --- lib/schemaUtils.js | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 99bf2cbd1..5b31f3000 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2768,7 +2768,7 @@ module.exports = { */ overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { options = _.merge({}, defaultOptions, options); - let testRequestPathVariables = Object.assign([], requestPathVariables), // clone requestPathVariables + let testRequestPathVariables = requestPathVariables.slice(), // clone requestPathVariables testSuite = {}, overwriteValues = [], testSuiteOverwriteRequest = {}, @@ -2804,6 +2804,7 @@ module.exports = { if (overwriteValue.overwrite === false) { newValue = orgValue + newValue; } + pathVar.schema.type = "string"; // Set schema as type string dynamic variable pathVar.schema.example = newValue; } }); From 8625e7e95e12fffefe0d95b716267d42dc78ff8a Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 9 Mar 2021 10:26:32 +0100 Subject: [PATCH 38/52] Lint fix --- lib/schemaUtils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 5b31f3000..361d4cc2d 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2804,7 +2804,7 @@ module.exports = { if (overwriteValue.overwrite === false) { newValue = orgValue + newValue; } - pathVar.schema.type = "string"; // Set schema as type string dynamic variable + pathVar.schema.type = 'string'; // Set schema as type string dynamic variable pathVar.schema.example = newValue; } }); From af070d34ae4744b11b31207cd3754aee255c5470 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Wed, 10 Mar 2021 19:52:59 +0100 Subject: [PATCH 39/52] Added option to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties which deletes nullable and adds "null" to type array if nullable is true --- lib/schemaUtils.js | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 361d4cc2d..b5bcc7087 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2555,9 +2555,12 @@ module.exports = { resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, schemaCache, PROCESSING_TYPE.VALIDATION); + // deletes nullable and adds "null" to type array if nullable is true + let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); + requestsTests.push( '// Response Validation\n' + - 'const schema = ' + JSON.stringify(resolvedSchema) + '\n' + + 'const schema = ' + JSON.stringify(jsonSchema) + '\n' + '\n' + '// Test whether the response matches the schema\n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + @@ -2567,6 +2570,7 @@ module.exports = { ); } catch (e) { + console.warn('JSON schema check failed', e); console.warn('invalid schemaValidation for ', content); } } @@ -2919,6 +2923,38 @@ module.exports = { return testRequestHeader; }, + /** + * function to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + * @param {*} oaSchema openAPI schema + * @returns {*} Modified openAPI schema object that is compatible with JSON schema validation + */ + convertUnsupportedJsonSchemaProperties: function(oaSchema) { + let jsonSchema = JSON.parse(JSON.stringify(oaSchema)); // Deep copy of the schema object + + // Convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + // let jsonSchemaNotSupported = ['nullable', 'discriminator', 'readOnly', 'writeOnly', 'xml', + // 'externalDocs', 'example', 'deprecated']; + + // Recurse through OpenAPI Schema + const traverse = (obj) => { + for (let k in obj) { + if (obj.hasOwnProperty(k) && obj[k] && typeof obj[k] === 'object') { + if (obj[k].nullable && obj[k].nullable === true) { + // deletes nullable and adds "null" to type array if nullable is true + let jsonTypes = []; + jsonTypes.push(obj[k].type); + jsonTypes.push('null'); + obj[k].type = jsonTypes; + delete obj[k].nullable; + } + traverse(obj[k]); + } + } + }; + traverse(jsonSchema); + return jsonSchema; + }, + /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From a8363df8e48bff7b22875b412c16338341800230 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Thu, 18 Mar 2021 11:58:58 +0100 Subject: [PATCH 40/52] Added the option to assign pm.environment variables from the response headers --- lib/schemaUtils.js | 67 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 58 insertions(+), 9 deletions(-) diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index b5bcc7087..851a31c51 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2531,7 +2531,8 @@ module.exports = { } if (contentType === APP_JSON) { - let resolvedSchema, assignPmVariablesCounter = 0; + let resolvedSchema, + assignPmVariablesCounter = 0; // Add JSON body check if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { @@ -2587,14 +2588,15 @@ module.exports = { '// pm.environment - Set ' + operation.operationId + '.id as environment variable \n' + 'if (jsonData.id) {\n' + ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + - ' console.log("pm.environment - use {{' + operation.operationId + '.id}} as variable for value", jsonData.id);\n' + + ' console.log("pm.environment - use {{' + operation.operationId + '.id}} ' + + 'as variable for value", jsonData.id);\n' + '};\n'); console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + operation.operationId + '.id}} as variable for "reponse.id'); assignPmVariablesCounter++; } - // Assign defined PmVariables + // Assign defined PmVariables for JSON response if (testSuite.assignPmVariables) { let testSuiteAssignPmVariables = testSuite.assignPmVariables; // Get the environmentVariables settings for the current operationId @@ -2604,10 +2606,14 @@ module.exports = { if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + assignEnvironmentVariables.forEach((environmentVariable) => { - if (environmentVariable.responseProp) { + // Set response body property as variable + if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { + let responseProp = (environmentVariable.responseProp) ? + environmentVariable.responseProp : environmentVariable.responseBodyProp; // Set variable name - let varName = operation.operationId + '.' + environmentVariable.responseProp; + let varName = operation.operationId + '.' + responseProp; if (environmentVariable.name) { varName = environmentVariable.name; } @@ -2618,13 +2624,13 @@ module.exports = { } requestsTests.push( '// pm.environment - Set ' + varName + ' as environment variable \n' + - 'if (jsonData.' + environmentVariable.responseProp + ') {\n' + - ' pm.environment.set("' + varName + '",jsonData.' + environmentVariable.responseProp + ');\n' + + 'if (jsonData.' + responseProp + ') {\n' + + ' pm.environment.set("' + varName + '",jsonData.' + responseProp + ');\n' + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + - 'jsonData.' + environmentVariable.responseProp + ');\n' + + 'jsonData.' + responseProp + ');\n' + '};\n'); console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + - 'as variable for "reponse.' + environmentVariable.responseProp + '"'); + 'as variable for "reponse.' + responseProp + '"'); assignPmVariablesCounter++; } }); @@ -2653,6 +2659,49 @@ module.exports = { } }); + + // Assign defined PmVariables for response headers + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the current operationId + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set response header property as variable + if (environmentVariable.responseHeaderProp) { + let headerProp = environmentVariable.responseHeaderProp; + // Set variable name + let varName = operation.operationId + '.' + headerProp; + if (environmentVariable.name) { + varName = environmentVariable.name; + } + + // Safe variable name + let safeVarName = varName.replace(/-/g, '') + .replace(/_/g, '').replace(/ /g, '') + .replace(/\./g, ''); + + requestsTests.push( + '// pm.environment - Set ' + varName + ' as environment variable \n' + + 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + + 'if (' + safeVarName + ' !== undefined) {\n' + + ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + '' + safeVarName + ');\n' + + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + 'as variable for "header.' + headerProp + '"'); + } + + }); + } + } + }); // Add test extensions that are defined in the test suite file From a879d1325781a5f2a1373164e84c27d6b9f37b95 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 11 Apr 2021 20:43:14 +0200 Subject: [PATCH 41/52] Moved all Postman Testsuite functions in a separate file for easier maintenance --- lib/pmTestSuite.js | 605 ++++++++++++++++++++++++++++++++++++++++++++ lib/schemaUtils.js | 617 +-------------------------------------------- 2 files changed, 616 insertions(+), 606 deletions(-) create mode 100644 lib/pmTestSuite.js diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js new file mode 100644 index 000000000..e20abe307 --- /dev/null +++ b/lib/pmTestSuite.js @@ -0,0 +1,605 @@ +const _ = require('lodash'), + deref = require('./deref.js'), + defaultOptions = require('../lib/options.js').getOptions('use'), + APP_JSON = 'application/json', + // Specifies types of processing Refs + PROCESSING_TYPE = { + VALIDATION: 'VALIDATION', + CONVERSION: 'CONVERSION' + }; + +module.exports = { + /** + * function to convert an openapi reponse object to object of postman tests + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} components - components defined in the OAS spec. These are used to + * resolve references while generating params. + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @param {object} schemaCache - object storing schemaFaker and schmeResolution caches + * @returns {*} array of all query params + */ + convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { + options = _.merge({}, defaultOptions, options); + let testEvent = {}, + testSuite = {}, + testSuiteSettings = {}, + testSuiteLimits = [], + testSuiteExtensions = [], + requestsTests = [], + swagResponse = {}; + + // Check for test suite flag, abort early + if (!options.testSuite) { + return testEvent; + } + + // Check test suite for later usage, potentially convert object + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + if (testSuite.generateTests) { + testSuiteSettings = options.testSuiteSettings.generateTests; + } + + // Limit the test generation to the following operations + if (testSuiteSettings.limitOperations) { + testSuiteLimits = testSuiteSettings.limitOperations; + } + + // Extend the generated test with additional operations + if (testSuite.extendTests) { + testSuiteExtensions = options.testSuiteSettings.extendTests; + + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId && + testExtension && testExtension.responseChecks) { + // Extend testSuiteSettings with testExtension settings + testSuiteSettings.responseChecks = _.merge( + {}, testSuiteSettings.responseChecks, testExtension.responseChecks + ); + } + }); + } + + } + + _.forOwn(operation.responses, (response, code) => { + // Skip the operations, if there are limits defined + if (operation.operationId && testSuiteLimits.length > 0 && + testSuiteLimits.indexOf(operation.operationId) === -1) { + return; // skip this response + } + + // Only support 2xx response checks + // TODO Investigate how to support other response codes like validation response 4xx or 5xx + if (!_.inRange(code, 200, 299)) { + return; // skip this response + } + + // Format response + swagResponse = response; + if (response.$ref) { + swagResponse = this.getRefObject(response.$ref, components, options); + } + + // Add status code check + // TODO Validate other request codes + // if (operationItem.method.toUpperCase() === 'GET') { + // requestsTests.push( + // '// Validate status code \n'+ + // 'pm.test("Status code should be '+code+'", function () {\n' + + // ' pm.response.to.have.status('+code+');\n' + + // '});\n' + // ) + // } + + // Add status success check + if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { + requestsTests.push( + '// Validate status 2xx \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Status code is 2xx", function () {\n' + + ' pm.response.to.be.success;\n' + + '});\n' + ); + } + + // Add response timer check + if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { + let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); + requestsTests.push( + '// Validate response time \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + + ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + + '});\n' + ); + } + + // Process the response content + _.forOwn(swagResponse.content, (content, contentType) => { + if (contentType) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { + requestsTests.push( + '// Validate content-type \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content-Type is ' + contentType + '", function () {\n' + + ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + + '});\n' + ); + } + + if (contentType === APP_JSON) { + let resolvedSchema, + assignPmVariablesCounter = 0; + + // Add JSON body check + if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { + requestsTests.push( + '// Response should have JSON Body\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response has JSON Body", function () {\n' + + ' pm.response.to.have.jsonBody();\n' + + '});\n'); + } + + // Add JSON schema check + if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && content.schema) { + let schemaContent = content.schema; + // if (content.schema.$ref) { + // schemaContent = this.getRefObject(content.schema.$ref, components, options); + // } + + try { + // When processing a reference, schema.type could also be undefined + resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, + schemaCache, PROCESSING_TYPE.VALIDATION); + + // deletes nullable and adds "null" to type array if nullable is true + let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); + + requestsTests.push( + '// Response Validation\n' + + 'const schema = ' + JSON.stringify(jsonSchema) + '\n' + + '\n' + + '// Test whether the response matches the schema\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Schema is valid", function() {\n' + + ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + + '});\n' + ); + } catch (e) { + console.warn('JSON schema check failed', e); + console.warn('invalid schemaValidation for ', content); + } + } + + // Automatic set the response.id as an environment variable for chaining option + if (operationItem.method.toUpperCase() === 'POST' && resolvedSchema !== undefined && + resolvedSchema.properties && resolvedSchema.properties.id) { + // Only set the jsonData once + if (assignPmVariablesCounter === 0) { + requestsTests.push('// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + requestsTests.push( + '// pm.environment - Set ' + operation.operationId + '.id as environment variable \n' + + 'if (jsonData.id) {\n' + + ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + + ' console.log("pm.environment - use {{' + operation.operationId + '.id}} ' + + 'as variable for value", jsonData.id);\n' + + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + + operation.operationId + '.id}} as variable for "reponse.id'); + assignPmVariablesCounter++; + } + + // Assign defined PmVariables for JSON response + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the current operationId + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set response body property as variable + if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { + let responseProp = (environmentVariable.responseProp) ? + environmentVariable.responseProp : environmentVariable.responseBodyProp; + // Set variable name + let varName = operation.operationId + '.' + responseProp; + if (environmentVariable.name) { + varName = environmentVariable.name; + } + // Only set the jsonData once + if (assignPmVariablesCounter === 0) { + requestsTests.push('// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + requestsTests.push( + '// pm.environment - Set ' + varName + ' as environment variable \n' + + 'if (jsonData.' + responseProp + ') {\n' + + ' pm.environment.set("' + varName + '",jsonData.' + responseProp + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + 'jsonData.' + responseProp + ');\n' + + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + 'as variable for "reponse.' + responseProp + '"'); + assignPmVariablesCounter++; + } + }); + } + } + + } + } + + }); + + // Process the response header + _.forOwn(swagResponse.headers, (header, headerKey) => { + + if (headerKey && header.name) { + // Add content-type check + if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { + requestsTests.push( + '// Validate header \n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Response header ' + header.name + ' is present", function () {\n' + + ' pm.response.to.have.header("' + header.name + '");\n' + + '});\n' + ); + } + } + + }); + + // Assign defined PmVariables for response headers + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the current operationId + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set response header property as variable + if (environmentVariable.responseHeaderProp) { + let headerProp = environmentVariable.responseHeaderProp; + // Set variable name + let varName = operation.operationId + '.' + headerProp; + if (environmentVariable.name) { + varName = environmentVariable.name; + } + + // Safe variable name + let safeVarName = varName.replace(/-/g, '') + .replace(/_/g, '').replace(/ /g, '') + .replace(/\./g, ''); + + requestsTests.push( + '// pm.environment - Set ' + varName + ' as environment variable \n' + + 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + + 'if (' + safeVarName + ' !== undefined) {\n' + + ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + '' + safeVarName + ');\n' + + '};\n'); + console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + 'as variable for "header.' + headerProp + '"'); + } + + }); + } + } + + }); + + // Add test extensions that are defined in the test suite file + if (testSuiteExtensions.length > 0) { + testSuiteExtensions.forEach((testExtension) => { + if (operation.operationId && testExtension.openApiOperationId && + operation.operationId === testExtension.openApiOperationId) { + + if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { + // Reset generated tests + requestsTests = []; + } + + // Add test extensions + if (testExtension.tests && testExtension.tests.length > 0) { + testExtension.tests.forEach((postmanTest) => { + try { + // Extend the generated tests, with the test extension scripts + requestsTests.push(postmanTest); + } catch (e) { + console.warn('invalid extendTests for ' + testExtension.openApiOperationI); + } + }); + } + } + }); + } + + // Add tests to postman item + if (requestsTests.length > 0) { + testEvent = { + listen: 'test', + script: requestsTests + }; + } + return testEvent; + }, + + /** + * function to overwrite a request body with values defined by the postman testsuite + * @param {*} requestBody object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestBodyByTests: function (requestBody, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestBody = Object.assign({}, requestBody), // clone requestBody before manipulation + bodyString, + find, + replace, + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is RAW requestBody, abort early + if (!options.testSuite && !requestBody.raw) { + return requestBody; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestBody) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestBody; + + // Overwrite values for Keys + let bodyData = JSON.parse(testRequestBody.raw); + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && overwriteValue.value) { + let orgValue = _.get(bodyData, overwriteValue.key), + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + + bodyData = _.set(bodyData, overwriteValue.key, newValue); + } + }); + bodyString = JSON.stringify(bodyData, null, 4); + + // Handle {{$randomInt}},{{$randomCreditCardMask}} conversion from string to number + find = ['"{{$randomInt}}"', '"{{$randomCreditCardMask}}"', '"{{$randomBankAccount}}"']; + replace = ['{{$randomInt}}', '{{$randomCreditCardMask}}', '{{$randomBankAccount}}']; + find.forEach(function (item, index) { + let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, '\\$1'); + bodyString = bodyString.replace(new RegExp(escapedFind, 'g'), replace[index]); + }); + + // Set testRequestBody.raw with stringified body + testRequestBody.raw = bodyString; + } + } + + return testRequestBody; + }, + + /** + * function to overwrite a request path variables with values defined by the postman testsuite + * @param {*} requestPathVariables request Path Variables object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestPathVariables = requestPathVariables.slice(), // clone requestPathVariables + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestPathVariables, abort early + if (!options.testSuite && requestPathVariables.length > 0) { + return requestPathVariables; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestPathVariables) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestPathVariables; + + // Overwrite value for path variable name + testRequestPathVariables.forEach((pathVar) => { + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && pathVar.name && overwriteValue.key === pathVar.name && + overwriteValue.value && pathVar.schema && pathVar.schema.example) { + let orgValue = pathVar.schema.example, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + pathVar.schema.type = 'string'; // Set schema as type string dynamic variable + pathVar.schema.example = newValue; + } + }); + }); + } + } + return testRequestPathVariables; + }, + + /** + * function to overwrite a request query params with values defined by the postman testsuite + * @param {*} requestQueryParam request Query Param object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestQueryParamByTests: function (requestQueryParam, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestQueryParam + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestQueryParam.key, abort early + if (!options.testSuite && !requestQueryParam.key) { + return requestQueryParam; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestQueryParams) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestQueryParams; + + // Overwrite value for query param key + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key && + overwriteValue.value && testRequestQueryParam.value) { + let orgValue = testRequestQueryParam.value, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestQueryParam.value = newValue; + } + }); + } + } + return testRequestQueryParam; + }, + + /** + * function to overwrite a request header with values defined by the postman testsuite + * @param {*} requestHeader request Header object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Modified requestBody object + */ + overwriteRequestHeaderByTests: function (requestHeader, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testRequestHeader = Object.assign({}, requestHeader), // clone requestHeader + testSuite = {}, + overwriteValues = [], + testSuiteOverwriteRequest = {}, + testSuiteOverwriteRequests = []; + + // Check for test suite flag and if there is no requestHeader.schema, abort early + if (!options.testSuite && !requestHeader.schema) { + return requestHeader; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + + if (testSuite.overwriteRequests) { + testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; + // Get the overwrite setting for the operationId + testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { + return or.openApiOperationId === operation.operationId; + }); + + if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestHeaders) { + overwriteValues = testSuiteOverwriteRequest.overwriteRequestHeaders; + // Overwrite value for header name + overwriteValues.forEach((overwriteValue) => { + if (overwriteValue.key && testRequestHeader.name && overwriteValue.key === testRequestHeader.name && + overwriteValue.value && testRequestHeader.schema && testRequestHeader.schema.example) { + let orgValue = testRequestHeader.schema.example, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestHeader.schema.example = newValue; + } + }); + } + } + return testRequestHeader; + }, + + /** + * function to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + * @param {*} oaSchema openAPI schema + * @returns {*} Modified openAPI schema object that is compatible with JSON schema validation + */ + convertUnsupportedJsonSchemaProperties: function(oaSchema) { + let jsonSchema = JSON.parse(JSON.stringify(oaSchema)); // Deep copy of the schema object + + // Convert unsupported OpenAPI(3.0) properties to valid JSON schema properties + // let jsonSchemaNotSupported = ['nullable', 'discriminator', 'readOnly', 'writeOnly', 'xml', + // 'externalDocs', 'example', 'deprecated']; + + // Recurse through OpenAPI Schema + const traverse = (obj) => { + for (let k in obj) { + if (obj.hasOwnProperty(k) && obj[k] && typeof obj[k] === 'object') { + if (obj[k].nullable && obj[k].nullable === true) { + // deletes nullable and adds "null" to type array if nullable is true + let jsonTypes = []; + jsonTypes.push(obj[k].type); + jsonTypes.push('null'); + obj[k].type = jsonTypes; + delete obj[k].nullable; + } + traverse(obj[k]); + } + } + }; + traverse(jsonSchema); + return jsonSchema; + } +}; diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index 851a31c51..c729e919f 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -13,6 +13,7 @@ const async = require('async'), openApiErr = require('./error.js'), ajvValidationError = require('./ajvValidationError'), utils = require('./utils.js'), + testSuite = require('./pmTestSuite.js'), defaultOptions = require('../lib/options.js').getOptions('use'), { Node, Trie } = require('./trie.js'), { validateSchema } = require('./ajvValidation'), @@ -2263,8 +2264,8 @@ module.exports = { _.forEach(reqParams.query, (queryParam) => { this.convertToPmQueryParameters(queryParam, REQUEST_TYPE.ROOT, components, options, schemaCache) .forEach((pmParam) => { - // Modify the request query params for the test suites - pmParam = this.overwriteRequestQueryParamByTests(pmParam, operation, operationItem, options); + // pmTestSuite - Modify the request query params for the test suites + pmParam = testSuite.overwriteRequestQueryParamByTests(pmParam, operation, operationItem, options); item.request.url.addQueryParams(pmParam); }); }); @@ -2284,8 +2285,8 @@ module.exports = { } }); item.request.url.variables.clear(); - // Modify the request paths variables for the test suites - reqParams.path = this.overwriteRequestPathByTests(reqParams.path, operation, operationItem, options); + // pmTestSuite - Modify the request paths variables for the test suites + reqParams.path = testSuite.overwriteRequestPathByTests(reqParams.path, operation, operationItem, options); item.request.url.variables.assimilate(this.convertPathVariables('param', pathVarArray, reqParams.path, components, options, schemaCache)); @@ -2302,8 +2303,8 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { - // Modify the request header for the test suites - header = this.overwriteRequestHeaderByTests(header, operation, operationItem, options); + // pmTestSuite - Modify the request header for the test suite + header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); }); @@ -2314,8 +2315,8 @@ module.exports = { reqBody = this.getRefObject(reqBody.$ref, components, options); } pmBody = this.convertToPmBody(reqBody, REQUEST_TYPE.ROOT, components, options, schemaCache); - // Modify the request body for the test suites - pmBody.body = this.overwriteRequestBodyByTests(pmBody.body, operation, operationItem, options); + // pmTestSuite - Modify the request body for the test suite + pmBody.body = testSuite.overwriteRequestBodyByTests(pmBody.body, operation, operationItem, options); item.request.body = pmBody.body; item.request.addHeader(pmBody.contentHeader); // extra form headers if encoding is present in request Body. @@ -2397,8 +2398,8 @@ module.exports = { }); } - // Generate tests for the response - pmTests = this.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); + // pmTestSuite - Generate tests for the response + pmTests = testSuite.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); if (!_.isEmpty(pmTests)) { // Add test event @@ -2408,602 +2409,6 @@ module.exports = { return item; }, - /** - * function to convert an openapi reponse object to object of postman tests - * @param {*} operation openapi operation object - * @param {*} operationItem - The operation item to get relevant information for the test generation - * @param {object} components - components defined in the OAS spec. These are used to - * resolve references while generating params. - * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @param {object} schemaCache - object storing schemaFaker and schmeResolution caches - * @returns {*} array of all query params - */ - convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { - options = _.merge({}, defaultOptions, options); - let testEvent = {}, - testSuite = {}, - testSuiteSettings = {}, - testSuiteLimits = [], - testSuiteExtensions = [], - requestsTests = [], - swagResponse = {}; - - // Check for test suite flag, abort early - if (!options.testSuite) { - return testEvent; - } - - // Check test suite for later usage, potentially convert object - if (options.testSuite && options.testSuiteSettings) { - testSuite = options.testSuiteSettings; - if (testSuite.generateTests) { - testSuiteSettings = options.testSuiteSettings.generateTests; - } - - // Limit the test generation to the following operations - if (testSuiteSettings.limitOperations) { - testSuiteLimits = testSuiteSettings.limitOperations; - } - - // Extend the generated test with additional operations - if (testSuite.extendTests) { - testSuiteExtensions = options.testSuiteSettings.extendTests; - - testSuiteExtensions.forEach((testExtension) => { - if (operation.operationId && testExtension.openApiOperationId && - operation.operationId === testExtension.openApiOperationId && - testExtension && testExtension.responseChecks) { - // Extend testSuiteSettings with testExtension settings - testSuiteSettings.responseChecks = _.merge( - {}, testSuiteSettings.responseChecks, testExtension.responseChecks - ); - } - }); - } - - } - - _.forOwn(operation.responses, (response, code) => { - // Skip the operations, if there are limits defined - if (operation.operationId && testSuiteLimits.length > 0 && - testSuiteLimits.indexOf(operation.operationId) === -1) { - return; // skip this response - } - - // Only support 2xx response checks - // TODO Investigate how to support other response codes like validation response 4xx or 5xx - if (!_.inRange(code, 200, 299)) { - return; // skip this response - } - - // Format response - swagResponse = response; - if (response.$ref) { - swagResponse = this.getRefObject(response.$ref, components, options); - } - - // Add status code check - // TODO Validate other request codes - // if (operationItem.method.toUpperCase() === 'GET') { - // requestsTests.push( - // '// Validate status code \n'+ - // 'pm.test("Status code should be '+code+'", function () {\n' + - // ' pm.response.to.have.status('+code+');\n' + - // '});\n' - // ) - // } - - // Add status success check - if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { - requestsTests.push( - '// Validate status 2xx \n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Status code is 2xx", function () {\n' + - ' pm.response.to.be.success;\n' + - '});\n' - ); - } - - // Add response timer check - if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { - let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); - requestsTests.push( - '// Validate response time \n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Response time is less than ' + maxMs + 'ms", function () {\n' + - ' pm.expect(pm.response.responseTime).to.be.below(' + maxMs + ');\n' + - '});\n' - ); - } - - // Process the response content - _.forOwn(swagResponse.content, (content, contentType) => { - if (contentType) { - // Add content-type check - if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { - requestsTests.push( - '// Validate content-type \n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Content-Type is ' + contentType + '", function () {\n' + - ' pm.expect(pm.response.headers.get("Content-Type")).to.include("' + contentType + '");\n' + - '});\n' - ); - } - - if (contentType === APP_JSON) { - let resolvedSchema, - assignPmVariablesCounter = 0; - - // Add JSON body check - if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { - requestsTests.push( - '// Response should have JSON Body\n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Response has JSON Body", function () {\n' + - ' pm.response.to.have.jsonBody();\n' + - '});\n'); - } - - // Add JSON schema check - if (_.get(testSuiteSettings, 'responseChecks.schemaValidation.enabled') && content.schema) { - let schemaContent = content.schema; - // if (content.schema.$ref) { - // schemaContent = this.getRefObject(content.schema.$ref, components, options); - // } - - try { - // When processing a reference, schema.type could also be undefined - resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, - schemaCache, PROCESSING_TYPE.VALIDATION); - - // deletes nullable and adds "null" to type array if nullable is true - let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); - - requestsTests.push( - '// Response Validation\n' + - 'const schema = ' + JSON.stringify(jsonSchema) + '\n' + - '\n' + - '// Test whether the response matches the schema\n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Schema is valid", function() {\n' + - ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + - '});\n' - ); - } - catch (e) { - console.warn('JSON schema check failed', e); - console.warn('invalid schemaValidation for ', content); - } - } - - // Automatic set the response.id as an environment variable for chaining option - if (operationItem.method.toUpperCase() === 'POST' && resolvedSchema !== undefined && - resolvedSchema.properties && resolvedSchema.properties.id) { - // Only set the jsonData once - if (assignPmVariablesCounter === 0) { - requestsTests.push('// Set response object as internal variable\n' + - 'let jsonData = pm.response.json();\n'); - } - requestsTests.push( - '// pm.environment - Set ' + operation.operationId + '.id as environment variable \n' + - 'if (jsonData.id) {\n' + - ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + - ' console.log("pm.environment - use {{' + operation.operationId + '.id}} ' + - 'as variable for value", jsonData.id);\n' + - '};\n'); - console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + - operation.operationId + '.id}} as variable for "reponse.id'); - assignPmVariablesCounter++; - } - - // Assign defined PmVariables for JSON response - if (testSuite.assignPmVariables) { - let testSuiteAssignPmVariables = testSuite.assignPmVariables; - // Get the environmentVariables settings for the current operationId - let assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { - let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; - - assignEnvironmentVariables.forEach((environmentVariable) => { - // Set response body property as variable - if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { - let responseProp = (environmentVariable.responseProp) ? - environmentVariable.responseProp : environmentVariable.responseBodyProp; - // Set variable name - let varName = operation.operationId + '.' + responseProp; - if (environmentVariable.name) { - varName = environmentVariable.name; - } - // Only set the jsonData once - if (assignPmVariablesCounter === 0) { - requestsTests.push('// Set response object as internal variable\n' + - 'let jsonData = pm.response.json();\n'); - } - requestsTests.push( - '// pm.environment - Set ' + varName + ' as environment variable \n' + - 'if (jsonData.' + responseProp + ') {\n' + - ' pm.environment.set("' + varName + '",jsonData.' + responseProp + ');\n' + - ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + - 'jsonData.' + responseProp + ');\n' + - '};\n'); - console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + - 'as variable for "reponse.' + responseProp + '"'); - assignPmVariablesCounter++; - } - }); - } - } - - } - } - - }); - - // Process the response header - _.forOwn(swagResponse.headers, (header, headerKey) => { - - if (headerKey && header.name) { - // Add content-type check - if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { - requestsTests.push( - '// Validate header \n' + - 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + - ' - Response header ' + header.name + ' is present", function () {\n' + - ' pm.response.to.have.header("' + header.name + '");\n' + - '});\n' - ); - } - } - - }); - - // Assign defined PmVariables for response headers - if (testSuite.assignPmVariables) { - let testSuiteAssignPmVariables = testSuite.assignPmVariables, - // Get the environmentVariables settings for the current operationId - assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { - let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; - - assignEnvironmentVariables.forEach((environmentVariable) => { - // Set response header property as variable - if (environmentVariable.responseHeaderProp) { - let headerProp = environmentVariable.responseHeaderProp; - // Set variable name - let varName = operation.operationId + '.' + headerProp; - if (environmentVariable.name) { - varName = environmentVariable.name; - } - - // Safe variable name - let safeVarName = varName.replace(/-/g, '') - .replace(/_/g, '').replace(/ /g, '') - .replace(/\./g, ''); - - requestsTests.push( - '// pm.environment - Set ' + varName + ' as environment variable \n' + - 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + - 'if (' + safeVarName + ' !== undefined) {\n' + - ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + - ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + - '' + safeVarName + ');\n' + - '};\n'); - console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + - 'as variable for "header.' + headerProp + '"'); - } - - }); - } - } - - }); - - // Add test extensions that are defined in the test suite file - if (testSuiteExtensions.length > 0) { - testSuiteExtensions.forEach((testExtension) => { - if (operation.operationId && testExtension.openApiOperationId && - operation.operationId === testExtension.openApiOperationId) { - - if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { - // Reset generated tests - requestsTests = []; - } - - // Add test extensions - if (testExtension.tests && testExtension.tests.length > 0) { - testExtension.tests.forEach((postmanTest) => { - try { - // Extend the generated tests, with the test extension scripts - requestsTests.push(postmanTest); - } - catch (e) { - console.warn('invalid extendTests for ' + testExtension.openApiOperationI); - } - }); - } - } - }); - } - - // Add tests to postman item - if (requestsTests.length > 0) { - testEvent = { - listen: 'test', - script: requestsTests - }; - } - return testEvent; - }, - - /** - * function to overwrite a request body with values defined by the postman testsuite - * @param {*} requestBody object - * @param {*} operation openapi operation object - * @param {*} operationItem - The operation item to get relevant information for the test generation - * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @returns {*} Modified requestBody object - */ - overwriteRequestBodyByTests: function (requestBody, operation, operationItem, options) { - options = _.merge({}, defaultOptions, options); - let testRequestBody = Object.assign({}, requestBody), // clone requestBody before manipulation - bodyString, - find, - replace, - testSuite = {}, - overwriteValues = [], - testSuiteOverwriteRequest = {}, - testSuiteOverwriteRequests = []; - - // Check for test suite flag and if there is RAW requestBody, abort early - if (!options.testSuite && !requestBody.raw) { - return requestBody; - } - - if (options.testSuite && options.testSuiteSettings) { - testSuite = options.testSuiteSettings; - } - - if (testSuite.overwriteRequests) { - testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; - // Get the overwrite setting for the operationId - testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestBody) { - overwriteValues = testSuiteOverwriteRequest.overwriteRequestBody; - - // Overwrite values for Keys - let bodyData = JSON.parse(testRequestBody.raw); - overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && overwriteValue.value) { - let orgValue = _.get(bodyData, overwriteValue.key), - newValue = overwriteValue.value; - - if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; - } - - bodyData = _.set(bodyData, overwriteValue.key, newValue); - } - }); - bodyString = JSON.stringify(bodyData, null, 4); - - // Handle {{$randomInt}},{{$randomCreditCardMask}} conversion from string to number - find = ['"{{$randomInt}}"', '"{{$randomCreditCardMask}}"', '"{{$randomBankAccount}}"']; - replace = ['{{$randomInt}}', '{{$randomCreditCardMask}}', '{{$randomBankAccount}}']; - find.forEach(function (item, index) { - let escapedFind = item.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, '\\$1'); - bodyString = bodyString.replace(new RegExp(escapedFind, 'g'), replace[index]); - }); - - // Set testRequestBody.raw with stringified body - testRequestBody.raw = bodyString; - } - } - - return testRequestBody; - }, - - /** - * function to overwrite a request path variables with values defined by the postman testsuite - * @param {*} requestPathVariables request Path Variables object - * @param {*} operation openapi operation object - * @param {*} operationItem - The operation item to get relevant information for the test generation - * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @returns {*} Modified requestBody object - */ - overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { - options = _.merge({}, defaultOptions, options); - let testRequestPathVariables = requestPathVariables.slice(), // clone requestPathVariables - testSuite = {}, - overwriteValues = [], - testSuiteOverwriteRequest = {}, - testSuiteOverwriteRequests = []; - - // Check for test suite flag and if there is no requestPathVariables, abort early - if (!options.testSuite && requestPathVariables.length > 0) { - return requestPathVariables; - } - - if (options.testSuite && options.testSuiteSettings) { - testSuite = options.testSuiteSettings; - } - - if (testSuite.overwriteRequests) { - testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; - // Get the overwrite setting for the operationId - testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestPathVariables) { - overwriteValues = testSuiteOverwriteRequest.overwriteRequestPathVariables; - - // Overwrite value for path variable name - testRequestPathVariables.forEach((pathVar) => { - overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && pathVar.name && overwriteValue.key === pathVar.name && - overwriteValue.value && pathVar.schema && pathVar.schema.example) { - let orgValue = pathVar.schema.example, - newValue = overwriteValue.value; - - if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; - } - pathVar.schema.type = 'string'; // Set schema as type string dynamic variable - pathVar.schema.example = newValue; - } - }); - }); - } - } - return testRequestPathVariables; - }, - - /** - * function to overwrite a request query params with values defined by the postman testsuite - * @param {*} requestQueryParam request Query Param object - * @param {*} operation openapi operation object - * @param {*} operationItem - The operation item to get relevant information for the test generation - * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @returns {*} Modified requestBody object - */ - overwriteRequestQueryParamByTests: function (requestQueryParam, operation, operationItem, options) { - options = _.merge({}, defaultOptions, options); - let testRequestQueryParam = Object.assign({}, requestQueryParam), // clone requestQueryParam - testSuite = {}, - overwriteValues = [], - testSuiteOverwriteRequest = {}, - testSuiteOverwriteRequests = []; - - // Check for test suite flag and if there is no requestQueryParam.key, abort early - if (!options.testSuite && !requestQueryParam.key) { - return requestPathVariables; - } - - if (options.testSuite && options.testSuiteSettings) { - testSuite = options.testSuiteSettings; - } - - if (testSuite.overwriteRequests) { - testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; - // Get the overwrite setting for the operationId - testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestQueryParams) { - overwriteValues = testSuiteOverwriteRequest.overwriteRequestQueryParams; - - // Overwrite value for query param key - overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key && - overwriteValue.value && testRequestQueryParam.value) { - let orgValue = testRequestQueryParam.value, - newValue = overwriteValue.value; - - if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; - } - testRequestQueryParam.value = newValue; - } - }); - } - } - return testRequestQueryParam; - }, - - /** - * function to overwrite a request header with values defined by the postman testsuite - * @param {*} requestHeader request Header object - * @param {*} operation openapi operation object - * @param {*} operationItem - The operation item to get relevant information for the test generation - * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @returns {*} Modified requestBody object - */ - overwriteRequestHeaderByTests: function (requestHeader, operation, operationItem, options) { - options = _.merge({}, defaultOptions, options); - let testRequestHeader = Object.assign({}, requestHeader), // clone requestHeader - testSuite = {}, - overwriteValues = [], - testSuiteOverwriteRequest = {}, - testSuiteOverwriteRequests = []; - - // Check for test suite flag and if there is no requestHeader.schema, abort early - if (!options.testSuite && !requestHeader.schema) { - return requestHeader; - } - - if (options.testSuite && options.testSuiteSettings) { - testSuite = options.testSuiteSettings; - } - - if (testSuite.overwriteRequests) { - testSuiteOverwriteRequests = options.testSuiteSettings.overwriteRequests; - // Get the overwrite setting for the operationId - testSuiteOverwriteRequest = testSuiteOverwriteRequests.find((or) => { - return or.openApiOperationId === operation.operationId; - }); - - if (testSuiteOverwriteRequest && testSuiteOverwriteRequest.overwriteRequestHeaders) { - overwriteValues = testSuiteOverwriteRequest.overwriteRequestHeaders; - // Overwrite value for header name - overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && testRequestHeader.name && overwriteValue.key === testRequestHeader.name && - overwriteValue.value && testRequestHeader.schema && testRequestHeader.schema.example) { - let orgValue = testRequestHeader.schema.example, - newValue = overwriteValue.value; - - if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; - } - testRequestHeader.schema.example = newValue; - } - }); - } - } - return testRequestHeader; - }, - - /** - * function to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties - * @param {*} oaSchema openAPI schema - * @returns {*} Modified openAPI schema object that is compatible with JSON schema validation - */ - convertUnsupportedJsonSchemaProperties: function(oaSchema) { - let jsonSchema = JSON.parse(JSON.stringify(oaSchema)); // Deep copy of the schema object - - // Convert unsupported OpenAPI(3.0) properties to valid JSON schema properties - // let jsonSchemaNotSupported = ['nullable', 'discriminator', 'readOnly', 'writeOnly', 'xml', - // 'externalDocs', 'example', 'deprecated']; - - // Recurse through OpenAPI Schema - const traverse = (obj) => { - for (let k in obj) { - if (obj.hasOwnProperty(k) && obj[k] && typeof obj[k] === 'object') { - if (obj[k].nullable && obj[k].nullable === true) { - // deletes nullable and adds "null" to type array if nullable is true - let jsonTypes = []; - jsonTypes.push(obj[k].type); - jsonTypes.push('null'); - obj[k].type = jsonTypes; - delete obj[k].nullable; - } - traverse(obj[k]); - } - } - }; - traverse(jsonSchema); - return jsonSchema; - }, - /** * function to convert an openapi query params object to array of query params * @param {*} reqParams openapi query params object From bbbf81a3e76e6e9955778e149f2fb4034ef3623b Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 11 Apr 2021 22:47:00 +0200 Subject: [PATCH 42/52] Merged version 2.5.1 from master --- CHANGELOG.md | 15 +++++ bin/openapi2postmanv2.js | 2 +- lib/deref.js | 4 +- lib/schemaUtils.js | 61 +++++++++++-------- package-lock.json | 8 +-- package.json | 4 +- .../examples_outside_schema.json | 20 ++++++ .../valid_openapi/required_in_parameters.json | 4 +- test/unit/base.test.js | 38 +++++++++--- test/unit/deref.test.js | 17 ++++++ test/unit/util.test.js | 8 ++- test/unit/validator.test.js | 38 ++++++++++++ 12 files changed, 173 insertions(+), 46 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b5f94b72e..61375c7c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,20 @@ # OpenAPI-Postman Changelog +#### v2.5.1 (March 19, 2021) +* Fixed wrongly defined maxLength and minLength options for schemaFaker. + +#### v2.5.0 (March 18, 2021) +* Fix for [#337](https://github.com/postmanlabs/openapi-to-postman/issues/337) - Fixed issue where non-required params were not disbled for urlencoded body. +* Fix for [#338](https://github.com/postmanlabs/openapi-to-postman/issues/338) - Fixed issue where examples which had non-truthy value were not present in converted collection. +* Fixed issue where resolved schema contain conflicting info regarding pattern and format resulting in validation mismatches. +* Fixed issue where definition for path variable in path caused no matched request for certain paths in validation. + +#### v2.4.0 (March 08, 2021) +* Fix for [#9396](https://github.com/postmanlabs/postman-app-support/issues/9396) - Fixed validation issue where result contained matched endpoints in incorrect order. +* Fix for [#328](https://github.com/postmanlabs/openapi-to-postman/issues/328) - Fixed incorrect usage of id field in sdk.variable and used key. +* Fix for [issue](https://community.postman.com/t/user-feedback-updating-api-elements/13308/13) where implicit headers generation and validation. +* Fix for [#329](https://github.com/postmanlabs/openapi-to-postman/issues/329) - Fixed issue where --test was failing for cli. + #### v2.3.0 (February 19, 2021) * Fixed [issue](https://community.postman.com/t/user-feedback-updating-api-elements/13308/13) where content type header was reported missing in validation even if present. * Feature request [#9046](https://github.com/postmanlabs/postman-app-support/issues/9046) - Added support for validation of request body of type urlencoded. diff --git a/bin/openapi2postmanv2.js b/bin/openapi2postmanv2.js index 000926ea3..0b26ea10e 100755 --- a/bin/openapi2postmanv2.js +++ b/bin/openapi2postmanv2.js @@ -160,7 +160,7 @@ function convert(swaggerData) { } if (testFlag) { - swaggerData = fs.readFileSync('../examples/sample-swagger.yaml', 'utf8'); + swaggerData = fs.readFileSync(path.resolve(__dirname, '..', 'examples', 'sample-swagger.yaml'), 'utf8'); convert(swaggerData); } else if (inputFile) { diff --git a/lib/deref.js b/lib/deref.js index e0c75e572..3faa4d92e 100644 --- a/lib/deref.js +++ b/lib/deref.js @@ -302,8 +302,8 @@ module.exports = { schema.type = 'string'; } - // Discard format if not supported by both json-schema-faker and ajv - if (!_.includes(SUPPORTED_FORMATS, schema.format)) { + // Discard format if not supported by both json-schema-faker and ajv or pattern is also defined + if (!_.includes(SUPPORTED_FORMATS, schema.format) || (schema.pattern && schema.format)) { delete schema.format; } } diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index c729e919f..c2773fdfd 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -100,8 +100,7 @@ const async = require('async'), schemaFaker.option({ requiredOnly: false, optionalsProbability: 1.0, // always add optional fields - minLength: 4, // for faked strings - maxLength: 4, + maxLength: 256, minItems: 1, // for arrays maxItems: 20, // limit on maximum number of items faked for (type: arrray) useDefaultValue: true, @@ -307,11 +306,15 @@ module.exports = { // All complicated logic removed // This simply replaces all instances of {text} with {{text}} // text cannot have any of these 3 chars: /{} - // and {text} cannot be followed by a } // {{text}} will not be converted - // https://regex101.com/r/9N1520/1 - return url - .replace(/(\{[^\/\{\}]+\})(?!\})/g, '{$1}'); + + let replacer = function (match, p1, offset, string) { + if (string[offset - 1] === '{' && string[offset + match.length + 1] !== '}') { + return match; + } + return '{' + p1 + '}'; + }; + return url.replace(/(\{[^\/\{\}]+\})/g, replacer); }, /** @@ -400,7 +403,7 @@ module.exports = { else if (examples) { let exampleToUse = _.get(examples, '[0].value'); - exampleToUse && (_.set(parameter, 'schema.example', exampleToUse)); + !_.isUndefined(exampleToUse) && (_.set(parameter, 'schema.example', exampleToUse)); } }, @@ -419,7 +422,7 @@ module.exports = { _.forOwn(serverVariables, (value, key) => { let description = this.getParameterDescription(value); variables.push(new sdk.Variable({ - id: key, + key: key, value: value.default || '', description: description })); @@ -427,7 +430,7 @@ module.exports = { } if (keyName) { variables.push(new sdk.Variable({ - id: keyName, + key: keyName, value: serverUrl, type: 'string' })); @@ -1697,11 +1700,9 @@ module.exports = { if (_.get(contentObj[URLENCODED], 'schema.type') === 'object') { description = _.get(contentObj[URLENCODED], ['schema', 'properties', key, 'description'], ''); - required = _.get(contentObj[URLENCODED], ['schema', 'properties', key, 'required'], false); + required = _.includes(_.get(contentObj[URLENCODED], ['schema', 'required']), key); enumValue = _.get(contentObj[URLENCODED], ['schema', 'properties', key, 'enum']); } - description = (required ? '(Required) ' : '') + description + - (enumValue ? ' (This can only be one of ' + enumValue + ')' : ''); !encoding[key] && (encoding[key] = {}); encoding[key].name = key; @@ -1711,6 +1712,7 @@ module.exports = { // for urlencoded body serialisation is treated similar to query param // reference https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#fixed-fields-13 encoding[key].in = 'query'; + _.isBoolean(required) && (encoding[key].required = required); encoding[key].description = description; params = this.convertParamsWithStyle(encoding[key], value, PARAMETER_SOURCE.REQUEST, components, @@ -1757,7 +1759,7 @@ module.exports = { if (_.get(contentObj[FORM_DATA], 'schema.type') === 'object') { description = _.get(contentObj[FORM_DATA], ['schema', 'properties', key, 'description'], ''); - required = _.get(contentObj[FORM_DATA], ['schema', 'properties', key, 'required'], false); + required = _.includes(_.get(contentObj[FORM_DATA], ['schema', 'required']), key); enumValue = _.get(contentObj[FORM_DATA], ['schema', 'properties', key, 'enum']); } description = (required ? '(Required) ' : '') + description + @@ -1886,8 +1888,11 @@ module.exports = { header = value; } header.name = key; - responseHeaders.push(this.convertToPmHeader(header, REQUEST_TYPE.EXAMPLE, - PARAMETER_SOURCE.RESPONSE, components, options, schemaCache)); + + if (!_.includes(IMPLICIT_HEADERS, _.toLower(key))) { + responseHeaders.push(this.convertToPmHeader(header, REQUEST_TYPE.EXAMPLE, + PARAMETER_SOURCE.RESPONSE, components, options, schemaCache)); + } } }); @@ -2140,7 +2145,7 @@ module.exports = { convertedPathVar = _.get(this.convertParamsWithStyle(element, fakedData, PARAMETER_SOURCE.REQUEST, components, schemaCache, options), '[0]', {}); - variableStore[element.name] = _.assign(convertedPathVar, { id: element.name, type: 'collection' }); + variableStore[element.name] = _.assign(convertedPathVar, { key: element.name, type: 'collection' }); } }); // accounting for the overriding of the root level and path level servers object if present at the operation level @@ -2171,7 +2176,7 @@ module.exports = { sanitizeResult.collectionVars.forEach((element) => { if (!variableStore[element.name]) { variableStore[element.name] = { - id: element.name, + key: element.name, value: element.value || '', description: element.description, type: 'collection' @@ -2303,10 +2308,12 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { - // pmTestSuite - Modify the request header for the test suite - header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); - item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, - components, options, schemaCache)); + if (!_.includes(IMPLICIT_HEADERS, _.toLower(_.get(header, 'name')))) { + // pmTestSuite - Modify the request header for the test suite + header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); + item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, + components, options, schemaCache)); + } }); // adding Request Body and Content-Type header @@ -2506,8 +2513,11 @@ module.exports = { }); }); - // keep endpoints with more fix matched segments first in result - _.each(_.orderBy(filteredPathItemsArray, ['fixedMatchedSegments', 'variableMatchedSegments'], ['desc']), (fp) => { + // order endpoints with more fix matched segments and variable matched segments (for tie in former) first in result + filteredPathItemsArray = _.orderBy(filteredPathItemsArray, ['fixedMatchedSegments', 'variableMatchedSegments'], + ['desc', 'desc']); + + _.each(filteredPathItemsArray, (fp) => { let path = fp.path, pathItemObject = fp.pathItem, score = fp.matchScore, @@ -3648,7 +3658,10 @@ module.exports = { schemaPathPrefix + '.requestBody.content', _.get(schemaPath, 'requestBody.content'), mismatchProperty, options); - _.each(_.filter(schemaHeaders, (h) => { return h.required; }), (header) => { + _.each(_.filter(schemaHeaders, (h) => { + // exclude non-required and implicit header from further validation + return h.required && !_.includes(IMPLICIT_HEADERS, _.toLower(h.name)); + }), (header) => { if (!_.find(reqHeaders, (param) => { return param.key === header.name; })) { // assign parameter example(s) as schema examples; diff --git a/package-lock.json b/package-lock.json index c33aefff4..f31da7f85 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,6 +1,6 @@ { "name": "openapi-to-postmanv2", - "version": "2.3.0", + "version": "2.5.1", "lockfileVersion": 1, "requires": true, "dependencies": { @@ -1408,9 +1408,9 @@ } }, "lodash": { - "version": "4.17.20", - "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.20.tgz", - "integrity": "sha512-PlhdFcillOINfeV7Ni6oF1TAEayyZBoZ8bcshTHqOYJYlrqzRK5hagpagky5o4HfCzzd1TRkXPMFq6cKk9rGmA==" + "version": "4.17.21", + "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz", + "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==" }, "lodash.clonedeep": { "version": "4.5.0", diff --git a/package.json b/package.json index ab2a5b60d..b482cca8a 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "openapi-to-postmanv2", - "version": "2.3.0", + "version": "2.5.1", "description": "Convert a given OpenAPI specification to Postman Collection v2.0", "homepage": "https://github.com/postmanlabs/openapi-to-postman", "bugs": "https://github.com/postmanlabs/openapi-to-postman/issues", @@ -121,7 +121,7 @@ "commander": "2.20.3", "js-yaml": "3.13.1", "json-schema-merge-allof": "0.7.0", - "lodash": "4.17.20", + "lodash": "4.17.21", "oas-resolver-browser": "2.5.1", "path-browserify": "1.0.1", "postman-collection": "3.6.6", diff --git a/test/data/valid_openapi/examples_outside_schema.json b/test/data/valid_openapi/examples_outside_schema.json index 5a05598d5..0303f5adc 100644 --- a/test/data/valid_openapi/examples_outside_schema.json +++ b/test/data/valid_openapi/examples_outside_schema.json @@ -7,6 +7,26 @@ "paths": { "/pet": { "post": { + "parameters": [ + { + "in": "query", + "name": "limit", + "schema": { + "type": "integer", + "example": 25 + }, + "examples": { + "zero": { + "value": 0, + "summary": "A sample limit value # Optional description" + }, + "max": { + "value": 50, + "summary": "A sample limit value" + } + } + } + ], "requestBody": { "content": { "application/json": { diff --git a/test/data/valid_openapi/required_in_parameters.json b/test/data/valid_openapi/required_in_parameters.json index 74f3214c4..65dba43c8 100644 --- a/test/data/valid_openapi/required_in_parameters.json +++ b/test/data/valid_openapi/required_in_parameters.json @@ -87,10 +87,10 @@ "content": { "multipart/form-data": { "schema": { + "required": [ "formParam1"], "properties": { "formParam1": { "description": "Description of formParam1", - "required": true, "type": "string" }, "formParam2": { @@ -112,10 +112,10 @@ "content": { "application/x-www-form-urlencoded": { "schema": { + "required": [ "urlencodedParam1"], "properties": { "urlencodedParam1": { "description": "Description of urlencodedParam1", - "required": true, "type": "string" }, "urlencodedParam2": { diff --git a/test/unit/base.test.js b/test/unit/base.test.js index 45fe91731..a66e12772 100644 --- a/test/unit/base.test.js +++ b/test/unit/base.test.js @@ -144,14 +144,14 @@ describe('CONVERT FUNCTION TESTS ', function() { expect(conversionResult.output[0].data).to.have.property('item'); expect(conversionResult.output[0].data).to.have.property('variable'); expect(conversionResult.output[0].data.variable).to.be.an('array'); - expect(conversionResult.output[0].data.variable[1].id).to.equal('format'); + expect(conversionResult.output[0].data.variable[1].key).to.equal('format'); expect(conversionResult.output[0].data.variable[1].value).to.equal('json'); - expect(conversionResult.output[0].data.variable[2].id).to.equal('path'); + expect(conversionResult.output[0].data.variable[2].key).to.equal('path'); expect(conversionResult.output[0].data.variable[2].value).to.equal('send-email'); - expect(conversionResult.output[0].data.variable[3].id).to.equal('new-path-variable-1'); + expect(conversionResult.output[0].data.variable[3].key).to.equal('new-path-variable-1'); // serialised value for object { R: 100, G: 200, B: 150 } expect(conversionResult.output[0].data.variable[3].value).to.equal('R,100,G,200,B,150'); - expect(conversionResult.output[0].data.variable[4].id).to.equal('new-path-variable-2'); + expect(conversionResult.output[0].data.variable[4].key).to.equal('new-path-variable-2'); // serialised value for array ["exampleString", "exampleString"] expect(conversionResult.output[0].data.variable[4].value).to.equal('exampleString,exampleString'); done(); @@ -409,6 +409,20 @@ describe('CONVERT FUNCTION TESTS ', function() { done(); }); }); + + it('[Github #338] Should contain non-truthy example from examples outside of schema instead of faked value' + + examplesOutsideSchema, function(done) { + Converter.convert({ type: 'file', data: examplesOutsideSchema }, + { schemaFaker: true, requestParametersResolution: 'example', exampleParametersResolution: 'example' }, + (err, conversionResult) => { + let rootRequest = conversionResult.output[0].data.item[0].request; + + expect(err).to.be.null; + expect(rootRequest.url.query[0].key).to.eql('limit'); + expect(rootRequest.url.query[0].value).to.eql('0'); + done(); + }); + }); }); it('[Github #117]- Should add the description in body params in case of urlencoded' + descriptionInBodyParams, function(done) { @@ -902,18 +916,19 @@ describe('CONVERT FUNCTION TESTS ', function() { requestUrl = conversionResult.output[0].data.item[0].request.url; collectionVars = conversionResult.output[0].data.variable; expect(requestUrl.host).to.eql(['{{baseUrl}}']); - expect(_.find(collectionVars, { id: 'baseUrl' }).value).to.eql('{{BASE_URI}}/api'); - expect(_.find(collectionVars, { id: 'BASE_URI' }).value).to.eql('https://api.example.com'); + expect(_.find(collectionVars, { key: 'baseUrl' }).value).to.eql('{{BASE_URI}}/api'); + expect(_.find(collectionVars, { key: 'BASE_URI' }).value).to.eql('https://api.example.com'); done(); }); }); - it('[Github #31] - should set optional params as disabled', function(done) { + it('[Github #31] & [GitHub #337] - should set optional params as disabled', function(done) { let options = { schemaFaker: true, disableOptionalParameters: true }; Converter.convert({ type: 'file', data: requiredInParams }, options, (err, conversionResult) => { expect(err).to.be.null; let requests = conversionResult.output[0].data.item[0].item, - request; + request, + urlencodedBody; // GET /pets // query1 required, query2 optional @@ -924,6 +939,13 @@ describe('CONVERT FUNCTION TESTS ', function() { expect(request.header[0].disabled).to.be.false; expect(request.header[1].disabled).to.be.true; + // POST /pets + // urlencoded body + urlencodedBody = requests[2].request.body.urlencoded; + expect(urlencodedBody[0].key).to.eql('urlencodedParam1'); + expect(urlencodedBody[0].disabled).to.be.false; + expect(urlencodedBody[1].key).to.eql('urlencodedParam2'); + expect(urlencodedBody[1].disabled).to.be.true; done(); }); }); diff --git a/test/unit/deref.test.js b/test/unit/deref.test.js index fb57fd31a..1911b968d 100644 --- a/test/unit/deref.test.js +++ b/test/unit/deref.test.js @@ -216,6 +216,23 @@ describe('DEREF FUNCTION TESTS ', function() { }); done(); }); + + it('should not contain format property in resolved schema if pattern is already defined', function(done) { + var schema = { + 'type': 'string', + 'format': 'date-time', + 'pattern': '^([0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])' + + '( (2[0-3]|[01][0-9]):([0-5][0-9]):([0-5][0-9]))?$' + }, + parameterSource = 'REQUEST', + output; + + output = deref.resolveRefs(schema, parameterSource, {}); + expect(output.type).to.equal('string'); + expect(output.format).to.be.undefined; + expect(output.pattern).to.eql(schema.pattern); + done(); + }); }); describe('resolveAllOf Function', function () { diff --git a/test/unit/util.test.js b/test/unit/util.test.js index 913f580ec..7c11546b2 100644 --- a/test/unit/util.test.js +++ b/test/unit/util.test.js @@ -231,7 +231,7 @@ describe('SCHEMA UTILITY FUNCTION TESTS ', function () { }, retVal = SchemaUtils.convertToPmCollectionVariables(serverVariables, null, null); expect(retVal).to.be.an('array'); - expect(retVal[0].id).to.equal('v1'); + expect(retVal[0].key).to.equal('v1'); expect(retVal[0].value).to.equal('v2.0'); }); @@ -249,9 +249,9 @@ describe('SCHEMA UTILITY FUNCTION TESTS ', function () { expect(retVal).to.be.an('array'); - expect(retVal[0].id).to.equal('v1'); + expect(retVal[0].key).to.equal('v1'); expect(retVal[0].value).to.equal('v2.0'); - expect(retVal[1].id).to.equal('baseUrl'); + expect(retVal[1].key).to.equal('baseUrl'); expect(retVal[1].value).to.equal('hello.com'); }); }); @@ -2017,6 +2017,8 @@ describe('SCHEMA UTILITY FUNCTION TESTS ', function () { expect(SchemaUtils.fixPathVariablesInUrl('{{a}}')).to.equal('{{a}}'); + expect(SchemaUtils.fixPathVariablesInUrl('/agents/{agentId}}')).to.equal('/agents/{{agentId}}}'); + expect(SchemaUtils.fixPathVariablesInUrl('{{a}}://{b}.com/{pathvar}/{morevar}')) .to.equal('{{a}}://{{b}}.com/{{pathvar}}/{{morevar}}'); diff --git a/test/unit/validator.test.js b/test/unit/validator.test.js index 535b1e8a3..49ba378a2 100644 --- a/test/unit/validator.test.js +++ b/test/unit/validator.test.js @@ -669,4 +669,42 @@ describe('VALIDATE FUNCTION TESTS ', function () { done(); }); }); + + describe('findMatchingRequestFromSchema function', function () { + it('#GITHUB-9396 Should maintain correct order of matched endpoint', function (done) { + let schema = { + paths: { + '/lookups': { + 'get': { 'summary': 'Lookup Job Values' } + }, + '/{jobid}': { + 'get': { + 'summary': 'Get Job by ID', + 'parameters': [ + { + 'in': 'path', + 'name': 'jobid', + 'schema': { + 'type': 'string' + }, + 'required': true, + 'description': 'Unique identifier for a job to retrieve.', + 'example': '{{jobid}}' + } + ] + } + } + } + }, + schemaPath = '{{baseUrl}}/{{jobid}}', + result; + + result = schemaUtils.findMatchingRequestFromSchema('GET', schemaPath, schema, { strictRequestMatching: true }); + + expect(result).to.have.lengthOf(2); + expect(result[0].name).to.eql('GET /{jobid}'); + expect(result[1].name).to.eql('GET /lookups'); + done(); + }); + }); }); From fba83630bc41200dd5e4f5c760a8a3a29eff4bd8 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 20 Apr 2021 17:28:41 +0200 Subject: [PATCH 43/52] Added support for removal of request headers, path variables, query params --- TESTGENERATION.md | 4 ++++ lib/pmTestSuite.js | 32 ++++++++++++++++++++++++-------- lib/schemaUtils.js | 4 ++-- 3 files changed, 30 insertions(+), 10 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 6845983f9..a09404162 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -201,6 +201,7 @@ Properties explained: to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - **overwrite (Boolean true/false | Default: true)** : Overwrites the request query param value OR attach the value to the original request query param value. + - **remove (Boolean true/false | Default: false)** : Removes the request query param - **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path Variables. - **key (string)** : The key that will be targeted in the request Path variables to overwrite/extend. @@ -209,6 +210,7 @@ Properties explained: to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - **overwrite (Boolean true/false | Default: true)** : Overwrites the request path variable value OR attach the value to the original request Path variable value. + - **remove (Boolean true/false | Default: false)** : Removes the request path variable - **overwriteRequestHeaders (Array)** : Array of key/value pairs to overwrite in the Postman Request Headers. - **key (string)** : The key that will be targeted in the request Headers to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the value in the request headers OR use @@ -216,6 +218,7 @@ Properties explained: to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - **overwrite (Boolean true/false | Default: true)** : Overwrites the request header value OR attach the value to the original request header value. + - **remove (Boolean true/false | Default: false)** : Removes the request headers - **overwriteRequestBody (Array)** : Array of key/value pairs to overwrite in the Postman Request Body. - **key (string)** : The key that will be targeted in the request body to overwrite/extend. - **value (string)** : The value that will be used to overwrite/extend the key in the request body OR use @@ -223,6 +226,7 @@ Properties explained: to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - **overwrite (Boolean true/false | Default: true)** : Overwrites the request body value OR attach the value to the original request body value. + - **remove (Boolean true/false | Default: false)** : Removes the request body property, including the value Postman request body before: diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index e20abe307..be7af9f38 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -381,7 +381,7 @@ module.exports = { // Overwrite values for Keys let bodyData = JSON.parse(testRequestBody.raw); overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && overwriteValue.value) { + if (overwriteValue.key && overwriteValue.hasOwnProperty('value')) { let orgValue = _.get(bodyData, overwriteValue.key), newValue = overwriteValue.value; @@ -390,6 +390,10 @@ module.exports = { } bodyData = _.set(bodyData, overwriteValue.key, newValue); + + if (overwriteValue.remove === true) { + bodyData = _.omit(bodyData, overwriteValue.key); + } } }); bodyString = JSON.stringify(bodyData, null, 4); @@ -446,11 +450,11 @@ module.exports = { overwriteValues = testSuiteOverwriteRequest.overwriteRequestPathVariables; // Overwrite value for path variable name - testRequestPathVariables.forEach((pathVar) => { + testRequestPathVariables.forEach((pathVar, index) => { overwriteValues.forEach((overwriteValue) => { if (overwriteValue.key && pathVar.name && overwriteValue.key === pathVar.name && - overwriteValue.value && pathVar.schema && pathVar.schema.example) { - let orgValue = pathVar.schema.example, + overwriteValue.hasOwnProperty('value') && pathVar.schema) { + let orgValue = (pathVar.schema.example ? pathVar.schema.example : null), newValue = overwriteValue.value; if (overwriteValue.overwrite === false) { @@ -458,6 +462,10 @@ module.exports = { } pathVar.schema.type = 'string'; // Set schema as type string dynamic variable pathVar.schema.example = newValue; + + if (overwriteValue.remove === true) { + testRequestPathVariables.splice(index, 1); + } } }); }); @@ -504,7 +512,7 @@ module.exports = { // Overwrite value for query param key overwriteValues.forEach((overwriteValue) => { if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key && - overwriteValue.value && testRequestQueryParam.value) { + overwriteValue.hasOwnProperty('value') && testRequestQueryParam.hasOwnProperty('value')) { let orgValue = testRequestQueryParam.value, newValue = overwriteValue.value; @@ -512,6 +520,10 @@ module.exports = { newValue = orgValue + newValue; } testRequestQueryParam.value = newValue; + + if (overwriteValue.remove === true) { + testRequestQueryParam = {}; + } } }); } @@ -556,14 +568,18 @@ module.exports = { // Overwrite value for header name overwriteValues.forEach((overwriteValue) => { if (overwriteValue.key && testRequestHeader.name && overwriteValue.key === testRequestHeader.name && - overwriteValue.value && testRequestHeader.schema && testRequestHeader.schema.example) { - let orgValue = testRequestHeader.schema.example, + overwriteValue.hasOwnProperty('value') && testRequestHeader.schema) { + let orgValue = (testRequestHeader.schema.example ? testRequestHeader.schema.example : null), newValue = overwriteValue.value; if (overwriteValue.overwrite === false) { newValue = orgValue + newValue; } testRequestHeader.schema.example = newValue; + + if (overwriteValue.remove === true) { + testRequestHeader = {}; + } } }); } @@ -576,7 +592,7 @@ module.exports = { * @param {*} oaSchema openAPI schema * @returns {*} Modified openAPI schema object that is compatible with JSON schema validation */ - convertUnsupportedJsonSchemaProperties: function(oaSchema) { + convertUnsupportedJsonSchemaProperties: function (oaSchema) { let jsonSchema = JSON.parse(JSON.stringify(oaSchema)); // Deep copy of the schema object // Convert unsupported OpenAPI(3.0) properties to valid JSON schema properties diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index c2773fdfd..acb6e5642 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2308,9 +2308,9 @@ module.exports = { // adding headers to request from reqParam _.forEach(reqParams.header, (header) => { + // pmTestSuite - Modify the request header for the test suite + header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); if (!_.includes(IMPLICIT_HEADERS, _.toLower(_.get(header, 'name')))) { - // pmTestSuite - Modify the request header for the test suite - header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); } From b25fa443fc2680cec95b7c32484a8426bf1680f3 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Wed, 28 Apr 2021 09:46:52 +0200 Subject: [PATCH 44/52] Added the getRefObject as a temp workaround for dependency Patch related to the stripping of "content-type" defined in OpenAPI https://github.com/postmanlabs/openapi-to-postman/issues/361 --- lib/pmTestSuite.js | 50 +++++++++++++++++++++++++++++++++++++++++++++- lib/schemaUtils.js | 4 ++-- 2 files changed, 51 insertions(+), 3 deletions(-) diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index be7af9f38..1b75333e4 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -617,5 +617,53 @@ module.exports = { }; traverse(jsonSchema); return jsonSchema; - } + }, + + // TODO remove this function, use a reference to schemaUtils instead + /** + * @param {*} $ref reference object + * @returns {Object} reference object from the saved components + * @no-unit-tests + */ + getRefObject: function($ref, components, options) { + options = _.merge({}, defaultOptions, options); + var refObj, savedSchema; + + savedSchema = $ref.split('/').slice(1).map((elem) => { + // https://swagger.io/docs/specification/using-ref#escape + // since / is the default delimiter, slashes are escaped with ~1 + return decodeURIComponent( + elem + .replace(/~1/g, '/') + .replace(/~0/g, '~') + ); + }); + // at this stage, savedSchema is [components, part1, parts] + // must have min. 2 segments after "#/components" + if (savedSchema.length < 3) { + console.warn(`ref ${$ref} not found.`); + return { value: `reference ${$ref} not found in the given specification` }; + } + + if (savedSchema[0] !== 'components' && savedSchema[0] !== 'paths') { + console.warn(`Error reading ${$ref}. Can only use references from components and paths`); + return { value: `Error reading ${$ref}. Can only use references from components and paths` }; + } + + // at this point, savedSchema is similar to ['components', 'schemas','Address'] + // components is actually components and paths (an object with components + paths as 1st-level-props) + refObj = _.get(components, savedSchema); + + if (!refObj) { + console.warn(`ref ${$ref} not found.`); + return { value: `reference ${$ref} not found in the given specification` }; + } + + if (refObj.$ref) { + return this.getRefObject(refObj.$ref, components, options); + } + + return refObj; + }, + }; diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index acb6e5642..3e017b968 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2310,10 +2310,10 @@ module.exports = { _.forEach(reqParams.header, (header) => { // pmTestSuite - Modify the request header for the test suite header = testSuite.overwriteRequestHeaderByTests(header, operation, operationItem, options); - if (!_.includes(IMPLICIT_HEADERS, _.toLower(_.get(header, 'name')))) { + // if (!_.includes(IMPLICIT_HEADERS, _.toLower(_.get(header, 'name')))) { item.request.addHeader(this.convertToPmHeader(header, REQUEST_TYPE.ROOT, PARAMETER_SOURCE.REQUEST, components, options, schemaCache)); - } + // } }); // adding Request Body and Content-Type header From 15b7417d74615d6f7fe9b106a302bd8bf72c0686 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Fri, 7 May 2021 22:22:37 +0200 Subject: [PATCH 45/52] Applied a patch to overcome the unwanted setting of maxItem:2 on the content schema. This messes up the schema validation used by the testSuite. Changed the fixed stackLimit from 10 to 999 to overcome "Error: Too many levels of nesting to fake this schema" in the schema validation Fix to prevent referenced path variables being used for overwrites --- lib/pmTestSuite.js | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index 1b75333e4..6c9353c8a 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -16,7 +16,7 @@ module.exports = { * @param {object} components - components defined in the OAS spec. These are used to * resolve references while generating params. * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. - * @param {object} schemaCache - object storing schemaFaker and schmeResolution caches + * @param {object} schemaCache - object storing schemaFaker and schemeResolution caches * @returns {*} array of all query params */ convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { @@ -154,8 +154,8 @@ module.exports = { try { // When processing a reference, schema.type could also be undefined - resolvedSchema = deref.resolveRefs(schemaContent, 'response', components, - schemaCache, PROCESSING_TYPE.VALIDATION); + resolvedSchema = deref.resolveRefs(_.cloneDeep(schemaContent), 'response', components, + schemaCache, 'VALIDATION', 'example', 0, {}, 999); // deletes nullable and adds "null" to type array if nullable is true let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); @@ -424,7 +424,7 @@ module.exports = { */ overwriteRequestPathByTests: function (requestPathVariables, operation, operationItem, options) { options = _.merge({}, defaultOptions, options); - let testRequestPathVariables = requestPathVariables.slice(), // clone requestPathVariables + let testRequestPathVariables = JSON.parse(JSON.stringify(requestPathVariables)), // clone requestPathVariables testSuite = {}, overwriteValues = [], testSuiteOverwriteRequest = {}, @@ -611,6 +611,12 @@ module.exports = { obj[k].type = jsonTypes; delete obj[k].nullable; } + if (obj[k].maxItems && obj[k].maxItems === 2 && obj[k].type === 'array') { + // deletes maxItems, which is added unwanted by resolveRefs() in combination with resolveFor CONVERSION + // TODO find another way to respect the maxItems that might be passed by OpenAPI + // Asked for assistance https://github.com/postmanlabs/openapi-to-postman/issues/367 + delete obj[k].maxItems; + } traverse(obj[k]); } } From 75f37ac3168f87094839d9bf7c590f1966e8cd6b Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 16 May 2021 23:25:55 +0200 Subject: [PATCH 46/52] Fixed lint errors --- lib/pmTestSuite.js | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index 6c5e2959e..873d5df75 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -174,7 +174,8 @@ module.exports = { ' pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});\n' + '});\n' ); - } catch (e) { + } + catch (e) { console.warn('JSON schema check failed', e); console.warn('invalid schemaValidation for ', content); } @@ -195,6 +196,7 @@ module.exports = { ' console.log("pm.environment - use {{' + operation.operationId + '.id}} ' + 'as variable for value", jsonData.id);\n' + '};\n'); + // eslint-disable-next-line no-console console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + operation.operationId + '.id}} as variable for "reponse.id'); assignPmVariablesCounter++; @@ -217,7 +219,7 @@ module.exports = { let responseProp = (environmentVariable.responseProp) ? environmentVariable.responseProp : environmentVariable.responseBodyProp; // Set variable name - let varName = operation.operationId + '.' + responseProp; + let varName = operation.operationId + '.' + responseProp; // eslint-disable-line one-var if (environmentVariable.name) { varName = environmentVariable.name; } @@ -233,6 +235,7 @@ module.exports = { ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + 'jsonData.' + responseProp + ');\n' + '};\n'); + // eslint-disable-next-line no-console console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + 'as variable for "reponse.' + responseProp + '"'); assignPmVariablesCounter++; @@ -280,24 +283,25 @@ module.exports = { if (environmentVariable.responseHeaderProp) { let headerProp = environmentVariable.responseHeaderProp; // Set variable name - let varName = operation.operationId + '.' + headerProp; + let varName = operation.operationId + '.' + headerProp; // eslint-disable-line one-var if (environmentVariable.name) { varName = environmentVariable.name; } // Safe variable name - let safeVarName = varName.replace(/-/g, '') + let safeVarName = varName.replace(/-/g, '') // eslint-disable-line one-var .replace(/_/g, '').replace(/ /g, '') .replace(/\./g, ''); requestsTests.push( - '// pm.environment - Set ' + varName + ' as environment variable \n' + - 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + - 'if (' + safeVarName + ' !== undefined) {\n' + - ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + - ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + - '' + safeVarName + ');\n' + + String('// pm.environment - Set ' + varName + ' as environment variable \n' + + 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + + 'if (' + safeVarName + ' !== undefined) {\n' + + ' pm.environment.set("' + varName + '",' + safeVarName + ');\n' + + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ') + + safeVarName + ');\n' + '};\n'); + // eslint-disable-next-line no-console console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + 'as variable for "header.' + headerProp + '"'); } @@ -325,7 +329,8 @@ module.exports = { try { // Extend the generated tests, with the test extension scripts requestsTests.push(postmanTest); - } catch (e) { + } + catch (e) { console.warn('invalid extendTests for ' + testExtension.openApiOperationI); } }); @@ -637,9 +642,10 @@ module.exports = { * @returns {Object} reference object from the saved components * @no-unit-tests */ - getRefObject: function($ref, components, options) { + getRefObject: function ($ref, components, options) { options = _.merge({}, defaultOptions, options); - var refObj, savedSchema; + var refObj, + savedSchema; savedSchema = $ref.split('/').slice(1).map((elem) => { // https://swagger.io/docs/specification/using-ref#escape @@ -714,7 +720,7 @@ module.exports = { * @returns {RegExp} Return a regex * @no-unit-tests */ - pathToRegExp: function(path) { + pathToRegExp: function (path) { const pattern = path // Escape literal dots .replace(/\./g, '\\.') @@ -727,7 +733,9 @@ module.exports = { // Replace wildcard with any zero-to-any character sequence .replace(/\*+/g, '.*') // Replace parameters with named capturing groups - .replace(/:([^\d|^\/][a-zA-Z0-9_]*(?=(?:\/|\\.)|$))/g, (_, paramName) => `(?<${paramName}>[^\/]+?)`) + .replace(/:([^\d|^\/][a-zA-Z0-9_]*(?=(?:\/|\\.)|$))/g, (_, paramName) => { + return `(?<${paramName}>[^\/]+?)`; + }) // Allow optional trailing slash .concat('(\\/|$)'); return new RegExp(pattern, 'gi'); @@ -739,7 +747,7 @@ module.exports = { * @param {*} url the entered URL is being evaluated for matching * @returns {Object} matching information */ - matchPath: function(path, url) { + matchPath: function (path, url) { const expression = path instanceof RegExp ? path : this.pathToRegExp(path), match = expression.exec(url) || false; // Matches in strict mode: match string should equal to input (url) From b841af0e6de86930bdbe1db92d9ce7d069d70f6f Mon Sep 17 00:00:00 2001 From: Tim <> Date: Sun, 16 May 2021 23:30:03 +0200 Subject: [PATCH 47/52] Removed unused variable --- lib/pmTestSuite.js | 5 ----- 1 file changed, 5 deletions(-) diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index 873d5df75..74434b2e4 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -2,11 +2,6 @@ const _ = require('lodash'), deref = require('./deref.js'), defaultOptions = require('../lib/options.js').getOptions('use'), APP_JSON = 'application/json', - // Specifies types of processing Refs - PROCESSING_TYPE = { - VALIDATION: 'VALIDATION', - CONVERSION: 'CONVERSION' - }, // These are the methods supported in the PathItem schema // https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#pathItemObject METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch', 'trace']; From 33a707c0a05fdb6490b001e28d76f3770c2359a5 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Wed, 19 May 2021 11:44:18 +0200 Subject: [PATCH 48/52] Added option to disable the request query param in Postman Added "ContentCheck" feature --- TESTGENERATION.md | 186 +++++++++++++++++++++++++++++++++++++++------ lib/pmTestSuite.js | 136 +++++++++++++++++++++++++++------ 2 files changed, 276 insertions(+), 46 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 8ddc8c331..399bb337d 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -77,6 +77,8 @@ The JSON test suite format consists out of 5 parts: will be generated for **all** operations). - **extendTests**: which refers the custom additions of manual created postman tests. ( see [Postman test suite extendTests](#postman-test-suite-extendtests)) +- **contentChecks**: which refers the additional Postman tests that check the content. ( + see [Postman test suite contentChecks](#postman-test-suite-contentchecks)) - **assignPmVariables**: which refers to specific Postman environment variables for easier automation. ( see [Postman test suite assignPmVariables](#postman-test-suite-assignpmvariables)) - **overwriteRequests**: which refers the custom additions/modifications of the OpenAPI request body. ( @@ -109,6 +111,132 @@ in `tests` array, will be added to the postman test scripts. - **overwrite (Boolean true/false)** : Resets all generateTests and overwrites them with the defined tests from the `tests` array. Default: false +## Postman test suite contentChecks + +Next to the generated tests, it is possible to define "content" checks where a property and the value of the response +body should exist and match a specific value or variable. + +The contentChecks are mapped based on the OpenApi operationId or the OpenApi Operation reference (method + path). +Anything added in `checkRequestBody` array, will be add as content check to the Postman tests. + +Properties explained: + +Target options: + +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be + tested. (example: `listPets`) +- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman Request + Body will be test (example: `GET::/pets`) + +These target options are both supported for defining a target. In case both are set for the same target, only +the `openApiOperationId` will be used for overwrites. + +Content check options: + +- **checkRequestBody (Array)** : Array of key/value pairs of properties & values in the Postman Request Body. + - **key (string)** : The key that will be targeted in the request body to check if it exists. + - **value (string)** : The value that will be used to check if the value in the request body matches. + +OpenAPI to Postman Testsuite Configuration: + +```json +{ + "version": 1.0, + "generateTests": { + "limitOperations": [], + "responseChecks": { + "StatusSuccess": { + "enabled": true + }, + "responseTime": { + "enabled": false, + "maxMs": 300 + }, + "headersPresent": { + "enabled": true + }, + "contentType": { + "enabled": true + }, + "jsonBody": { + "enabled": true + }, + "schemaValidation": { + "enabled": true + } + } + }, + "extendTests": [], + "contentChecks": [ + { + "openApiOperation": "GET::/contacts/{audienceId}", + "checkRequestBody": [ + { + "key": "value[0].name", + "value": "John" + }, + { + "key": "value[0].id", + "value": 1 + } + ] + } + ] +} +``` + +API Response: + +```json +{ + "value": [ + { + "id": 1, + "name": "John", + "createdDt": "2021-05-10T08:29:34.810Z" + }, + { + "id": 2, + "name": "Marco", + "createdDt": "2021-05-10T08:33:14.110Z" + } + ] +} +``` + +Part of Postman Testsuite checks: + +```javascript +// Set response object as internal variable +let jsonData = pm.response.json(); + +// Response body should have property "value[0].name" +pm.test("[GET] /contacts/{audienceId} - Content check if property 'value[0].name' exists", function () { + pm.expect((typeof jsonData.value[0].name !== "undefined")).to.be.true; +}); + +// Response body should have value "John" for "value[0].name" +if (typeof jsonData.value[0].name !== "undefined") { + pm.test("[GET] /contacts/{audienceId} - Content check if value for 'value[0].name' matches 'John'", function () { + pm.expect(jsonData.value[0].name).to.eql("John"); + }) +} +; + +// Response body should have property "value[0].id" +pm.test("[GET] /contacts/{audienceId} - Content check if property 'value[0].id' exists", function () { + pm.expect((typeof jsonData.value[0].id !== "undefined")).to.be.true; +}); + +// Response body should have value "1" for "value[0].id" +if (typeof jsonData.value[0].id !== "undefined") { + pm.test("[GET] //contacts/{audienceId} - Content check if value for 'value[0].id' matches '1'", function () { + pm.expect(jsonData.value[0].id).to.eql(1); + }) +} +; +``` + ### Postman test suite targeting for variables & overwrites It is possible to assign variables and overwrite query params, headers, request body data with values specifically for @@ -125,8 +253,8 @@ separator symbol. This will allow targeting for very specific OpenApi items. -To facilitate managing the filtering, we have included wildcard options for the `openApiOperation` option, supporting the -methods & path definitions. +To facilitate managing the filtering, we have included wildcard options for the `openApiOperation` option, supporting +the methods & path definitions. REMARK: Be sure to put quotes around the target definition. @@ -146,11 +274,11 @@ A combination of wildcards for the method and path parts are even possible. ## Postman test suite assignPmVariables -To facilitate automation, we provide the option set "pm.environment" variables with values from the response. -The assigning of the pm.environment are mapped based on the OpenApi operationId. +To facilitate automation, we provide the option set "pm.environment" variables with values from the response. The +assigning of the pm.environment are mapped based on the OpenApi operationId. -REMARK: By default the test suite will create pm.environment variable for the ID property in the response object, if -ID is present in the reponse. +REMARK: By default the test suite will create pm.environment variable for the ID property in the response object, if ID +is present in the reponse. Anything added in `assignPmVariables` array, will be used to generate specific pm.environment variables based on the postman response body. @@ -161,8 +289,8 @@ Target options: - **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman pm.environment variable will be set. (example: `listPets`) -- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman pm.environment variable - will be set. (example: `GET::/pets`) +- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman + pm.environment variable will be set. (example: `GET::/pets`) These target options are both supported for defining a target. In case both are set for the same target, only the `openApiOperationId` will be used for overwrites. @@ -172,8 +300,8 @@ EnvironmentVariables options: - **environmentVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. - **responseProp (string)** : The property for which the value will be taken in the response body and set as the pm.environment value. - - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite - the default generated variable name + - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite the + default generated variable name Example: @@ -203,11 +331,12 @@ Example: ``` This will generate the following: + - pm.environment for "post-accounts.id" - use {{post-accounts.id}} as variable for "reponse.id - pm.environment for "get-accounts" - use {{server-address}} as variable for "reponse.value[0].servers[0]" -This information on the assignment of the pm.environment will be published in the Postman Test script and during -the conversion via the CLI +This information on the assignment of the pm.environment will be published in the Postman Test script and during the +conversion via the CLI This results in the following functions on the Postman Test pane: @@ -218,15 +347,15 @@ let jsonData = pm.response.json(); // pm.environment - Set post-accounts.id as environment variable if (jsonData.id) { - pm.environment.set("post-accounts.id",jsonData.id); - console.log("pm.environment - use {{post-accounts.id}} as variable for value", jsonData.id); -}; + pm.environment.set("post-accounts.id", jsonData.id); + console.log("pm.environment - use {{post-accounts.id}} as variable for value", jsonData.id); +} // pm.environment - Set post-accounts.servers[0] as environment variable if (jsonData.value[0].servers[0]) { - pm.environment.set("server-address",jsonData.value[0].servers[0]); - console.log("pm.environment - use {{server-address}} as variable for value", jsonData.value[0].servers[0]); -}; + pm.environment.set("server-address", jsonData.value[0].servers[0]); + console.log("pm.environment - use {{server-address}} as variable for value", jsonData.value[0].servers[0]); +} ``` ## Postman test suite overwriteRequests @@ -254,8 +383,9 @@ Overwrite options: - **value (string)** : The value that will be used to overwrite/extend the value in the request Query Param OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - - **overwrite (Boolean true/false | Default: true)** : Overwrites the request query param value OR attach the value - to the original request query param value. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request query param value OR attach the value to + the original request query param value. + - **disable (Boolean true/false | Default: false)** : Disables the request query param in Postman - **remove (Boolean true/false | Default: false)** : Removes the request query param - **overwriteRequestPathVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Path Variables. @@ -263,8 +393,8 @@ Overwrite options: - **value (string)** : The value that will be used to overwrite/extend the value in the request path variable OR use the [Postman Dynamic variables](https://learning.postman.com/docs/writing-scripts/script-references/variables-list/) to use dynamic values like `{{$guid}}` or `{{$randomInt}}`. - - **overwrite (Boolean true/false | Default: true)** : Overwrites the request path variable value OR attach the value to the - original request Path variable value. + - **overwrite (Boolean true/false | Default: true)** : Overwrites the request path variable value OR attach the value + to the original request Path variable value. - **remove (Boolean true/false | Default: false)** : Removes the request path variable - **overwriteRequestHeaders (Array)** : Array of key/value pairs to overwrite in the Postman Request Headers. - **key (string)** : The key that will be targeted in the request Headers to overwrite/extend. @@ -359,6 +489,14 @@ OpenAPI to Postman Testsuite Configuration: "key": "$filter", "value": "{{$randomInt}}", "overwrite": false + }, + { + "key": "$search", + "remove": true + }, + { + "key": "$select", + "disable": true } ], "overwriteRequestHeaders": [ @@ -400,7 +538,9 @@ OpenAPI to Postman Testsuite Configuration: ``` The `overwriteRequestQueryParams` example will overwrite the "$count" query param value with the boolean "true", the -"$filter" with dynamic value for "$randomInt", for the "get-accounts" OpenAPI operationId. +"$filter" with dynamic value for "$randomInt", for the "get-accounts" OpenAPI operationId. The "$search" query parameter +will be removed, so it will not exist in the Postman collection. The "$select" query parameter will be marked as " +disabled" in the Postman collection The `overwriteRequestHeaders` example will overwrite the "team-id" header value with a "{{$randomInt}}", for the "get-accounts" OpenAPI operationId. diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index 74434b2e4..d625db0c9 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -132,7 +132,7 @@ module.exports = { if (contentType === APP_JSON) { let resolvedSchema, - assignPmVariablesCounter = 0; + pmVariablesCounter = 0; // Add JSON body check if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { @@ -176,25 +176,76 @@ module.exports = { } } + // Add content checks for JSON response + if (testSuite.contentChecks) { + let testSuiteContentChecks = testSuite.contentChecks, + // Get the checkRequestBody settings for the operationId or openApiOperation + setContentChecksForOperation = testSuiteContentChecks.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (setContentChecksForOperation && setContentChecksForOperation.checkRequestBody) { + let requestBodyChecks = setContentChecksForOperation.checkRequestBody; + requestBodyChecks.forEach((check) => { + // Only set the jsonData once + if (pmVariablesCounter === 0) { + requestsTests.push('// Set response object as internal variable\n' + + 'let jsonData = pm.response.json();\n'); + } + + if (check.key) { + requestsTests.push( + '// Response body should have property "' + check.key + '"\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content check if property \'' + check.key + '\' exists", function() {\n' + + ' pm.expect((typeof jsonData.' + check.key + ' !== "undefined")).to.be.true;\n' + + '});\n' + ); + } + + if (check.value) { + let checkValue = check.value; + if (!(typeof check.value === 'number' || typeof check.value === 'boolean')) { + checkValue = '"' + check.value + '"'; + } + requestsTests.push( + '// Response body should have value "' + check.value + '" for "' + check.key + '"\n' + + 'if (typeof jsonData.' + check.key + ' !== "undefined") {\n' + + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + + ' - Content check if value for \'' + check.key + '\' matches \'' + + check.value + '\'", function() {\n' + + ' pm.expect(jsonData.' + check.key + ').to.eql(' + checkValue + ');\n' + + '})};\n' + ); + } + pmVariablesCounter++; + }); + } + } + // Automatic set the response.id as an environment variable for chaining option if (operationItem.method.toUpperCase() === 'POST' && resolvedSchema !== undefined && resolvedSchema.properties && resolvedSchema.properties.id) { + let opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + // Only set the jsonData once - if (assignPmVariablesCounter === 0) { + if (pmVariablesCounter === 0) { requestsTests.push('// Set response object as internal variable\n' + 'let jsonData = pm.response.json();\n'); } requestsTests.push( - '// pm.environment - Set ' + operation.operationId + '.id as environment variable \n' + + '// pm.environment - Set ' + opsRef + '.id as environment variable \n' + 'if (jsonData.id) {\n' + - ' pm.environment.set("' + operation.operationId + '.id",jsonData.id);\n' + - ' console.log("pm.environment - use {{' + operation.operationId + '.id}} ' + + ' pm.environment.set("' + opsRef + '.id",jsonData.id);\n' + + ' console.log("pm.environment - use {{' + opsRef + '.id}} ' + 'as variable for value", jsonData.id);\n' + '};\n'); // eslint-disable-next-line no-console - console.log('- pm.environment for "' + operation.operationId + '.id" - use {{' + - operation.operationId + '.id}} as variable for "reponse.id'); - assignPmVariablesCounter++; + console.log('- pm.environment for "' + opsRef + '.id" - use {{' + + opsRef + '.id}} as variable for "reponse.id'); + pmVariablesCounter++; } // Assign defined PmVariables for JSON response @@ -212,14 +263,19 @@ module.exports = { // Set response body property as variable if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { let responseProp = (environmentVariable.responseProp) ? - environmentVariable.responseProp : environmentVariable.responseBodyProp; + environmentVariable.responseProp : + environmentVariable.responseBodyProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + // Set variable name - let varName = operation.operationId + '.' + responseProp; // eslint-disable-line one-var + let varName = opsRef + '.' + responseProp; // eslint-disable-line one-var if (environmentVariable.name) { varName = environmentVariable.name; } // Only set the jsonData once - if (assignPmVariablesCounter === 0) { + if (pmVariablesCounter === 0) { requestsTests.push('// Set response object as internal variable\n' + 'let jsonData = pm.response.json();\n'); } @@ -231,9 +287,9 @@ module.exports = { 'jsonData.' + responseProp + ');\n' + '};\n'); // eslint-disable-next-line no-console - console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + 'as variable for "reponse.' + responseProp + '"'); - assignPmVariablesCounter++; + pmVariablesCounter++; } }); } @@ -276,9 +332,13 @@ module.exports = { assignEnvironmentVariables.forEach((environmentVariable) => { // Set response header property as variable if (environmentVariable.responseHeaderProp) { - let headerProp = environmentVariable.responseHeaderProp; + let headerProp = environmentVariable.responseHeaderProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + // Set variable name - let varName = operation.operationId + '.' + headerProp; // eslint-disable-line one-var + let varName = opsRef + '.' + headerProp; // eslint-disable-line one-var if (environmentVariable.name) { varName = environmentVariable.name; } @@ -297,7 +357,7 @@ module.exports = { safeVarName + ');\n' + '};\n'); // eslint-disable-next-line no-console - console.log('- pm.environment for "' + operation.operationId + '" - use {{' + varName + '}} ' + + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + 'as variable for "header.' + headerProp + '"'); } @@ -515,16 +575,25 @@ module.exports = { // Overwrite value for query param key overwriteValues.forEach((overwriteValue) => { - if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key && - overwriteValue.hasOwnProperty('value') && testRequestQueryParam.hasOwnProperty('value')) { - let orgValue = testRequestQueryParam.value, - newValue = overwriteValue.value; + if (overwriteValue.key && testRequestQueryParam.key && overwriteValue.key === testRequestQueryParam.key) { - if (overwriteValue.overwrite === false) { - newValue = orgValue + newValue; + // Test suite - Overwrite/extend query param value + if (overwriteValue.hasOwnProperty('value') && testRequestQueryParam.hasOwnProperty('value')) { + let orgValue = testRequestQueryParam.value, + newValue = overwriteValue.value; + + if (overwriteValue.overwrite === false) { + newValue = orgValue + newValue; + } + testRequestQueryParam.value = newValue; + } + + // Test suite - Disable query param + if (overwriteValue.disable === true) { + testRequestQueryParam.disabled = true; } - testRequestQueryParam.value = newValue; + // Test suite - Remove query param if (overwriteValue.remove === true) { testRequestQueryParam = {}; } @@ -708,6 +777,27 @@ module.exports = { return false; }, + /** + * Converts combination of the OpenApi method & path to a uniform OpenApiOperationRef. + * @param {object} operationItem the OpenApi operation item to match + * @param {string} format the format of OpenApi Operation reference (MATCH or PMVAR) + * @returns {string} Return a OpenApiOperation reference + * @no-unit-tests + */ + getOpenApiOperationRef: function (operationItem, format) { + const operationFormat = format || 'MATCH'; + if (operationItem.method && operationItem.path) { + if (operationFormat === 'MATCH') { + return operationItem.method.toUpperCase() + '::/' + operationItem.path; + } + if (operationFormat === 'PMVAR') { + return operationItem.method.toLowerCase() + '-' + operationItem.path.replace('{', ':') + .replace('}', '').replace(/#|\//g, '-'); + } + } + return ''; + }, + /** * Converts a string path to a Regular Expression. * Transforms path parameters into named RegExp groups. From d8d10a17cf3bb67792267f91b48b3e8be6f94d8b Mon Sep 17 00:00:00 2001 From: Tim <> Date: Mon, 24 May 2021 11:05:42 +0200 Subject: [PATCH 49/52] Extended the test suite assignPmVariables function by response header, request body or a plain value --- TESTGENERATION.md | 17 ++--- lib/pmTestSuite.js | 156 +++++++++++++++++++++++++++++++++++---------- lib/schemaUtils.js | 17 ++++- 3 files changed, 146 insertions(+), 44 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 399bb337d..b3e38ba5e 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -277,8 +277,8 @@ A combination of wildcards for the method and path parts are even possible. To facilitate automation, we provide the option set "pm.environment" variables with values from the response. The assigning of the pm.environment are mapped based on the OpenApi operationId. -REMARK: By default the test suite will create pm.environment variable for the ID property in the response object, if ID -is present in the reponse. +REMARK: By default the test suite will create a pm.environment variable for the ID property in the response object, if ID +is present in the response. Anything added in `assignPmVariables` array, will be used to generate specific pm.environment variables based on the postman response body. @@ -297,11 +297,12 @@ the `openApiOperationId` will be used for overwrites. EnvironmentVariables options: -- **environmentVariables (Array)** : Array of key/value pairs to overwrite in the Postman Request Query params. - - **responseProp (string)** : The property for which the value will be taken in the response body and set as the - pm.environment value. - - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite the - default generated variable name +- **environmentVariables (Array)** : Array of key/value pairs to set the Postman variables. + - **responseBodyProp (string)** : The property for which the value will be taken in the response body and set the value as the pm.environment value. + - **responseHeaderProp (string)** : The property for which the value will be taken in the response header and set the value as the pm.environment value. + - **requestBodyProp (string)** : The property for which the value will be taken in the request body and set the value as the pm.environment value. + - **value (string)** : The defined value that will be set as the pm.environment value. + - **name (string OPTIONAL | Default: openApiOperationId.responseProp)** : The name that will be used to overwrite the default generated variable name Example: @@ -336,7 +337,7 @@ This will generate the following: - pm.environment for "get-accounts" - use {{server-address}} as variable for "reponse.value[0].servers[0]" This information on the assignment of the pm.environment will be published in the Postman Test script and during the -conversion via the CLI +conversion via the CLI. This results in the following functions on the Postman Test pane: diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index d625db0c9..9dbab4b8a 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -16,21 +16,20 @@ module.exports = { * resolve references while generating params. * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. * @param {object} schemaCache - object storing schemaFaker and schemeResolution caches - * @returns {*} array of all query params + * @returns {*} array of all tests */ convertResponsesToPmTest: function (operation, operationItem, components, options, schemaCache) { options = _.merge({}, defaultOptions, options); - let testEvent = {}, - testSuite = {}, + let testSuite = {}, testSuiteSettings = {}, testSuiteLimits = [], testSuiteExtensions = [], - requestsTests = [], + pmTests = [], swagResponse = {}; // Check for test suite flag, abort early if (!options.testSuite) { - return testEvent; + return pmTests; } // Check test suite for later usage, potentially convert object @@ -85,7 +84,7 @@ module.exports = { // Add status code check // TODO Validate other request codes // if (operationItem.method.toUpperCase() === 'GET') { - // requestsTests.push( + // pmTests.push( // '// Validate status code \n'+ // 'pm.test("Status code should be '+code+'", function () {\n' + // ' pm.response.to.have.status('+code+');\n' + @@ -95,7 +94,7 @@ module.exports = { // Add status success check if (_.get(testSuiteSettings, 'responseChecks.StatusSuccess.enabled')) { - requestsTests.push( + pmTests.push( '// Validate status 2xx \n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Status code is 2xx", function () {\n' + @@ -107,7 +106,7 @@ module.exports = { // Add response timer check if (_.get(testSuiteSettings, 'responseChecks.responseTime.enabled')) { let maxMs = _.get(testSuiteSettings, 'responseChecks.responseTime.maxMs'); - requestsTests.push( + pmTests.push( '// Validate response time \n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Response time is less than ' + maxMs + 'ms", function () {\n' + @@ -121,7 +120,7 @@ module.exports = { if (contentType) { // Add content-type check if (_.get(testSuiteSettings, 'responseChecks.contentType.enabled')) { - requestsTests.push( + pmTests.push( '// Validate content-type \n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Content-Type is ' + contentType + '", function () {\n' + @@ -136,7 +135,7 @@ module.exports = { // Add JSON body check if (_.get(testSuiteSettings, 'responseChecks.jsonBody.enabled')) { - requestsTests.push( + pmTests.push( '// Response should have JSON Body\n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Response has JSON Body", function () {\n' + @@ -159,7 +158,7 @@ module.exports = { // deletes nullable and adds "null" to type array if nullable is true let jsonSchema = this.convertUnsupportedJsonSchemaProperties(resolvedSchema); - requestsTests.push( + pmTests.push( '// Response Validation\n' + 'const schema = ' + JSON.stringify(jsonSchema) + '\n' + '\n' + @@ -189,12 +188,13 @@ module.exports = { requestBodyChecks.forEach((check) => { // Only set the jsonData once if (pmVariablesCounter === 0) { - requestsTests.push('// Set response object as internal variable\n' + + pmTests.push( + '// Set response object as internal variable\n' + 'let jsonData = pm.response.json();\n'); } if (check.key) { - requestsTests.push( + pmTests.push( '// Response body should have property "' + check.key + '"\n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Content check if property \'' + check.key + '\' exists", function() {\n' + @@ -208,7 +208,11 @@ module.exports = { if (!(typeof check.value === 'number' || typeof check.value === 'boolean')) { checkValue = '"' + check.value + '"'; } - requestsTests.push( + if (check.value.includes('{{') && check.value.includes('}}')) { + checkValue = 'pm.environment.get("' + check.value.replace(/{{|}}/g,'') + '")'; + } + + pmTests.push( '// Response body should have value "' + check.value + '" for "' + check.key + '"\n' + 'if (typeof jsonData.' + check.key + ' !== "undefined") {\n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + @@ -232,12 +236,13 @@ module.exports = { // Only set the jsonData once if (pmVariablesCounter === 0) { - requestsTests.push('// Set response object as internal variable\n' + + pmTests.push( + '// Set response object as internal variable\n' + 'let jsonData = pm.response.json();\n'); } - requestsTests.push( + pmTests.push( '// pm.environment - Set ' + opsRef + '.id as environment variable \n' + - 'if (jsonData.id) {\n' + + 'if (typeof jsonData.id !== "undefined") {\n' + ' pm.environment.set("' + opsRef + '.id",jsonData.id);\n' + ' console.log("pm.environment - use {{' + opsRef + '.id}} ' + 'as variable for value", jsonData.id);\n' + @@ -260,6 +265,27 @@ module.exports = { let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; assignEnvironmentVariables.forEach((environmentVariable) => { + let i = 0; + // Set value as variable + if (environmentVariable.value) { + // Set variable name + let opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + let varName = opsRef + '.var-' + i; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + pmTests.push( + '// pm.environment - Set a value for ' + varName + '\n' + + 'pm.environment.set("' + varName + '",' + environmentVariable.value + ')\n' + ); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "' + environmentVariable.value + '"'); + i++; + } + // Set response body property as variable if (environmentVariable.responseProp || environmentVariable.responseBodyProp) { let responseProp = (environmentVariable.responseProp) ? @@ -276,12 +302,13 @@ module.exports = { } // Only set the jsonData once if (pmVariablesCounter === 0) { - requestsTests.push('// Set response object as internal variable\n' + + pmTests.push( + '// Set response object as internal variable\n' + 'let jsonData = pm.response.json();\n'); } - requestsTests.push( - '// pm.environment - Set ' + varName + ' as environment variable \n' + - 'if (jsonData.' + responseProp + ') {\n' + + pmTests.push( + '// pm.environment - Set ' + varName + ' as variable for jsonData.' + responseProp + ' \n' + + 'if (typeof jsonData.' + responseProp + ' !== "undefined") {\n' + ' pm.environment.set("' + varName + '",jsonData.' + responseProp + ');\n' + ' console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + 'jsonData.' + responseProp + ');\n' + @@ -306,7 +333,7 @@ module.exports = { if (headerKey && header.name) { // Add content-type check if (_.get(testSuiteSettings, 'responseChecks.headersPresent.enabled')) { - requestsTests.push( + pmTests.push( '// Validate header \n' + 'pm.test("[' + operationItem.method.toUpperCase() + '] /' + operationItem.path + ' - Response header ' + header.name + ' is present", function () {\n' + @@ -348,7 +375,7 @@ module.exports = { .replace(/_/g, '').replace(/ /g, '') .replace(/\./g, ''); - requestsTests.push( + pmTests.push( String('// pm.environment - Set ' + varName + ' as environment variable \n' + 'let ' + safeVarName + ' = pm.response.headers.get("' + headerProp + '"); \n' + 'if (' + safeVarName + ' !== undefined) {\n' + @@ -375,7 +402,7 @@ module.exports = { if (testExtension && testExtension.overwrite && testExtension.overwrite === true) { // Reset generated tests - requestsTests = []; + pmTests = []; } // Add test extensions @@ -383,7 +410,7 @@ module.exports = { testExtension.tests.forEach((postmanTest) => { try { // Extend the generated tests, with the test extension scripts - requestsTests.push(postmanTest); + pmTests.push(postmanTest); } catch (e) { console.warn('invalid extendTests for ' + testExtension.openApiOperationI); @@ -394,14 +421,7 @@ module.exports = { }); } - // Add tests to postman item - if (requestsTests.length > 0) { - testEvent = { - listen: 'test', - script: requestsTests - }; - } - return testEvent; + return pmTests; }, /** @@ -660,6 +680,76 @@ module.exports = { return testRequestHeader; }, + /** + * function to assign PM variables with values defined by the request body from OpenApi + * @param {*} requestBody request Body object + * @param {*} operation openapi operation object + * @param {*} operationItem - The operation item to get relevant information for the test generation + * @param {object} options - a standard list of options that's globally passed around. Check options.js for more. + * @returns {*} Postman Test scripts array + */ + assignVariablesByRequestBody: function (requestBody, operation, operationItem, options) { + options = _.merge({}, defaultOptions, options); + let testSuite = {}, + pmTests = []; + + // Check for test suite flag and if there is RAW requestBody, abort early + if (!options.testSuite && !requestBody.raw) { + return requestBody; + } + + if (options.testSuite && options.testSuiteSettings) { + testSuite = options.testSuiteSettings; + } + if (testSuite.assignPmVariables) { + let testSuiteAssignPmVariables = testSuite.assignPmVariables, + // Get the environmentVariables settings for the operationId or openApiOperation + assignPmVariablesForOperation = testSuiteAssignPmVariables.find((or) => { + return this.isMatchOperationItem(operationItem, or); + }); + + if (assignPmVariablesForOperation && assignPmVariablesForOperation.environmentVariables) { + let assignEnvironmentVariables = assignPmVariablesForOperation.environmentVariables; + + assignEnvironmentVariables.forEach((environmentVariable) => { + // Set request body property as variable + if (environmentVariable.requestBodyProp) { + let responseProp = environmentVariable.requestBodyProp, + opsRef = (operation.operationId) ? + operation.operationId : + this.getOpenApiOperationRef(operationItem, 'PMVAR'); + + // Set variable name + let varName = opsRef + '.' + responseProp; // eslint-disable-line one-var + if (environmentVariable.name) { + varName = environmentVariable.name; + } + + // Set variable value + const requestBodyObj = JSON.parse(requestBody.raw); + if (typeof requestBodyObj[responseProp] !== 'undefined') { + let requestBodyValue = requestBodyObj[responseProp]; + if (!(typeof requestBodyValue === 'number' || typeof requestBodyValue === 'boolean')) { + requestBodyValue = '"' + requestBodyValue + '"'; + } + pmTests.push( + '// pm.environment - Set ' + varName + ' as environment variable from request body \n' + + 'pm.environment.set("' + varName + '",' + requestBodyValue + ');\n' + + 'console.log("pm.environment - use {{' + varName + '}} as variable for value", ' + + requestBodyValue + ');\n' + ); + // eslint-disable-next-line no-console + console.log('- pm.environment for "' + opsRef + '" - use {{' + varName + '}} ' + + 'as variable for "request.' + responseProp + '"'); + } + } + }); + } + } + + return pmTests; + }, + /** * function to convert unsupported OpenAPI(3.0) properties to valid JSON schema properties * @param {*} oaSchema openAPI schema diff --git a/lib/schemaUtils.js b/lib/schemaUtils.js index a9fb3fc57..732a57e4d 100644 --- a/lib/schemaUtils.js +++ b/lib/schemaUtils.js @@ -2119,7 +2119,8 @@ module.exports = { displayUrl, reqUrl = '/' + operationItem.path, pmBody, - pmTests, + pmTests = [], + pmTestVars = [], authMeta, swagResponse, localServers = _.get(operationItem, 'properties.servers'), @@ -2330,6 +2331,9 @@ module.exports = { pmBody = this.convertToPmBody(reqBody, REQUEST_TYPE.ROOT, components, options, schemaCache); // pmTestSuite - Modify the request body for the test suite pmBody.body = testSuite.overwriteRequestBodyByTests(pmBody.body, operation, operationItem, options); + // pmTestSuite - Assign PM vars based on the request body for the test suite + pmTestVars = testSuite.assignVariablesByRequestBody(pmBody.body, operation, operationItem, options); + item.request.body = pmBody.body; item.request.addHeader(pmBody.contentHeader); // extra form headers if encoding is present in request Body. @@ -2414,9 +2418,16 @@ module.exports = { // pmTestSuite - Generate tests for the response pmTests = testSuite.convertResponsesToPmTest(operation, operationItem, components, options, schemaCache); - if (!_.isEmpty(pmTests)) { + // Merge pmTests array + const pmCombinedTests = [...pmTests, ...pmTestVars]; + + // Add tests to postman item + if (pmTests.length > 0) { // Add test event - item.events.add(pmTests); + item.events.add({ + listen: 'test', + script: pmCombinedTests + }); } return item; From cff6d7157b3e5e827e177daae9acbef47e47b7d6 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 25 May 2021 15:51:13 +0200 Subject: [PATCH 50/52] Corrected the incorrect checkRequestBody variable definition to checkResponseBody --- TESTGENERATION.md | 10 +++++----- lib/pmTestSuite.js | 8 ++++---- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index b3e38ba5e..0742c1dc3 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -117,7 +117,7 @@ Next to the generated tests, it is possible to define "content" checks where a p body should exist and match a specific value or variable. The contentChecks are mapped based on the OpenApi operationId or the OpenApi Operation reference (method + path). -Anything added in `checkRequestBody` array, will be add as content check to the Postman tests. +Anything added in `checkResponseBody` array, will be add as content check to the Postman tests. Properties explained: @@ -133,9 +133,9 @@ the `openApiOperationId` will be used for overwrites. Content check options: -- **checkRequestBody (Array)** : Array of key/value pairs of properties & values in the Postman Request Body. - - **key (string)** : The key that will be targeted in the request body to check if it exists. - - **value (string)** : The value that will be used to check if the value in the request body matches. +- **checkResponseBody (Array)** : Array of key/value pairs of properties & values in the Postman Response Body. + - **key (string)** : The key that will be targeted in the response body to check if it exists. + - **value (string)** : The value that will be used to check if the value in the response body matches. OpenAPI to Postman Testsuite Configuration: @@ -170,7 +170,7 @@ OpenAPI to Postman Testsuite Configuration: "contentChecks": [ { "openApiOperation": "GET::/contacts/{audienceId}", - "checkRequestBody": [ + "checkResponseBody": [ { "key": "value[0].name", "value": "John" diff --git a/lib/pmTestSuite.js b/lib/pmTestSuite.js index 9dbab4b8a..19f82ec03 100644 --- a/lib/pmTestSuite.js +++ b/lib/pmTestSuite.js @@ -178,14 +178,14 @@ module.exports = { // Add content checks for JSON response if (testSuite.contentChecks) { let testSuiteContentChecks = testSuite.contentChecks, - // Get the checkRequestBody settings for the operationId or openApiOperation + // Get the checkResponseBody settings for the operationId or openApiOperation setContentChecksForOperation = testSuiteContentChecks.find((or) => { return this.isMatchOperationItem(operationItem, or); }); - if (setContentChecksForOperation && setContentChecksForOperation.checkRequestBody) { - let requestBodyChecks = setContentChecksForOperation.checkRequestBody; - requestBodyChecks.forEach((check) => { + if (setContentChecksForOperation && setContentChecksForOperation.checkResponseBody) { + let responseBodyChecks = setContentChecksForOperation.checkResponseBody; + responseBodyChecks.forEach((check) => { // Only set the jsonData once if (pmVariablesCounter === 0) { pmTests.push( From b7c487741d47c9fe2412f9a8508997cac2a9c185 Mon Sep 17 00:00:00 2001 From: Tim <> Date: Tue, 25 May 2021 16:13:41 +0200 Subject: [PATCH 51/52] Corrected the incorrect checkRequestBody variable definition to checkResponseBody --- TESTGENERATION.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/TESTGENERATION.md b/TESTGENERATION.md index 0742c1dc3..ef6260283 100644 --- a/TESTGENERATION.md +++ b/TESTGENERATION.md @@ -117,19 +117,19 @@ Next to the generated tests, it is possible to define "content" checks where a p body should exist and match a specific value or variable. The contentChecks are mapped based on the OpenApi operationId or the OpenApi Operation reference (method + path). -Anything added in `checkResponseBody` array, will be add as content check to the Postman tests. +Anything added in `checkResponseBody` array, will be added as content check to the Postman tests. Properties explained: Target options: -- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Request Body will be +- **openApiOperationId (String)** : Reference to the OpenApi operationId for which the Postman Response Body will be tested. (example: `listPets`) -- **openApiOperation (String)** : Reference to combination of the OpenApi method & path, for which the Postman Request - Body will be test (example: `GET::/pets`) +- **openApiOperation (String)** : Reference to the combination of the OpenApi method & path, for which the Postman Response + Body will be tested (example: `GET::/pets`) These target options are both supported for defining a target. In case both are set for the same target, only -the `openApiOperationId` will be used for overwrites. +the `openApiOperationId` will be used for content checks. Content check options: From 14b8ff917c83597b82446b93f1c9b15b9397706f Mon Sep 17 00:00:00 2001 From: thim81 Date: Wed, 13 Apr 2022 16:28:58 +0200 Subject: [PATCH 52/52] Update README.md --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 7f8964b08..afda85bab 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,7 @@ +# THIS BRANCH IS DEPRECATED, IN FAVOR OF A STANDALONE PACKAGE [Portman](https://github.com/apideck-libraries/portman) + --- + ![postman icon](https://raw.githubusercontent.com/postmanlabs/postmanlabs.github.io/develop/global-artefacts/postman-logo%2Btext-320x132.png) *Supercharge your API workflow.*