Skip to content

Testing

About the testing, PyFake-API-Server-Surveillance has some prerequisites or rules of it. PyFake-API-Server-Surveillance has 4 types tests: unit test, integration test, system test and end-to-end test.

  • Unit test:

in directory test/unit_test

  • Integration test:

in directory test/integration_test

  • System test:

in directory test/system_test

  • End-to-End test:

in configuration file .github/workflows/action-e2e-test.yaml

The end-to-end test to simulate the usages of really scenarios for the customized action.

Requirements for testing

PyFake-API-Server-Surveillance uses test framework PyTest. So please make sure its tool pytest could work finely in your current runtime environment for development.

poetry run pytest --version

pytest --version

Apart from PyTest, PyFake-API-Server-Surveillance also use another dependencies and some of them is PyTest plugin.

  • coverage

    For recording, calculating the test coverage of source code and generating report about it.

  • pytest-cov

    Let PyTest supports all feature of coverage.

  • pytest-rerunfailures

    Let PyTest supports re-run feature.

Please refer to the configuration pyproject.toml if you need.

Run test

It has 2 ways to run test when you develop PyFake-API-Server-Surveillance.

Specific test module

If you're developing one independent feature and only want to run a single module test, you just need to run pytest with the file path.

poetry run pytest <.py file path>

pytest <.py file path>

If you just want to one specific test item, you could use option -k.

poetry run pytest <.py file path> -k 'test_item'

pytest <.py file path> -k 'test_item'

Run test more efficiency

In default from the PyTest configuration of PyFake-API-Server-Surveillance, it would re-run the test 3 times if it get fail at running the test. For being convenience at developing, you could set value as 0 at option --reruns.

pytest <.py file path> --reruns 0

All the one specific test type tests

After you finish development of one or more features, it suggests you to run all tests to guarantee that you don't break up the code. If you want to run all tests, you would need to use shell script scripts/run_all_tests.sh to reach it. The shell script accept one argument --- test type. You have 3 types value could use:

  • all-test

    Run entire all tests under directory test.

  • unit-test

    Run all unit tests and the test modules are in directory test/unit_test.

  • integration-test

    Run all integration tests and the test modules are in directory test/integration_test.

  • system-test

    Run all system tests and the test modules are in directory test/system_test.

And how about for end-to-end test ... ?

The end-to-end test in PyFake-API-Server is only for customzied action recently. It must push the code to remote GitHub repository to verify its feature. We don't consider to test it in local side currently.

poetry run bash ./scripts/run_all_tests.sh <test type>

bash ./scripts/run_all_tests.sh <test type>

How to add test sub-package?

It would auto-detect the test sub-packages and test modules under test directory. So we don't do anything and just add the new test sub-package or test module directly.

You will see the running result includes the new path of all test modules which are in new test sub-package.

>>> bash ./scripts/ci/<the shell script what you modify> 'unix'

Argument of shell script

It has an argument of the shell script about what OS the current runtime environment is in.

  • unix

    For Unix OS or MacOS.

  • windows

    For Windows OS.