Automated questionnaire testing

One of our objectives at BOTdesign is to collect and classify patient data in order to provide healthcare professionals with a clear and efficient account of the patient’s state of health.

This information gathering takes the form of conversational robots, which ask patients questions and collect their answers. Depending on the answers, the robots will refine certain topics and lighten others. In the end, the number of possible combinations is exponential. Testing all these combinatorics each time an element of a questionnaire or the execution engine evolves quickly becomes a tedious and time-consuming task.

Requirement analysis

The tool or process we are going to implement must meet the following requirements:
  • Provide proof As a medical device, every time we upgrade the MAX solution, we need to keep “proof” of the tests we have carried out. In most cases, these are screenshots, so that in the event of a quality audit, we can present the tests carried out for each version change.
  • Can be written by non-developers At BOTdesign, it’s the business team that sets the parameters for the robots. They must remain autonomous when it comes to writing tests corresponding to the settings.
  • Automatic or manual execution Tests must be launched automatically when a new version of a medical questionnaire is deployed, but also manually if you want to replay all the scenarios.
  • Maintainable The tool or process must remain maintainable. This means requiring less development time than the elements it tests. We’ve all experienced front-end component testing, which requires more time to redo the test than it took to build the functionality.
With all these needs in mind, our choice fell on the Playwright tool.

Setting up the tool

We heard about Playwright at this year’s Devoxx conference, where we were present for the first time.

The tool seemed to fit our need and after some research, it integrates very well with Cucumber which would then address the issue of accessibility to non-developers.

Initial tests and validation of feasibility

After a few days of exploration and testing, Raphael Pech , our Fullstack intern, was able to automatically run a bot questionnaire. The demonstration of the tool and what Raphael Pech was able to achieve in just a few days convinced us. Playwright is easy to use, can generate screenshots of the application in progress, and is easily automated. Another interesting feature is that it can run tests in different client configurations, for example with Firefox, Chrome or Safari, or with mobile resolutions to ensure that all elements appear correctly on the screen.

Integration with Cucumber

Cucumber is a tool for writing test scenarios in a way that is intelligible to humans. In other words, in a way that is not possible with a programming language. This will enable our business team to describe test scenarios for bot questionnaires without any programming knowledge.

A test scenario is called a specification; it is represented by a .feature file that will contain features which in turn contain the test steps. A specification file looks like this: 
				
					# language: fr
Fonctionnalité: 'Test du questionnaire Oncogériatrie'
Test du déroulement d'un questionnaire Oncogériatrie sur MAX

@Notice
Scénario: oncogeriatrie
    Soit  je me connecte à MAX en tant que 'patient'
    Et    je choisis le questionnaire "Oncogériatrie"
    Quand je démarre le questionnaire avec le bouton "Cliquez ici"
    Et    je réponds "65"
    Et    je réponds "157"
    Et    je clique sur le choix "Oui"
    Et    je clique sur le choix "Oui"
    Et    je clique sur le choix "Non"
    Et    je clique sur le choix "Non"
    Et    je clique sur le choix exact "Oui - Continuer"
    Et    je clique sur le choix "Non"
    Et    je clique sur le choix "Oui"
    Et    je clique sur le choix "Moyen"
    Et    je clique sur le choix "Non"
    Et    je passe la question
    Et    je clique sur le choix "Le patient lui-même"
    Et    je clique sur le choix "Oui - Continuer"
    Et    je clique sur le choix "8"
    Et    je clique sur le choix "Terminer"
    Alors le questionnaire est terminé

We start by declaring the language in which the specification is written (English is also possible). Then the name, spec description and scenario name are made explicit; in this case, an Oncogeriatrics questionnaire.

Then come all the test steps. Each step is preceded by a keyword:

  • Either, to describe the input data
  • When, to describe the actions to be taken for the test
  • Then, for the expected result

It’s easy to read and understand, as well as to produce.

On the technical side, you need to write functions corresponding to each possible action which, thanks to Playwright, will execute the action on the HEKO interface.

Automatisation

To automate the process, a Gitlab pipeline is created in the project to run the tests. Each time a modification is made to this project, the tests are replayed. Similarly, the deployment pipeline for our validation environment has been modified, and every time a deployment of the validation environment is made, it launches the scenario test pipeline. This ensures that any changes made in production will not break our questionnaire configurations.
What’s more, Gitlab pipelines store test results, lists of screenshots and details of actions taken. This enables us to understand what happened when a test failed, and guarantees maximum traceability of our tests.

Conclusion

In the end, the project is currently running on our qualification environments, and we plan to include it on our production environments very soon. It will have taken 120 hours of work, including skills upgrading on Playwright and Cucumber, to complete this project.

Business users will no longer have to scroll through the questionnaires by hand during parameterization or after each application modification.

Test reports are archived with screenshots.

The most complex questionnaires now take only 45s to unroll, compared with more than 5 minutes for a job that knows the questionnaire.

Mission accomplished!

Editors : Frédéric Combes et Raphaël Pech

Discover more from BOTdesign

Subscribe now to keep reading and get access to the full archive.

Continue reading