Depending on the frequency of modification of its EDI platform, its criticality and its type, it may become necessary to provide tools for the management, execution and reporting of its tests. Indeed, efficiency limits are quickly reached with spreadsheets and EDI platform consoles. Let’s see in this article what types of tools are available to manage its tests, run them and produce test campaign reports.
Testing your EDI platform requires creating and maintaining test cases. These tests will generally have associated attributes: name, description, data files, links to functional documentation etc. In order to manage this data correctly, it may be necessary to use tools, but the needs will be different according to the types of tests.
Unit Tests (UT) are managed directly in the code. The data and descriptions of the tests should be next to the code, and live with it. Test management is therefore done with code management, and does not require any particular tool, except a possible test framework in the programming language of the platform.
Platform tests (PT) focus on regression tests. They are therefore much more stable than UTs. It is necessary to manage the test data (data to be sent, reference data to be compared with what is transformed), the test scripts (sequence of commands to be executed) and the configurations for the tests (platforms under test settings, connection credentials…)
When you have few test cases and run them very occasionaly, their management can be done with files in directories and a spreadsheet. This solution will quickly reach its limits if you have to manage a history, modify and run the tests regularly.
Most often, it is necessary to use more powerful tools for test management, both to store test data and to describe test cases: test frameworks or commercial solutions can be used to describe the execution of the tests.
Regarding performance and load tests, you can rely on open-source bricks such as JMeter and Gatling, but you will still have to put in place the necessary infrastructure to store the test data, which can be very important.
End-to-end testing is based on business cases. They have the same tooling needs as platform tests to manage configurations, data, and test scripts. The difference with PTs is that some test actions will be performed on other IS components, such as ERP or CRM, but this does not fundamentally change the test structure and the same types of management tools may be used. So we can rely on the same tools as PTs.
As with test management, the tooling requirements differ according to the test categories:
They are executed by the EDI platform during the “construction” (or compilation) of a new version of the platform. No particular tool is needed, except a test framework in the platform’s programming language.
Whether for functional or non-functional tests, they must run by taking into account platform network connections. It is therefore necessary to select tools that support the protocols used by the different partners: FTP, SFTP, HTTP, Email, etc. It is possible to use open-source frameworks like Cucumber or Robot Framework, but in this case it is necessary to develop the different types of connection based on other open-source components. Extra developments must be done to efficiently validate results of EDI translationv in order to take into account the diversity of the potential file formats (iDoc, HL7, XML, EDIFACT, etc.).
It is also possible to hijack the use of JMeter to test these connections, but this component is not intended for functional validation. However, it is perfectly suited for performance and load tests.
Finally, there are commercial solutions, such as Spindev, that combine the management of different types of connection, as well as native support for different types of files.
These tests have the objective of verifying the overall validity of a particular business requirement, so it might be tempting to use the end-of-line tools GUI for certain steps. For example, one might want to check for an update in an ERP using its web interface. While this may be appropriate for a small number of tests, this method will quickly be too slow on larger volumes of tests. It can be tempting to automate tests via the graphical interface, it is to take the risk of having tests that will be fragile because dependent on an interface that is brought to change regularly. It is good practice to run these tests with the same tools as Platform Tests.
It is important to be able to track test results over time, primarily for platform testing and end-to-end testing.
Unit tests are intended to live with the code, they are modified at the same time, and must always be error free before a possible deployment of changes in the code. The need for reporting and tracking is therefore extremely limited, and can be covered by unit test frameworks.
The results of PT and E2E tests are intended to be exchanged between teams: for example between QA or development teams and business teams. This may also be the case between an EDI provider and its customer.
In the case where the number of flows and changes is small, a spreadsheet can be enough. On the other hand, as the number of tests and updates increases, reporting requirements tends to be greater. For example, it will be useful to follow the history of a particular test or subset of tests.
It may also be necessary to integrate these test results into more global tracking solutions as well as into the company’s ticket management tools. In this case, it is essential to use tools that are part of the company’s software factory. Spindev offers access to its platform via a REST API, allowing its integration with the classic tools of development teams.
Now that we have described the different types of tools needed for the testing of EDI projects, in a next blog post we will study various parameters to consider to choose the most EDI testing suitable solution.