Open Banking: Stress the System before the System Stresses You!
How to run stress testing in the absence of market TPPs
Every bank’s IT system is stress tested before it goes live. Under open banking, the regulator specifically requires detailed results and market testing with TPPs. This article covers our approach to complete stress testing in the absence of relevant TPPs.
In our previous article we looked at what the equivalence criteria for open banking mean. Beyond equivalence, the regulator wants to ensure a stable open banking network under all market conditions, particularly in situations where there are high volumes of AIS and PIS requests. At first sight this would seem to be a fairly simple task, as the ability of any system to withstand a heavy load should be tested before it goes live. Banks are used to technical performance and stress testing these days, are they not? Yet in today’s open banking world, this is easier said than done. Why? Because there is no such product as open banking yet. The market lacks the experience of any future use. The regulation leaves quite some leeway in terms of how stress testing is actually carried out, requiring only that market participants prove they are ready. But ready for what? Let us take a step back.
What is stress testing?
Stress testing consists of four different elements, as shown in Figure 1 and listed in the EBA Guideline Article 4. First, a bank must test whether it can handle multiple TPPs accessing its systems simultaneously. It then must be capable of handling a high number of requests from TPPs within a short timeframe at all endpoints. Simultaneously, it must handle those concurrent open sessions over a longer period. Last but not least, it must be capable of handling requests for large volumes of data. All this is basically nothing new for any bank, as existing systems have to fulfill the same criteria.
It would be in the interests of all market participants to test open banking functionalities to ensure that the whole open banking ecosystem, and the investment all participants have made in it, pays off in the form of better and more services for the end client. It is in the nature of the regulation that banks have been forced to enhance their IT systems to support open banking. As laid out in the first article of our mini-series, the open banking market consists of ASPSPs (banks), their clients, and TPPs. These TPPs are typically relatively new fintech companies, i.e. new market participants offering services to clients. New market participants usually start by focusing their services on specific areas, initially leaving out other, more complex services. This leads to an open banking market where only a few market participants on the TPP side have production-ready solutions that allow ASPSPs to test under realistic circumstances. Not only that, but TPPs’ solutions tend to cover only a narrow scope of the mandated open banking APIs. The regulation requires banks to open their system to third parties. But it cannot require new market participants to enter the market and force any other party to provide new customer services based on the open banking APIs. Nevertheless, the regulator demands proof of testing with market participants.
Most TPPs focus on very simple functionalities under open banking, such as displaying account information from other banks. But finding a TPP that can handle standing orders or file payments is almost impossible. Additionally, it can be difficult for a small bank to find a suitable partner for testing when the few that are available focus on testing with large retail banks.
more information, more open banking
High number of requests, concurrent sessions and volume
The regulation (the EBA Guideline) states: “[…] the ASPSP should have in place processes to establish and assess how the dedicated interface performs when subjected to an extremely high number of requests from PISPs, AISPs and CBPII, […]”. No one can currently estimate the future volume of open banking requests and its evolution. Setting up IT architecture with excess performance capacity comes at a cost. As all open banking action relies on consent to access account information or execute payments on accounts, we suggest setting the actual volume of stress testing as a factor of the maximum concurrent online banking users the bank expects. The actual factor depends on how many actions a bank expects a single user to be able to execute within a given timeframe, as well as on the readiness of the open banking market in general. We believe that setting the maximum number of concurrent open banking users for stress testing as equal to the current peak of online banking users is a safe assumption, and will usually be accepted by the regulatory authorities, provided that targeting the open banking API endpoints with as many users does not lead to significant delay in the user journey.
Similar requirements should be met to stress test the high number of endpoint calls and concurrent sessions. Again, this should also be derived from online banking usage. The regulation suggests concluding the testing with market participants. Owing to the absence of TPPs this must, however, be tested internally using software, i.e. by creating a «virtual TPP». This virtual TPP can be programmed to simulate large volumes and calls to all endpoints. There are multiple load testing tools that can be used, such as Gatling, Tsung, The Grinder, and many more. Since implementing these tools typically takes several weeks and specific technical know-how, we recommend including this in the overall open banking plan to avoid underestimating this task.
Supporting access by multiple PISPs, AISPs and CBPIIs
Testing multiple access by multiple TPPs is an essential part of the work to be carried out. Open banking requires new functionality in terms of being able to automatically onboard TPPs and compare each TPP request with a national registry. Testing those functionalities is also challenging given the lack of TPPs ready to test the onboarding. While creating different types of virtual TPPs to simulate multiple onboarding and access is an option, the number of TPPs that are willing to test onboarding with an ASPSP is typically higher than for other use cases. This is true for two reasons. The first is that onboarding is step number one, required to unlock further open banking use cases. The second is that for many TPPs, a high number of ASPSP connections is a sales tool that enables it to position itself as ahead of the pack and acquire further business.
Additional considerations: open banking versioning and test facilities
There is yet another element of open banking testing that is important to raise: Different versions of open banking co-exist on the same national market. Each version has a limited lifetime and a different functionality scope. In the UK, for example, the nine largest banking groups, the so-called CMA9, already went live with open banking in January 2018. This go-live represents version 1 of open banking in the UK. Now for the PSD2 go-live in mid-September, the expected version is 3.1, while at the same time version 1 still needs to be maintained, as some of the productive functionality of TPPs continues to depend on it. Offering and maintaining multiple releases is a serious challenge not only in terms of stress testing, but testing in general. On top of that, each jurisdiction leaves some room for interpretation within each of the open banking versions (and believe us, this room for interpretation is actually being used!). While banks are given what initially appears to be a generous three to six months to adapt to the next version, it is vital to integrate open banking release planning not only in the testing framework, but also in the open banking operating model.
Last but not least, all banks were asked to provide a testing facility by mid-March 2019 at the latest, with a productive sandbox from mid-June onward. Looking back it is clear that these testing facilities mostly failed to address the stress testing points and the release version points mentioned above. This has gone by more or less unnoticed, as the limited availability of TPPs has meant use of these testing facilities has been only scant. The few available TPPs have mainly focused on a handful of large retail banks.
The intention behind the regulator’s call for the industry to complete stress testing is good – especially in terms of ensuring that the system will handle open banking calls as efficiently and quickly as actions in online banking. But some of the challenges mentioned seem to have surprised the regulatory authorities. Therefore, all of them now accept and acknowledge that a bank cannot be forced to complete stress testing with a market participant if there is none available or willing. Completing stress testing in-house with a mock-up and software will therefore form a major part of the stress testing, and to a large extent is acceptable from a regulatory reporting perspective. And rest assured, many issues are typically discovered and fixed already in the course of these tests. As long as you are able to set up a virtual TPP for testing, we believe this is a viable alternative solution to complete the mandatory stress testing. Virtual TPPs will remain the standard during the build-up of open banking ecosystems.
As issues and incidents will probably occur after going live, and given that open banking is a synonym for fast-paced change in the financial world, you should stay tuned for the last article in our series. We will be looking at how to support open banking as an organization. As you have learned by now, open banking requires us all to look beyond the borders of past concepts and endeavors.
