8 January, 2018
Dan Martland's Two-Part Blog Series for GDPR:Report
Edge Testing's Head of Technical Testing Dan Martland recently wrote a two-part blog series on GDPR for GDPR:Report. Dan looks at how to reduce GDPR compliance risk when developing new products and whether data masking is vital for your business.
What Have You Missed? Reducing GDPR Compliance Risk When Developing New Products
Regardless of how your company or organisation is using the data it holds, you will be affected by the General Data Protection Regulation (GDPR) in May 2018. If you are not confident that your customer data is being managed in a way that satisfies GDPR requirements then your organisation is probably at risk. The penalties for non-compliance have been well-publicised but awareness of the penalties doesn’t seem to have had much impact, with only 19% of UK CIOs having any current plans in place to deal with it.
The challenge of how to mask customer data or build accurate and useable synthetic data, while retaining referential integrity for testing new products and services is bringing the testing community to the forefront of data handling practices. There is an overarching requirement to ensure that individuals’ data is processed securely and that a transparent audit trail is in place for compliance purposes.
The new rules could affect the pace of innovation and change: an essential part of developing new software (or enhancing existing systems) is making sure that the changes will work with all the variations in customer data that emerge over time, especially in systems that have been live for a long period of time. Without access to representative customer data, the task of development and testing becomes much more complex.
Here we look at the major data related challenges presented by GDPR and the transformative role test data management can make in a new product/service development environment.
Most wanted: Test data management
Arguably, poor test data management represents the biggest risk to a business in terms of breaching GDPR legislation. Test data can be the ‘forgotten man’ when building business-driven test scenarios, but it is central to GDPR compliance.
Test data management spans a broad range of quality assurance driven disciplines that support all IT and Business test phases including:
• The generation of non-production data sets that are representative of live data so that realistic and relevant tests can be conducted before releasing or updating a new service or product
• Building synthetic data where it is not possible or acceptable to use ‘real’ data
• Ensuring data can be shared across IT and business teams
• Enabling data to ‘time travel’ to support complex business test scenarios that may occur within a specific date range or sequence
• Planning effective backup and restore capabilities
• Supporting effective build and deployment of test environments.
Consent and transparency go hand in hand
GDPR requires that data subjects provide specific and active consent covering the use of their data. This need for consent requires that processes be in place that will ensure proper recording of both the granting and withdrawal of that consent. Consent for third-party processing is also affected as the Data Owner is liable for data, wherever the data may be handled.
Having secured consent, it is important to define and then manage legitimate data use and length of storage before archival and deletion within the scope of that consent. It is here that test data management will be crucial, particularly if regulators were to demand evidence of due diligence.
It is and has been common practice to use copies of real, live data to test systems, but this is not tenable in the world of GDPR. Individuals need to give explicit and informed consent that their data can be used for testing, which is not something that can be baked into consent granted for other purposes – indeed, attempting to do so could be regarded as a breach of the GDPR regulations in its own right.
The GDPR legislation also states that individuals have the right to data portability, a concept thatallows customers to move, copy or transfer personal data from one IT environment to another in a safe and secure way.
This relatively new concept will require significant testing, and ensuring compliance for data in-flight will be a major exercise for organisations that have high volumes of live data in non-protected environments. Test data management gives testers access to the data in a structured and readable format, enabling them to confirm that the original data has been removed from the ‘source’ system.
These considerations will drive changes to how late-phase testing such as User Acceptance Testing (UAT) is defined, planned and undertaken, driving the creation of new risk based test scenarios and affecting the number of ‘Must Tests’ that will need to be executed within a UAT test window.
With the fines for non-compliance so high, it is essential to ensure that any new functionality is not downgraded or negatively impacted by the changes. In terms of ongoing assurance, this clearly increases the need for regression testing across projects. GDPR testing and test cases will now need to be added to your regression pack.
Is data masking vital for your business?
Real and synthetic data are the two most common types of test data in use today, and present different challenges in the light of GDPR regulation.
While the use of synthetic data may minimise risk, it is not always feasible to generate such data, in which case anonymization would be essential. Using synthetic data and data masking will be two of the approaches that organisations will now need to consider when moving away from using copies of live production data.
Data masking protects the original source data by using powerful tools to anonymise the information being handled. Some tools provide a level of indirection through a ‘snapshot’ capability wherein the user works on an isolated, anonymised copy of the underlying data (the snapshot) rather than the underlying database. Other strategies are also available, for example, dynamic anonymization where the result of a query is anonymised in real-time so there is no need to take a snapshot.
In addition, new assurance processes and procedures will be needed to ensure that personal data is not exposed to persons who are not authorised to handle it.
Tools and data discovery
In terms of data discovery, 75% of organisations said the complexity of modern IT services means they can’t always know where all customer data resides. A retail client recently conducted a discovery exercise and found terabytes of ‘forgotten’ customer data that was more than ten years old. Under GDPR, any organisation holding on to data for ten years needs to be able to justify the length of time the data has been retained. This will have an impact on Business as Usual and risk management within the organisation.
Another example of long-term data retention gone wrong is the TalkTalk breach revealed last year where some of the data that was breached was ten years old. Ultimately, one of Talk Talk’s acquisitions, a small regional cable and TV company, was revealed as the source of the leaked data.
To ensure GDPR compliance the use of personal data in all test environments including backups and personal copies created by testers must be fully documented. An understanding of all real-data sources and the current location of that data is key to ensuring that no real personal data is being exposed to developers, testers, business users and other team members.
Some organisations have legacy, poorly supported IT systems, with unstructured data making tracing the data highly complex. The problem is compounded when organisations also have emails on file relating to individuals containing names, addresses, telephone number and contact information. Those responsible for GDPR compliance must be able to examine the databases and find the related email attachments and data related to an individual.
Unfortunately, finding the right information within gigabytes of data can be a hugely time-consuming task. Testing teams can simplify the search for the data using the same tools that would be typically used for test automation.
GDPR is not just another regulatory requirement; the transition to becoming a GDPR-compliant organisation is a major undertaking and maintaining compliance requires an ongoing commitment and new ways of testing.
Indeed, GDPR compliance will be one of the major IT challenges over the next few years, with initial compliance followed by continuous testing of that compliance.
A robust test data management strategy will save money – and indeed could save an organisation full stop. We see the growing importance of test data management and the provision of ongoing assessments to prove compliance, as a necessary investment to guard organisations against non-compliance.
Back to News