In developing products and services, it is now a cliché to say that quality assurance is a critical part of software development process. It is even a bigger cliché to try and underscore its importance in a heavily regulated and sensitive industry such as healthcare. However, given the tremendous flux the industry has seen in recent years, the healthcare software product is often out with a short shrift in the quality assurance. In this whitepaper, we will take a look at these quality assurance (QA) aspects which often get overlooked. These aspects should be part of a checklist of a QA initiative as and when applicable. For the sake of brevity, we are focusing only on the care delivery products for now.

Healthcare solutions are increasingly being delivered on mobile applications; relevant implications have been highlighted in each aspect as applicable.

1.Data privacy: Adherence to strict authorization and access control is the norm while designing and testing healthcare software products. It is also, one of the most focused aspects of testing. However, some aspects which could escape the scrutiny of a QA could be listed as under:

•Departmental boundaries: Access to protected health information (PHI) should be limited only to the care team of the patient and not the entire facility.
The definition of care team often varies in different facilities but more often it is limited to associated departments in the care workflow.

•Report generated data: Data access restrictions are robustly implemented in

the core Electronic Health Record (EHR) workflow of a product. It might not be implemented for drill down reports in the reporting module. Reporting is often implemented as part of a data warehouse and access restrictions are not carried over to the Reporting module. From a QA perspective, if the user does not have access to the patient’s data, the relevant PHI fields should be masked.

2.Data Security and Integrity:

•Application vulnerability (Security testing): Healthcare remains one of the most targeted sectors for hackers. The Identity Theft Resource Center (ITRC) conducted a research and presented the data breach report for 2015. The report

lists healthcare at top of the list with a whopping 66.7% among all the breach records originating from healthcare. As a healthcare software product QA, there should be renewed focus on application security and vulnerability. This would include Open Web Application Security Project 10 (OWASP 10) and Common Vulnerability and Exposures (CVE).

•Clinical workflow audit: The audit functionality in application is often focused on changes done to the PHI. However, changes introduced during a clinical workflow such as change in prescription and diagnosis is equally important. Hence, as a part of standard QA process clinical workflow audit should be verified.

•Encryption: There are two aspects to encryption, which a QA must focus on namely the type of encryption implemented and the state of data. Health Insurance Portability and Accountability Act (HIPAA) recommends a Federal Information Processing Standards (FIPS) 140-2 validated encryption mechanism. Data should be encrypted both in motion and at rest. While encryption of ‘data in motion’ is usually implemented through the Transport Layer Security (TLS) layer of security, and hence easy to validate, verification of encryption of ‘data at rest’ is a little tricky. That is because it could be implemented at device level (hard disk encryption, not considered sufficient), database level, and at application level.

Mobile implication: If a mobile application stores data locally, either on a permanent or an intermediate basis. It should be verified at an appropriate level and checked if the encryption has been implemented. The data should be particularly checked in the web and hybrid applications; because there are no strong JavaScript-based Encryption modules available.

3.Conformance: It will be trite to say that healthcare is a heavily standard driven Solution domain. In this context, ‘Conformance’ can be defined as an adherence by Health IT product to the applicable standards. These standards could either be required for regulatory compliance and certifications, or for interoperability.

•Regulatory compliance: Compliance is mandated by regulations; hence this could be termed as most important of the conformance criteria’s. Examples include HIPAA compliance, Food and Drug Administration (FDA) regulations, data security regulations, and so on.

•Functional completeness: Often Health IT solutions need to conform to functional completeness standards. These are either mandated by regulatory authorities such as the American Recovery and Reinvestment Act (ARRA) Health Information Technology for Economic and Clinical Health (HITECH) a meaningful use of standards or could be by industry standard bodies now-defunct as Certification Commission for Healthcare

Information Technology (CCHIT).

•Interoperability: Interoperability conformance refers to semantic and syntactic interoperability agreement between two participating healthcare entities. These could either be directed by regulations or subject to inter-entity agreements.

As an Health IT QA it is imperative to verify that conformance to any existing standards are not broken during development cycles. The regulatory and business workflow implications could be serious.

Mobile implication: For a mobile device, it is important to take a closer look at the role played by it in a clinical workflow and verify if it falls under the FDA definition of a medical device.

If so, relevant guidelines will have to be followed Also, if the m-health application implements interoperability standards such as Institute of Electrical and Electronics Engineers (IEEE) – 11073x and Integrating the Healthcare Enterprise (IHE)–Primary Ciliary Dyskinesia (PCD) conformance needs to be taken care.

4.Test data management: The effectiveness of QA initiative depends on the coverage of test scenario validations. This in turn depends on the diversity of test data available. This becomes more prominent where automation is a key component in the overall QA strategy to ensure validity of clinical and non-clinical workflows. An effective test data management strategy in the context of Healthcare IT applications should have the following characteristics:

•It should be production ‘like’ data

•It should contain ‘inconsistent data’ or in other words, the nature and state of data in production should be reflected in the test data and should not be cleansed.

•Should be able to quickly refresh or rollback when needed.

•To ensure quick refresh could be on a subset of production data.

•However, referential integrity of the
data and external links should not be compromised.

•The PHI should be masked or scrambled to ensure the compliance.

The data which reflects the ‘truth’ of data
in production or the ‘gold copy’ as it is often referred to in testing world should be the bedrock of an effective test data management strategy.

5.Data validation: Valid data is critical to business operations in any industry, and more so in healthcare. Though, this is focused on in any QA initiative, there are some areas which are more critical than others. Some aspects of

product don’t get the attention towards detail as they ought to.

Following are two areas in a healthcare application that are either critical or often overlooked:

•Dosage data: Medication dosage data is the single most important and most complex data in a care delivery cycle.

Medication dosage information is a complex function of weight of the drug, volume, physical form, weight of patient, frequency of administration, and route of administration. An incorrect translation or calculation could result in under or over dosage.

For example, there was a undesirable outcome with devastating consequences in the radiation poisoning adverse event during a cancer treatment at Panama City.

Medication dosage data should be subject to stringent manual and automated QA checks. An independent dosage calculator outside the scope of the healthcare application should be relied upon, while verifying the calculated medication dosage data.

•Reporting data: Reporting data often gets a short shrift during QA and validation activities because of the sheer volume of it. This applies to clinical and non-clinical reports. With the increasing popularity of Population Health Management paradigm in healthcare reports, they have now acquired center stage in the care delivery process. Their complexity has increased many folds with sophisticated risk stratification and prediction techniques. From a QA perspective, clinical and non-clinical reports should be verified by focused test scenarios and random verification with source data. Automation techniques play a key role here.

6.Performance: Unlike e-commerce or social media platforms which handle millions of transactions per second across vast geographical boundaries, Health IT systems for most of the time cater to a limited user group in a defined geographical boundary. However, given the implications, performance is still a critical aspect in success of an Health IT product. Those around care delivery ensure availability of clinical data at the point of care. Central processing unit (CPU intensive Health IT solutions such as Digital Imaging and Communications in Medicine (DICOM) image manipulation systems are also applicable here.

Healthcare QA initiatives should focus on the performance aspect while validating an Health IT system. This applies to Software as

a Service (SaaS) systems and traditional desktop based systems as well, in various contexts. Inputs from a Performance testing would help product architects to define aspects such as the recommended local data size, archival strategy, scalability model, and so on.

Mobile Implication: If m-health application stores data locally or performs any CPU intensive operations, it should be extensively evaluated for performance.

7. Usability: Aspects of usability are often
crystallized before reaching QA, though this is not directly related to typical QA responsibilities. During QA cycles, IT should be kept towards any usability related issues. That is because; unlike other industries IT adoption is a big problem

in healthcare. Also, usability issues can lead to devastating clinical issues. For example, a patient suffered an adverse event in UCSF Benioff Children’s Hospital who got overdosed by more than 300%. This was caused because medication management interface did not intuitively prevent users from making mistakes.

8. Automation: It would be trite to repeat the utility of automation techniques in a QA initiative. However, QA automation has much more importance in healthcare than in any other industry. That is primarily because healthcare is standard driven and at a high premium and is put on ‘conformance’ in a Health IT solution. So, post maturity of an Health IT solution, it makes sense for any QA initiative to invest in automation to seamlessly maintain conformance.