Editor's Note: This is part 5 of a 7 part series on Process Risk Assessments.
Last week, we saw how capturing uncertainty and detectability risk brings clarity and dimension while assessing risk when used appropriately. These elements are especially important when creating a data-driven risk assessment approach, like those enabled in QbDVision.
This week, we will look at the practical aspects of process risk assessments from a different lens: that of the people executing or using these tools.
A common question that arises when designing a risk management plan relates to the appropriate number of risk levels. Should there be 3 or 5 or 10 levels? Should we use a square matrix (5x5) or non-square (5x10)? It is common practice to select 4 or 5 levels. Fewer than 4 levels may not provide sufficient resolution and more than 5 leads to difficulties deciding between the minute difference in risk. That being said, the FDA uses a 3x3 matrix for its internal CAPA system.
While there is no prescriptive guidance available to answer this question, it is important to first acknowledge an important aspect of answering this question is people. Risk Assessments are a data-based, but still subjective, evaluations of a situation by subject matter experts (SMEs). And, almost always, the process for defining risk is collaborative in nature. People get together and discuss, debate, and decide on risks. These evaluations need to be understood, managed, and updated by the SMEs and external stakeholders, especially when being used to define the scope of work. The results need to be understood in the broader context of the organization that will act upon them in the subsequent parts of the development process. While it is best for the principles and rules of assignment to be algorithmic and data-driven using a meritocratic process for deciding on the ultimate values, the final results need to be simple and clear to guide future actions.
Ultimately, a risk assessment results in a categorization. Fundamentally, it is a method for information chunking, distilling, and grouping complex sets of information for communication and decision-making purposes. Therefore, an important consideration for selecting the dimensions of risk is its intended use. Will people understand it?
Do you have a shared organizational lexicon to accurately describe nuances in risk across a 10x10 matrix? And, most importantly, will your people know how to manage the variances between them? Do you have time, resources, and capital to manage the complexity of such a large grouping? The answer to these questions will undoubtedly vary from team-to-team and organization-to-organization. And they may evolve over time as your organization grows and learns. Simplicity is a good long term bet. QbDVision provides flexibility to define whatever structures suit your organization. Each element of your schema can be annotated with risk and score labels, as well as clear descriptions that are provided inline to users when selecting risk values.
Concordant with evaluating your team's ability to understand the risk rating broadly is the assessment of their ability to know why it was given that assignment in the first place. Traditional matrix-based assessments are often lacking in their ability to record justifications. That is often because the same tool for viewing risk is being used for assessing risk: it’s just one spreadsheet. As seen in the Risk Assessment Examples for Manufacturing Processes in from earlier in our series, there is significant depth to the justification for each level. Imagine making that assessment, but then only having “High” as the result. That is an important aspect that is lost in today’s Excel-based processes. One way teams try to overcome this is by overloading this information in the risk schema itself - i.e., making a 10x10 to attempt to capture more detail about the variances. QbDVision makes it easy to capture layered and historical justifications at each level, similar to what you see in the table below. This way, when re-evaluating risk, you can drill into those justifications and decide: did we add a new source of risk, has the risk of a known source changed, etc. When you separate the justification behind this risk from the risk rating, it is easier to simplify your risk schema.
Risk Assessment Examples for Manufacturing Processes
Another consideration for deciding on the dimensions of your risk schema is organizational comparability. We often see different departments or groups with different risk schemas. While there can be sound reasoning for the variances, it is important to weigh the benefits of normalization efforts. With the prevalence of paper or document-based approaches, it is not surprising to see teams take this type of disjointed approach. Why spend the effort to normalize when there is little chance of doing a cross-project or drug comparison? As the industry continues to move towards structured platforms, like QbDVision, this will become more important. You’ll want to take Risk Assessments coming from a Materials Management group and evaluate them in context to your process development. When it only takes a click of a button to evaluate risk across projects or phases or to compare to organizational history or knowledge, you’ll unlock the power of this normalization.
A final question often encountered relates to the definitions of each discrete category and the individual layers. QbDVision provides three default RMP configurations, 3x3, 4x4, and 5x5 with accompanying definitions. Each default RMP can be copied and edited to create a new RMP configured to your organization’s needs.
In the end, people and their individual thoughts are a critical, non-linear (and uncontrollable!) element of assessing risk. Prior knowledge, experience, intuition, pattern matching skills, and recall abilities remain essential to assessing and acting on risk information. Applying those skills to a structured and data-driven approach can accelerate and maximize risk understanding. When designing that structure, it’s important to advocate for a strong foundation and a simple and concise result.
Next week, we will discuss how QbDVision risk management plans can be configured to handle the topics and situations reviewed in this series.
This post is part 5 of 7 in a series on practical risk management for pharmaceutical process development.