Loaded Room Studies for Performance Qualification: Can We Just Say No?
I recently watched your webinar "Modern Rules for an Old Practice: Sensor Placement for GxP Temperature Mapping" and found the information presented quite useful. I'm in a Quality Engineering role at a US biotech company. My role includes oversight of validation programs, specifically on qualification of temperature-controlled rooms. I have some questions about loaded-room mapping studies, but I'll explain a bit about our current work…
We are installing a large new freezer that will store final product in bulk form; this room will be temperature mapped during qualification. We are going to perform an empty room mapping study as part of our Operation Qualification. We had been planning a fully loaded study during our Performance Qualification, but are starting to question this approach for a variety of reasons:
- The long-held precedent of fully loading all storage positions (to provide a room loading bracket along with the empty room) is not sustainable. This freezer will have a large number of tank storage positions that would have to be filled, at a time when tank availability is not predictable.
- No one is ready to commit tanks of frozen product to fill our storage positions, no matter how robust the design and prior Installation and Operation Qualification testing that happens.
- Alternate or simulated loads (such as large empty cardboard boxes) have been used elsewhere, but their fabrication, setup and breakdown is getting cost-prohibitive.
In addition to an empty-room study, we are considering writing a position paper that provides engineering calculations, e.g. BTU delivered per unit volume of room space, vs. another validated freezer for which we have loaded (though not fully) PQ data. In this approach, we are trying to obviate fully loaded room studies, primarily to manage cost and no longer apply an unsustainable precedent.
My questions for you are:
- Have you heard a similar approach, i.e. no loaded room testing for a temperature-controlled room, with other regulated companies?
- Is there something missing here regarding proof of concept for a loaded-room testing scenario, even as a partially loaded room? This is my instinct, that some form of loaded room study is needed to balance out the baseline empty room and prove the theory developed in the position paper.
Very interested in your thoughts...
Quality Engineering & Validation
First off, thanks for watching our webinar!
This is a great set of questions, and I fully support doing whatever we can do (logically, sensibly, scientifically) to decrease the amount of unnecessary work, while increasing the value we get from the work we must do. That is the heart of risk assessment.
This isn't too difficult of a scenario because we are dealing with large frozen stuff, which is perhaps the easiest of all scenarios in mapping. By that I mean that we usually have a large acceptable operating range, no real low limit, and material with a LOT of thermal inertia. My quick answer to your questions would be to simply operate on a fundamental assumption: An empty chamber represents the worst case scenario.
But, this takes some rationalizing. So, my thinking is this: once you have loaded the chamber, it will be full of frozen product, which will have an incredibly high thermal mass. Not to mention the mass of stainless steel, which is significant despite having a lower specific heat. I would assume that the freezer for which you have data on loading would support this claim. This is assuming that you aren't using the freezer to cool the product and thanks to the frozen state. A cooling and freezing process is very different than frozen storage. (As a sidebar question: if your customers are hesitant to commit tanks of frozen product, will they commit empty tanks that would be a great placebo for loaded mapping?)
To address your actual questions:
Have I heard of not performing a loaded-room study? Yes. I know of FDA-regulated companies forgoing loaded-room testing for temperature-controlled rooms. However - please take this answer with a grain of salt. There are many companies that don't do loaded mapping, either through naiveté or lack of resources.
There are also those who haven't done loaded mapping simply because they haven't gotten to it. And there absolutely are firms that, by virtue of their process design or quality policies, don't perform loaded-room/chamber mappings. The studies can be regarded as impractical, too expensive, or not valued in terms of the data provided.
This is a difficult phenomenon because the regulations don't tell us how to perform studies. The guidance generally defaults to the most risk-intolerant position. Additionally, many companies won't willingly share information with outsiders for obvious reasons.
In 19 years of experience writing protocols, performing studies, and helping customers field audits, I have found that auditors are generally reasonable people. For that reason, I think that if you have a scientific and well-documented rationale for your choices, auditors will be cooperative. Moreover, I think they'll be quite interested. And if they find fault with your approach, the feedback is typically: "Do it differently next time, please." As opposed to: "You did it wrong, we are giving you a warning." Quite often, and this may be the case at your biotech firm, that the biggest source of friction in divergent approaches comes not from our regulators, but our own internal quality culture.
- Regarding your question about the need for loaded studies to balance out the empty room studies; I'm not sure I have the same understanding of the terms you are using. I assume that by position paper, you mean a paper based on your hypothesis and rationale – perhaps published in a peer-reviewed journal like The Journal of Validation Technology. I would be very interested to read such a paper! But, I will say that it sounds a little complex for a freezer.
While there is a huge investment in that freezer in terms of product, it is still just a freezer. My thinking is that, defrost cycles aside, there is not much risk because you are already doing an empty mapping. Is there a real risk that somewhere in the loaded freezer, some product will actually thaw? I can see an argument to try and operate with the highest set-point possible to get operating efficiency and reduce the possibly large costs of a freezer like this. But, this is not a quality issue. If it were, then some of the efficiency savings from operating at a high set-point should be redirected to fund the mapping effort – but yet again, perhaps administratively difficult to achieve.
I would do something much more simple yet (in my opinion) robust. I would perform the empty mapping, then use real-time data collection on every product tank in the freezer. Experience tells me that your customers will likely feel more comfortable knowing that their product is being monitored continuously. Otherwise, you risk looking as though product is stored in what could easily be considered an unqualified area. Finally, direct, continuous monitoring is an established and acceptable alternative to qualification. After you do this for a while, you will have a data set of real temperatures that will be even better than a loaded chamber PQ.
I recognize that you have the harder job than I; you actually have to execute, I can do "armchair" validation and simply discuss the options. You also have the hard work of getting multiple stakeholders with different objectives to support it. That is often the most nuanced and important work we do in quality and validation.
Anyhow, I'm grateful for the chance to offer an opinion. I'm also very interested in what you actually decide to do. And if you end up publishing your position paper, I'd like to read it, so please keep in touch!
Thanks again, and best of luck!
Sr. Regulatory Compliance Expert