A Review of the USP Workshop on Alternative Methods

Last week, the USP hosted a workshop on alternative microbiological methods. Dr. Tony Cundell, a member of the microbiology expert committee, briefly summarized the workshop and the status of the revision to chapter 1223 in the Rapid Microbiology Methods LinkedIn Group:

The USP Alternative Microbiological Methods Workshop, March 16-17 at USP Headquarters attracted 66 registrants. The USP Microbiology Expert Committee presented revisions to USP<1223> 

The revisions included a discussion on the limitations of the CFU, acceptable procedures, and performance, results and decision equivalence option. Also the concept of the non-inferiority test was promoted as a statistical tool. The revision will be published June, 2015 with an official date of December, 2015. 

Invited speakers from the FDA, JP, the pharmaceutical industry and academia generally supported the revisions.

I appreciate Tony’s post.  I also attended this workshop and although the invited speakers, as he stated, generally agreed with the revision, a number of attendees (including myself) were left with unanswered questions and an uncertainty of the appropriateness of the revised chapter 1223 as well as a proposed chapter on a rapid sterility test.  But there were also productive presentations that provided an overview of current regulatory expectations, the use of statistics and industry’s perspectives on validation.  A brief summary of my take away messages from the meeting is presented below.

Erika Pfeiler (CDER, FDA) described the Agency’s focus on alternative microbiology methods (AMMs). She reviewed FDA’s policies and stated that CDER has approved AMMs for water testing, environmental monitoring, bioburden testing, microbial limits (for release and stability) and sterility testing (for release and stability). She also stated that FDA welcomes submissions for the use of AMMs, they are routinely approved, different approaches to validation are acceptable and validation studies should depend on your product and process.  I agree with Erika’s position and continue to appreciate the Agency’s support for the implementation of new microbiology technologies.

Stephen Wicks (EDQM) reviewed the current revision to Ph. Eur. chapter 5.1.6. This draft is currently available for public review (through Pharmeuropa) and you can provide comments until March 31. I am encouraged by Stephen’s statement that the revised chapter 5.1.6 will be fairly aligned with PDA’s Technical Report No. 33 (TR33).

Nobuyasu Yamaguchi (Osaka University) provided an overview of the Japanese Pharmacopeia’s (JP) new informational chapter on rapid microbiological methods (RMM). The chapter will be published in the 17th edition of the JP with an English version being provided in the summer of 2015. When speaking on validation, he stated that the RMM must be equivalent or better than the current method. However, he also stated that a correlation between the RMM and the current method would not be necessary. When I asked him how you could demonstrate the RMM is equivalent or better than the current method without correlating the two, his response was that the validity of the new method should be confirmed. When asked for an example, he could not think of a RMM that would fit this scenario.  Therefore, in my opinion, the JP should consider providing additional clarification regarding the demonstration of equivalence.

Edwin van den Heuve (Eindhoven University of Technology) presented a detailed review of how to use non-inferiority models when statistically comparing data sets arising from AMM validation studies.  I thought his presentation was well received and his recommendations offer the industry additional statistical tools when validating AMMs.

Separately, representatives from the USP expert committee provided their overviews of changes to USP 1223 and a new proposed chapter, 71.1.

James Akers summarized the purpose for the changes to USP 1223. He stated that microbiology analysis is transitioning from largely growth-based methods to methods that are based more on molecular biology. The term “molecular biology” was also used in the revision process.  When questioned to clarify if the term “molecular” meant nucleic acid amplification techniques (NAAT), he responded that the term was not limited to NAAT but to methods that worked “at the molecular level.” It is my opinion that this term should be better defined in chapter 1223 as many technologies may work at the molecular level.

He then described the CFU as a cell count estimate, which we all agreed on, since the CFU may arise from a single cells or a clump of cells. Following an extended discussion on the CFU, he stated that the CFU can no longer be used as the “Gold Standard” for microbial enumeration, and that the true nature of the CFU be considered when working with AMMs that provide alternate signals. I generally agree with his position.

He mentioned that a technology that provided enhanced colony detection (based on growth) would not require validation as a completely new method. I also agree, and PDA’s TR33 described these types of methods as automated technologies.

Next, he stated as we move away from growth-based methods the traditional route to microbial identification is not possible and that a laboratory can use specific PCR probes for this process.  I do not necessarily agree with this position because today’s PCR probes are generally used for presence/absence testing for a specific microbial target of interest (e.g., E. coli) and not for microbial ID unless PCR is used in conjunction with gene sequencing. Furthermore, if the starting material is DNA, the potential for detecting residual DNA or DNA from dead cells can lead to a false positive response. This is why most PCR-based systems require a sufficient amount of viable cells (e.g., from an isolated colony) to avoid this phenomenon. We may have to wait for the publication of the final revision of chapter 1223 to understand the direction the USP is taking.

Edward Tidswell subsequently went into greater detail on the changes to chapter 1223. Essentially, his presentation was a comparison of the original USP 1223 with the proposed revision that was published in 2014 for public comment.  He stated the core intent of the revision was about patient safety and a paradigm shift to unshackle ourselves from the CFU and to embrace a concept of “decision quality.” That’s good. He also stated the revision simplifies the validation process. I disagree. The draft was very confusing to follow and was the reason for my submission of a comprehensive list of comments and questions to the committee last year. I also understand that others in the industry, including organizations such as the PDA, responded with similar comments.  Unfortunately, Ed did not address any of the industry’s responses sent to USP during the public comment period, and now we must wait until the final revision is published mid-year to determine if our comments were adequately addressed.

Following Ed’s presentation, one attendee asked what would happen if the published chapter still requires clarification in order to understand the document’s guidance and recommendations. The response from USP was as follows:  the chapter can be revised in the next revision cycle (i.e., 2015-2020). In my opinion, this is not an efficient process considering the industry voiced its opinions last year and attendees repeated a number of their submitted comments during the workshop. For example, David Roesti (Novartis) commented (during this year’s workshop and during last year’s public comment period) that the recommended use of a 0.2 delta value when calculating non inferiority was too strict. He also stated that they successfully used a 0.3 value when validating a rapid sterility test and FDA and EMA approved their approach.  However, if they would apply the 0.2 value to their test data, the test for non inferiority would fail. USP’s response to his comments was as follows: we should address this ASAP…but it probably won’t happen until after the final revision is published. I should note that David Roesti also presented his company’s work on validating a rapid sterility test that was approved by more than 50 countries including the US and EU. It is my opinion that the USP should align their recommendations (sooner rather than later) with the validation strategies that have been accepted by the global regulators.

Tony Cundell followed with an overview of the equivalence section of the draft 1223 chapter. He opened his presentation by stating that existing method validation approaches are limiting RMM implementation. I disagree. A number of companies around the world have successfully validated alternative and rapid methods, including those used for sterility testing, which have gained FDA and EMA approvals by following the validation guidance in documents such as PDA’s TR33.

He then walked the attendees through the same table for equivalence that was presented in the 2014 draft chapter.  The four equivalency options are still extremely confusing and it is not clear from his presentation whether the final draft will provide additional clarification on how each of these options are to be applied.

The next topic focused on a proposed rapid sterility test chapter numbered 71.1. According to James Akers, the purpose of such a chapter is to encourage the development of a rapid sterility test for stakeholders that require a time-to-result that is much faster than the conventional 14-day assay.  Stakeholders were identified as compounding pharmacies, cell therapy companies and firms that produce radiopharmaceuticals. During presentations by James Akers and Tony Cundell, it was disclosed that the USP will advocate a limit of detection (LOD) of 10-100 cells for a rapid sterility test, which, as the presenters explained, will provide an acceptable level of patient safety for those products that have a very limited shelf life. In fact, the USP expert committee is working towards writing a new chapter describing a compendial rapid sterility test based on reverse transcriptase (RT)-PCR (using universal primers and probes) and flow cytometry that would take between 6 to 48 hours to complete. Tony also stated that we cannot have a rapid sterility test that only specialized labs can perform.  I agree; however, I do have additional concerns regarding their position.

First, the selection of RT-PCR and flow cytometry may be premature at this time. The USP based the selection of these two platforms on a stakeholder survey in which six respondents recommended RT-PCR and two respondents recommended flow cytometry. That’s a total of eight (8) stakeholders. In my opinion, the numbers of respondents does not necessarily reflect the views of the industry as a whole.

Furthermore, I do not believe that RT-PCR may be as sensitive or robust as what may be required for a rapid sterility test.  The supplier of a commercial, off-the-shelf (COTS) RT-PCR system has previously reported their LOD at 100-1000 cells, which may be too high for an acceptable rapid sterility test. Interesting, this same system was removed from the market two years ago due to issues associated with robustness and sensitivity.  Additionally, flow cytometry may not possess the required LOD for a rapid sterility test (e.g., some flow cytometry systems report a sensitivity level of at least 100 cells with good repeatability). Therefore, additional work in this area is warranted. It is also my opinion that a rapid sterility test that is longer than 48 hours (e.g., 72 hours) may still be acceptable for the stakeholders while meeting the needs of the patient. For this reason, the end-user should dictate the required time-to-result, not the USP.

Second, it is my opinion that an acceptable LOD of 10-100 cells may not necessarily be aligned with regulatory expectations. During the same workshop, Simleen Kaur (CBER, Office of Compliance and Biologics Quality, FDA) presented an overview of technologies that the Agency examined while developing a rapid sterility test. She stated that studies were conducted using 1 and 10 cells, but that she did not know at what level FDA would ultimately accept in terms of LOD for a rapid sterility test (i.e., 1 or 10 cells).  Interestingly, James Akers was quick to interject that we need to be very careful about using a single cell level as an appropriate LOD for a rapid sterility test. Only time will tell what organization will really take the lead in this debate.  I will say; however, that multiple companies around the world have successfully validated (and have gained regulatory approval for) rapid sterility tests by challenging their products with single cell concentrations of stressed organisms (and even below this level, such as 0.5 and 0.05 cells).  The approved technologies have included solid phase cytometry and ATP bioluminescence.

Lastly, if the USP wants to avoid “a rapid sterility test that only specialized labs can perform,” how can they recommend RT-PCR at this time? In the absence of a COTS RT-PCR system, I would expect that a lab wanting to validate a rapid sterility test would have to acquire all the instrumentation, probes and primers and conduct the necessary studies with expertise that have significant molecular biology skills to qualify the method for its intended use. Wouldn’t this be a considered a “specialized lab?”

Given all of these points, it is my opinion that the USP expert committee may need to rethink its position regarding a rapid sterility test chapter and to utilize the appropriate experts (e.g., in the form of an advisory panel) to help guide the process forward.

In conclusion, the workshop provided for good discussions and consideration for the implementation of rapid and alternative methods across our industry. Unfortunately, we will have to wait for the publication of the final chapter 1223, which is due out this summer, to see what changes the expert committee has made, if any, to the comments that were submitted last year. And if the chapter does not provide additional clarity and direction, the industry can always rely on PDA’s TR33 and Ph. Eur. 5.1.6.