The Rapid Micro Blog


Our blog will keep you informed of new and noteworthy technologies, reviews of recent publications and presentations, upcoming conferences and training events, and what's changing in the rapid and alternative microbiological methods world.

A Review of the USP Workshop on Alternative Methods

Last week, the USP hosted a workshop on alternative microbiological methods. Dr. Tony Cundell, a member of the microbiology expert committee, briefly summarized the workshop and the status of the revision to chapter 1223 in the Rapid Microbiology Methods LinkedIn Group:

The USP Alternative Microbiological Methods Workshop, March 16-17 at USP Headquarters attracted 66 registrants. The USP Microbiology Expert Committee presented revisions to USP<1223> 
The revisions included a discussion on the limitations of the CFU, acceptable procedures, and performance, results and decision equivalence option. Also the concept of the non-inferiority test was promoted as a statistical tool. The revision will be published June, 2015 with an official date of December, 2015. 
Invited speakers from the FDA, JP, the pharmaceutical industry and academia generally supported the revisions.

I appreciate Tony’s post.  I also attended this workshop and although the invited speakers, as he stated, generally agreed with the revision, a number of attendees (including myself) were left with unanswered questions and an uncertainty of the appropriateness of the revised chapter 1223 as well as a proposed chapter on a rapid sterility test.  But there were also productive presentations that provided an overview of current regulatory expectations, the use of statistics and industry’s perspectives on validation.  A brief summary of my take away messages from the meeting is presented below.

Erika Pfeiler (CDER, FDA) described the Agency’s focus on alternative microbiology methods (AMMs). She reviewed FDA’s policies and stated that CDER has approved AMMs for water testing, environmental monitoring, bioburden testing, microbial limits (for release and stability) and sterility testing (for release and stability). She also stated that FDA welcomes submissions for the use of AMMs, they are routinely approved, different approaches to validation are acceptable and validation studies should depend on your product and process.  I agree with Erika’s position and continue to appreciate the Agency’s support for the implementation of new microbiology technologies.

Stephen Wicks (EDQM) reviewed the current revision to Ph. Eur. chapter 5.1.6. This draft is currently available for public review (through Pharmeuropa) and you can provide comments until March 31. I am encouraged by Stephen’s statement that the revised chapter 5.1.6 will be fairly aligned with PDA’s Technical Report No. 33 (TR33).

Nobuyasu Yamaguchi (Osaka University) provided an overview of the Japanese Pharmacopeia’s (JP) new informational chapter on rapid microbiological methods (RMM). The chapter will be published in the 17th edition of the JP with an English version being provided in the summer of 2015. When speaking on validation, he stated that the RMM must be equivalent or better than the current method. However, he also stated that a correlation between the RMM and the current method would not be necessary. When I asked him how you could demonstrate the RMM is equivalent or better than the current method without correlating the two, his response was that the validity of the new method should be confirmed. When asked for an example, he could not think of a RMM that would fit this scenario.  Therefore, in my opinion, the JP should consider providing additional clarification regarding the demonstration of equivalence.

Edwin van den Heuve (Eindhoven University of Technology) presented a detailed review of how to use non-inferiority models when statistically comparing data sets arising from AMM validation studies.  I thought his presentation was well received and his recommendations offer the industry additional statistical tools when validating AMMs.

Separately, representatives from the USP expert committee provided their overviews of changes to USP 1223 and a new proposed chapter, 71.1.

James Akers summarized the purpose for the changes to USP 1223. He stated that microbiology analysis is transitioning from largely growth-based methods to methods that are based more on molecular biology. The term “molecular biology” was also used in the revision process.  When questioned to clarify if the term “molecular” meant nucleic acid amplification techniques (NAAT), he responded that the term was not limited to NAAT but to methods that worked “at the molecular level.” It is my opinion that this term should be better defined in chapter 1223 as many technologies may work at the molecular level.

He then described the CFU as a cell count estimate, which we all agreed on, since the CFU may arise from a single cells or a clump of cells. Following an extended discussion on the CFU, he stated that the CFU can no longer be used as the “Gold Standard” for microbial enumeration, and that the true nature of the CFU be considered when working with AMMs that provide alternate signals. I generally agree with his position.

He mentioned that a technology that provided enhanced colony detection (based on growth) would not require validation as a completely new method. I also agree, and PDA’s TR33 described these types of methods as automated technologies.

Next, he stated as we move away from growth-based methods the traditional route to microbial identification is not possible and that a laboratory can use specific PCR probes for this process.  I do not necessarily agree with this position because today’s PCR probes are generally used for presence/absence testing for a specific microbial target of interest (e.g., E. coli) and not for microbial ID unless PCR is used in conjunction with gene sequencing. Furthermore, if the starting material is DNA, the potential for detecting residual DNA or DNA from dead cells can lead to a false positive response. This is why most PCR-based systems require a sufficient amount of viable cells (e.g., from an isolated colony) to avoid this phenomenon. We may have to wait for the publication of the final revision of chapter 1223 to understand the direction the USP is taking.

Edward Tidswell subsequently went into greater detail on the changes to chapter 1223. Essentially, his presentation was a comparison of the original USP 1223 with the proposed revision that was published in 2014 for public comment.  He stated the core intent of the revision was about patient safety and a paradigm shift to unshackle ourselves from the CFU and to embrace a concept of “decision quality.” That’s good. He also stated the revision simplifies the validation process. I disagree. The draft was very confusing to follow and was the reason for my submission of a comprehensive list of comments and questions to the committee last year. I also understand that others in the industry, including organizations such as the PDA, responded with similar comments.  Unfortunately, Ed did not address any of the industry’s responses sent to USP during the public comment period, and now we must wait until the final revision is published mid-year to determine if our comments were adequately addressed.

Following Ed’s presentation, one attendee asked what would happen if the published chapter still requires clarification in order to understand the document’s guidance and recommendations. The response from USP was as follows:  the chapter can be revised in the next revision cycle (i.e., 2015-2020). In my opinion, this is not an efficient process considering the industry voiced its opinions last year and attendees repeated a number of their submitted comments during the workshop. For example, David Roesti (Novartis) commented (during this year’s workshop and during last year’s public comment period) that the recommended use of a 0.2 delta value when calculating non inferiority was too strict. He also stated that they successfully used a 0.3 value when validating a rapid sterility test and FDA and EMA approved their approach.  However, if they would apply the 0.2 value to their test data, the test for non inferiority would fail. USP’s response to his comments was as follows: we should address this ASAP…but it probably won’t happen until after the final revision is published. I should note that David Roesti also presented his company’s work on validating a rapid sterility test that was approved by more than 50 countries including the US and EU. It is my opinion that the USP should align their recommendations (sooner rather than later) with the validation strategies that have been accepted by the global regulators.

Tony Cundell followed with an overview of the equivalence section of the draft 1223 chapter. He opened his presentation by stating that existing method validation approaches are limiting RMM implementation. I disagree. A number of companies around the world have successfully validated alternative and rapid methods, including those used for sterility testing, which have gained FDA and EMA approvals by following the validation guidance in documents such as PDA’s TR33.

He then walked the attendees through the same table for equivalence that was presented in the 2014 draft chapter.  The four equivalency options are still extremely confusing and it is not clear from his presentation whether the final draft will provide additional clarification on how each of these options are to be applied.

The next topic focused on a proposed rapid sterility test chapter numbered 71.1. According to James Akers, the purpose of such a chapter is to encourage the development of a rapid sterility test for stakeholders that require a time-to-result that is much faster than the conventional 14-day assay.  Stakeholders were identified as compounding pharmacies, cell therapy companies and firms that produce radiopharmaceuticals. During presentations by James Akers and Tony Cundell, it was disclosed that the USP will advocate a limit of detection (LOD) of 10-100 cells for a rapid sterility test, which, as the presenters explained, will provide an acceptable level of patient safety for those products that have a very limited shelf life. In fact, the USP expert committee is working towards writing a new chapter describing a compendial rapid sterility test based on reverse transcriptase (RT)-PCR (using universal primers and probes) and flow cytometry that would take between 6 to 48 hours to complete. Tony also stated that we cannot have a rapid sterility test that only specialized labs can perform.  I agree; however, I do have additional concerns regarding their position.

First, the selection of RT-PCR and flow cytometry may be premature at this time. The USP based the selection of these two platforms on a stakeholder survey in which six respondents recommended RT-PCR and two respondents recommended flow cytometry. That’s a total of eight (8) stakeholders. In my opinion, the numbers of respondents does not necessarily reflect the views of the industry as a whole.

Furthermore, I do not believe that RT-PCR may be as sensitive or robust as what may be required for a rapid sterility test.  The supplier of a commercial, off-the-shelf (COTS) RT-PCR system has previously reported their LOD at 100-1000 cells, which may be too high for an acceptable rapid sterility test. Interesting, this same system was removed from the market two years ago due to issues associated with robustness and sensitivity.  Additionally, flow cytometry may not possess the required LOD for a rapid sterility test (e.g., some flow cytometry systems report a sensitivity level of at least 100 cells with good repeatability). Therefore, additional work in this area is warranted. It is also my opinion that a rapid sterility test that is longer than 48 hours (e.g., 72 hours) may still be acceptable for the stakeholders while meeting the needs of the patient. For this reason, the end-user should dictate the required time-to-result, not the USP.

Second, it is my opinion that an acceptable LOD of 10-100 cells may not necessarily be aligned with regulatory expectations. During the same workshop, Simleen Kaur (CBER, Office of Compliance and Biologics Quality, FDA) presented an overview of technologies that the Agency examined while developing a rapid sterility test. She stated that studies were conducted using 1 and 10 cells, but that she did not know at what level FDA would ultimately accept in terms of LOD for a rapid sterility test (i.e., 1 or 10 cells).  Interestingly, James Akers was quick to interject that we need to be very careful about using a single cell level as an appropriate LOD for a rapid sterility test. Only time will tell what organization will really take the lead in this debate.  I will say; however, that multiple companies around the world have successfully validated (and have gained regulatory approval for) rapid sterility tests by challenging their products with single cell concentrations of stressed organisms (and even below this level, such as 0.5 and 0.05 cells).  The approved technologies have included solid phase cytometry and ATP bioluminescence.

Lastly, if the USP wants to avoid “a rapid sterility test that only specialized labs can perform,” how can they recommend RT-PCR at this time? In the absence of a COTS RT-PCR system, I would expect that a lab wanting to validate a rapid sterility test would have to acquire all the instrumentation, probes and primers and conduct the necessary studies with expertise that have significant molecular biology skills to qualify the method for its intended use. Wouldn’t this be a considered a “specialized lab?”

Given all of these points, it is my opinion that the USP expert committee may need to rethink its position regarding a rapid sterility test chapter and to utilize the appropriate experts (e.g., in the form of an advisory panel) to help guide the process forward.

In conclusion, the workshop provided for good discussions and consideration for the implementation of rapid and alternative methods across our industry. Unfortunately, we will have to wait for the publication of the final chapter 1223, which is due out this summer, to see what changes the expert committee has made, if any, to the comments that were submitted last year. And if the chapter does not provide additional clarity and direction, the industry can always rely on PDA’s TR33 and Ph. Eur. 5.1.6.

5 Comments

  1. Great Summary for all of us who were not able to attend. Thank you Michael.

    What is your thought about the committee's point of view about the use of the EMD Milliflex rapid sterility test as a sterility test and as a bioburden test for liquid filterable substances as alternatives to the classical based methods?

    ReplyDelete
  2. Thanks Daniel. The committee did not specifically discuss rapid methods for filterable samples, however, it is clear that ATP bioluminescence including using the system you mentioned is acceptable for use and has been approved by more than 50 countries for Novartis' vaccine products.

    ReplyDelete
  3. Michael this is a useful account of your view of the recent USP Alternative Microbiological Methods Workshop.

    I would make the following observations:

    Erika Pfeiler, CDER: Erika stated that around 5 AMA approvals are made annually. These approvals would be tied to NDAs or supplements for drug products and may under estimate the AMM implementations for water, environmental, and in-process bioburden monitoring.

    Stephen Wicks, EDQM: The alignment of 5.1.6 and PDA TR#33 may be due to an absence of out-of-the-box thinking by both parties retaining the positions first taken by the original PDA task force in the 2000 reversion of TR#33.

    Nobuyasu Yamaguchi, Osaka University: His reply suggested to me that he may not recalled or was prepared to give an example during his presentation but his comment suggest that the JP is open to the acceptable performance option.

    Jim Akers, USP: I believe the confusion over the term molecular-based methods was pointed out to the committee in the comments on the USP <1223> in-process revision and will be clarified in the official chapter. RT-PCR targets RNA which is a better measure of cell viability than DNA.

    Jim Akers, USP: The revised chapter will better define modern microbiological methods.
    Yes there is potential to sequence any amplified nucleic acid to identify the microorganisms in the failed test.

    Ed Tidswell, USP: Unfortunately providing more equivalency options and statistical tools to supply more flexibility may increase the complexity of method validation. The performance equivalency option that is currently employed is retained in the <1223> revision. Future revision will be the responsibility of the 2015-2020 cycle microbiology expert committee.

    Tony Cundell, USP: Companies are getting regulatory approval for the implementation of AMMs but the pace of implementation has been disappointing. The intent of the expert committee is assist with the validation of the next generation of methods and address the unmet needs of stakeholders who make short lived products.

    Based on the rationale presented the committee identified RT-PCR and flow cytometry as potential candidates for rapid sterility testing. I regret I did not emphasis the surveys were directed towards representative scientist working on these technologies for benchmarking purposes and to recruit collaborators for proof of concept studies and not a broad solicitation of stakeholder opinion. Clearly we need to establish a consensus on the users requirement specifications.

    New technologies are being marketed with automation platforms so they will be manageable for routine microbiological testing labs.

    Michael thanks again for your thoughtful comments.

    Tony Cundell

    ReplyDelete
  4. Thank you for your additional observations Tony.

    Both the PDA and EDQM based their recommended validation strategies on industry practices and regulatory expectations which have not changed significantly since the revised TR33 has been published. In fact, these same practices are used today.

    For example, companies that have validated alternative and rapid methods that do not rely on the CFU have used the strategies outlined in the revised TR33 with great success. And even though a number of these methods generate a non-CFU signal, firms have correlated the alternative (non-CFU) signal to the CFU in support of demonstrating that the new method is equivalent to the existing or compendial method. This is actually supported by one of the equivalency sections in the proposed USP 1223:

    “Performance equivalence requires the demonstration of equivalent or better results with respect to validation criteria—such as accuracy, precision, specificity, limit of detection, limit of qualification, robustness, and ruggedness—that may be appropriate for the intended use of the alternative qualitative or quantitative method.”

    Therefore, if the compendial test is measured in CFUs then it stands to reason that the alternative method, including one that does not report a CFU signal, is required to demonstrate equivalent results to the CFU method. And this is exactly what companies have done when following the strategies outlined in the revised PDA TR33.

    You also state that “Companies are getting regulatory approval for the implementation of AMMs but the pace of implementation has been disappointing.” I do not disagree with your assessment, although this slow pace cannot be solely due to a lack of understanding of how to validate these methods as demonstrated by a number of successful validation case studies within the industry. There are other factors at work, including a lack of technical champions within an organization or trying to apply a technology to an application or product matrix that simple work work. However, it is my opinion that your follow-on statement, “The intent of the expert committee is assist with the validation of the next generation of methods and address the unmet needs of stakeholders who make short lived products” appears to exclude a great number of companies manufacturing pharmaceutical and biotech products that are not short-lived. If it was the USP’s intention to cater to short-lived products, it is my opinion that the USP should have made this perfectly clear up front.

    Finally, it is my opinion that any mis-alignment between alternative method documents are more associated with a lack of cooperation rather than anything else, including what you have described as an absence of out-of-the-box thinking. As Chair of the PDA TR33 revision task force team, I made numerous requests with the USP microbiology expert committee to share information on our respective revision projects in hopes of minimizing differences in validation strategies. Unfortunately, USP declined, and the TR33 task force team proceeded accordingly. It is fortunate that EDQM accepts many of the strategies already in place within TR33 as best practices for validating alternative methods. In fact, EDQM’s proposed appendix describes three case studies from real-life validation programs conducted by European pharmaceutical companies. It is my opinion that this is the reason why the two documents are currently aligned.

    ReplyDelete
  5. Michael,

    Raising the tent poles to accommodate other poorly served stakeholders does not exclude pharmaceutical companies from the concern of the USP microbiology expert committee.

    Information was shared with stakeholders prior to the USP workshop. The leadership of the committee routinely updated stakeholders of the committee's activities in a compendial session at the annual PDA Global Pharmaceutical Microbiology Conference and presented at other industry meetings in the both US and Europe. Revision to USP <1223> was specifically discussed at these conferences. The in-process revision was published online in the Pharmacopeial Forum with a 90-day comment period. Numerous comments were received that were supportive and provided constructive criticism that was incorporated into the revision.

    Tony

    ReplyDelete
Previous Post Next Post

Contact Form