27. Any easily identified analyst errors, such as calculation mistakes, should be specified with particularity and supported by evidence. (925:20 (Gerraughty)). In some instances, however, "the subtle influences which can result in test variability are not apparent when such an assay or test investigation is carried out." (1954:13 (Cooper)). Thus, because it can be difficult to pin down the exact cause of a problem (1833:13 (Atwater)), it is unrealistic to expect that the cause of analyst error always will be determined and documented. (1416:22 (Bolton)).
28. In recognizing the existence of less readily identifiable mistakes and the influence of variables unrelated to the purity or potency of the drug under scrutiny, the Court does not intend to create a means for firms to avoid the performance and documentation of an informal laboratory investigation. The inability to identify an error's cause with confidence affects retesting procedures, see PP 38-39 infra, but does not affect the failure inquiry required for initial out-of-specification results.
29. Other problems more serious than single out-of-specification results, from multiple out-of-specification results to product mixups and contamination, require full-scale inquiries involving quality control and assurance personnel in addition to laboratory workers in order to identify the exact process or nonprocess-related errors.
30. Extending beyond the laboratory and often labeled formal investigations, (1563:25 (Rhodes)), these inquiries should follow the outline the Government provided, with firms paying particular attention to any necessary corrective action, whether reprimanding, retraining or firing employees, remixing batches or adjusting processes. Thus, in the failure report firms must: (1) state the reason for the investigation; (2) provide a summation of the process sequences that may have caused the problem; (3) outline the corrective actions necessary to save the batch and prevent a similar recurrence; (4) list other batches and products possibly affected, the results of their investigations, and any required corrective action; and finally, (5) preserve the comments and signatures of all production and quality control personnel who conducted the investigation and approved any reprocessed material after additional testing.
31. The outcome of the failure investigation will determine whether additional batches of the same product and related products also require remedial measures. (1709:10 (Rhodes)). Process-related errors suggest the need to examine other batches of the problem product as well as other products made according to similar procedures. Addressing nonprocess-related errors requires an examination of other batches or products the trouble-making employee or machine may have handled.
32. Thus, the elements of a "thorough" investigation necessarily will vary with the nature of the problem identified. However, all failure investigations must be performed promptly, within thirty business days of the problem's occurrence, and recorded in written investigation or failure reports.
33. The outlier test provides an alternative means of invalidating an initial out-of-specification result. If the failure investigation of an initial out-of-specification result proves inconclusive, firms searching for a better explanation can utilize this method.
34. Significant limits accompany the outlier test, however. The USP specifically warns against using outlier tests too often, and thus, as a general rule,
firms must be careful not to reject results frequently on this basis. (2036:9 (Baldassarre)).
35. In addition, the utility of the outlier test varies with the type of assay performed.
The USP expressly allows firms to apply this test to biological and antibiotic assays, see Ex. 7 at 1503; (202:21, 798:20 (Mulligan)), but is silent on its use with chemical tests. (1239:8 (Mulligan)). Although some experts advocated use of the outlier method for chemical assays, (1395:1 (Bolton)), other testimony suggested that firms generally do not rely on outlier tests to invalidate chemical test results. (1238:21 (Mulligan)). In the Court's view the silence of the USP with respect to chemical testing and outliers is prohibitory.
36. The substantial innate variability of microbiological assays, (1916:1 (Cooper)), supports this distinction. (1238:11 (Mulligan)). Chemical assays are considerably more precise than biological and microbiological assays, (1915:12 (Cooper)), since only the latter testing is "subject to whims of microorganisms." (1915:18 (Cooper)).
37. In addition to triggering a failure investigation, out-of-specification results also generate the need for retesting. A retest is defined as additional testing on the same sample, and thus it necessarily follows an initial test. An analyst performing a retest takes the second aliquot from either: (1) the sample that was the source of the first aliquot (1022:5 (Bolton), 1533:21 (Rhodes)); or (2) the larger sample previously collected for laboratory purposes.
(1534:8 (Rhodes)). These procedures are equivalent. (1839:17, 1840:16 (Atwater)). Thus, whether retesting is performed at the finished product or blend stage, such testing should be performed from the same bottle of tablets or capsules and the same drum ore mixer, respectively. (661:2 (Mulligan); 1423:3 (Bolton)).
38. Retesting is proper only after a failure investigation is underway, (235:1, 661:2 (Mulligan)), since the outcome of the failure investigation itself, in part, determines when retesting is appropriate.
Retesting is necessary if a failure investigation indicates that analyst error caused an initial out-of-specification result. (144:1, 1023:18 (Mulligan)). A retest is similarly acceptable when review of the analyst's work is inconclusive. (664:12 (Mulligan)). In these instances, retesting substitutes for or supplements the original tests which have been rejected or questioned, respectively. In the case of nonprocess and process-related errors, however, retesting is suspect.
(1024:10 (Gerraughty)). Because the initial tests are genuine, in these circumstances, additional testing alone cannot infuse the product with quality. (1024:15 (Gerraughty)).
39. As a general matter, the amount of retesting required also varies with the problem identified. Out-of-specification results attributed to analyst error require limited retesting. (1529:19 (Rhodes)). Here, retesting merely supplants the first round of initial tests. More extensive retesting should follow an inconclusive failure investigation, since firms need to determine whether the out-of-specification result is a mere anomaly or a reason to reject the batch. (1390:1 (Bolton)).
40. The USP contemplates retesting for quality control purposes (1937:14 (Cooper)), although it does not prescribe or recommend the number of individual tests that must be performed in order to reach a definitive conclusion about the quality of a product. (1929:1 (Cooper)). Thus, the number of retests performed before a firm concludes that an unexplained out-of-specification result is invalid or that a product is unacceptable is a matter of scientific judgment. (488:7, 816:9 (Mulligan); 1959:3 (Cooper)). Yet the goal of retesting is clear; firms must do enough testing to isolate the out-of-specification result, in order to reach the point at which the additional testing overcomes the out-of-specification result. (805:4 (Mulligan)).
41. Nevertheless, retesting cannot continue ad infinitum. Because such a practice is not scientifically valid, a firm's predetermined testing procedure should contain a point at which testing ends and the product is evaluated. At this time, if the results are not satisfactory, the batch must be rejected. (1958:7 (Cooper)).
42. When evaluating retest results, it is important to consider them in the context of the overall record of the product. Relevant to this review are the history of the product,
the type of tests performed,
and any results
obtained for the batch at other stages of testing.
As such, retesting determinations will vary on a case-by-case basis, a necessary corollary of which is that an inflexible retesting rule, designed to be applied in every circumstance, is inappropriate. (1476:9 (Bolton)).
43. Resampling, in contrast, is a more controversial practice. Typically resampling occurs after the initial test and the retests have produced out-of-specification results, thereby indicating a more serious problem. When performing a resample, an analyst leaves the laboratory and takes a new sample from the universe of the batch. (1534:13 (Rhodes)).
44. Resampling is appropriate where provided by the USP (1044:21 (Gerraughty)), as in cases of content uniformity and dissolution testing. (1045:3 (Gerraughty)). Similarly, in the limited circumstances in which a failure investigation suggests that the original sample is unrepresentative, resampling is acceptable. (1024:19 (Gerraughty)). Evidence, not mere suspicion, must support a resample designed to rule out preparation error in the first sample.
(1955:8, 1956:1 (Cooper)).
45. Absent these limited exceptions outlined above, however, firms cannot rely on resampling to release a product that has failed testing and retesting. (1045:19 (Gerraughty)).
46. The need for remixing arises during the blend stage when testing reveals problems with content uniformity. The regulations themselves allow reworking, which essentially is remixing. (906:11 (Gerraughty)). As evidenced by the Generic Drug Office directive and the consent agreement between the FDA and Eli Lilly, remixing is allowed in this circumstance. (558:2 (Mulligan)).
47. The instance of remixing, however, is directly related to its propriety. Occasional remixing is acceptable, (674:2 (Mulligan)), but frequent or wholesale remixing is not. The need to remix often provides a clear indication that the process is invalid (907:5, 1021:9 (Gerraughty)), and casts doubt on those batches that passed through testing without incident. (576:14 (Mulligan)).
48. Although averaging test data can be a rational and valid approach, (1578:4 (Rhodes)), as a general rule, firms should avoid this practice, because averages hide the variability among individual test results.
49. This phenomenon is particularly troubling if testing generates both out-of-specification and passing individual results which when averaged are within specification. Here, relying on the average figure without examining and explaining the individual out-of-specification results is highly misleading and unacceptable. (812:20 (Mulligan); 1255:11 (Gerraughty)); see also (1481:20, 1482:8 (Bolton) (averaging blend assay results invites problems at finished product stage where values of 89, 89, and 92, given range of 90 to 110, should be met with more testing even though average equals 90)).
50. Although averaging camouflages variability, an average may provide more information about the batch's true assay value that any single test result.
(1872:4 (Atwater)). Thus, in the case of microbiological assay testing, the USP prefers an average, (1932:2 (Cooper)), when reaching an ultimate judgment about the quality of the product. (1929:22 (Cooper)). It is good practice to include out-of-specification results in the average, unless an outlier test indicates that an out-of-specification result is an anomaly. (1579:23 (Rhodes)).
51. Finally, the average of individual content uniformity tests at the finished product stage can act as a proxy for the assay value. (1231-33 (Gerraughty)). Though this estimate cannot substitute for final product assay testing, it can provide some information about a batch.
8. Releasing a Batch for Distribution
52. Section 211.165(a), in relevant part, provides:
For each batch of drug product, there shall be appropriate laboratory determination of satisfactory conformance to final specifications for the drug product, including the identity and strength of each active ingredient, prior to release.
21 C.F.R. 211.165(a). And section 211.165(f) specifies:
Drug products failing to meet established standards or specifications and any other relevant quality control criteria shall be rejected. Reprocessing may be performed. Prior to acceptance and use, reprocessed material must meet appropriate standards, specifications, and any other relevant criteria.