If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
For process optimisation Design of Experiments (DoE) has long been established as a more powerful strategy than a One Factor at a Time approach. Nevertheless, DoE is not widely used especially in the field of cell-based bioassay development although it is known that complex interactions often exist. We believe that biopharmaceutical manufacturers are reluctant to move beyond standard practices due to the perceived costs, efforts, and complexity.
We therefore introduce the integrated DoE (ixDoE) approach to target a smarter use of DoEs in the bioassay setting, specifically in optimising resources and time. Where in a standard practice 3 to 4 separate DoEs would be performed, our ixDoE approach includes the necessary statistical inference from only a single experimental set. Hence, we advocate for an innovative, ixDoE approach accompanied by a suitable statistical analysis strategy and present this as a practical guide for a typical bioassay development from basic research to biopharmaceutical industry.
Cell-based bioassays represent a key analytical technology from basic research to drug-discovery or within a quality control unit in the biopharmaceutical industry. Due to the presence of cells and the multiple handling steps, these assays often show high scattering [
]. This necessitates laborious optimisation in order to detect potential test sample differences with the required resolution. This is especially the case if the assays need to perform robustly, such as in a quality control department for the release of biopharmaceutical product or within stability studies.
The preferred method for obtaining optimal assay performance is by using a Design of Experiment (DoE) approach. While in pharmaceutical industry DoE slowly but steadily emerges for various analytical methods (e.g. mass spectrometry [
]), the practical implementation of DoE in the field of bioassays is still limited. This is especially remarkable as, due to the presence of cells, bioassays often contain complexities such as 2-factor interactions. These interactions cannot be revealed with One Factor at a Time (OFaT) approaches where only one factor is varied and the corresponding effect is studied. If there are known or suspected interactions between parameters a DoE experiment is needed to study the whole (multivariate) design space simultaneously [
]. Further, a DoE is much more efficient in comparison to OFaTs as every factor can be evaluated in every run which results in a higher statistical power, i.e. the chance to detect a real effect is higher. Finally, full randomization within OfaTs is not possible but necessary to exclude confounding effects [
]. This survey revealed that:“Researchers do not use DoE approaches to optimise a biological process because the majority of them believe that this kind of design is very difficult to employ. In addition, many researchers stated that they were not aware of DoE, while others did not fully understand the benefits of these approaches in assay development”.
A second explanation for the hesitation to apply DoE is likely to be cost and time related as up to four independent DoE-approaches are performed during assay development via SCOV or SCOR i.e., screening, characterisation, optimising, verification/robustness [
To overcome the hurdle of high costs and time we have developed the so-called integrated DoE (ixDoE) which enables us to optimise an assay with a single DoE approach.
This smart, risk-based approach allows us to extract almost the same information as with the textbook procedure and at the same time reduces the costs and time by more than half. The ixDoE approach can be divided into 3 steps (see Fig. 1) with the typical planning, experimental, and data analysis phases for the central single DoE which is framed by the assay feasibility and the verification before and after the DoE, respectively.
Fig. 1– DoE Strategies. a Overview of classical SCOV/SCOR DoEs versus ixDoE including estimated relative timelines; b Workflow of ixDoE including the detailed phases. The bullet points correspond to the sub headers in the methods section where they are described in detail. Assay feasibility and verification are framing the ixDoE.
Given our positive results with the ixDoE approach across multiple assays, it is our aim to provide fellow researchers a blueprint to help overcome the current inertia. Therefore, we present a best practice procedure including detailed steps how to plan, perform, and evaluate the ixDoE approach.
Methods
We propose the ixDoE approach consisting only of a single DoE, framed by assay feasibility and verification as visualised in Fig. 1. We include a detailed description, background and strategical consideration of this approach in this chapter. Having successfully applied this method to many assays, we also provide the data of one representative example of a cell-based bioassay with the different steps for visualisation of the ixDoE concept.
Materials and assay performance
For the assay cells designed with the PathHunter® from eurofins/DiscoverX were used. The assay principle is a cell-based receptor dimerization assay. This assay utilises the Enzyme Fragment Complementation (EFC) technology, where the β-galactosidase (β-gal) enzyme is split into two fragments, ProLink (PK) and Enzyme Acceptor (EA). Both fragments alone have no β-gal activity. The cells have been engineered to co-express one target receptor subunit fused to PK and a second target receptor subunit to EA. Binding of soluble ligand to the receptor subunits induces receptor dimerization and subsequently leads to the complementation of the two β-gal fragments. This results in the formation of a functional β-gal enzyme that hydrolyzes a substrate to generate a chemiluminescent signal.
For analysing the assay response the PathHunter® bioassay detection kit from DiscoverX was used as substrate. The Readout was done with a microplate reader (SpectraMax M3) using the SoftMaxPro software. Planning and analysing of the DoE was done with Design-Expert [
R2A cells were thawed, washed with assay medium, and added to the assay plate in the needed cell numbers according to the DoE run sheet. The cells were incubated at +37°C and 5 % CO2 in an incubator for the respective time span. During incubation the ligand was prepared in octuplicates (with 12 dosage points each) by serial dilution in a separate test plate and transferred to the assay plate. After addition of the ligand, the assay plates were incubated in an incubator for the respective time span. For the read-out the respective volume of substrate was added to the assay plate. The assay plates were shaken and incubated at room temperature for the respective time span. The luminescence of the test plates was measured using a microplate reader. Measured data were fitted with a 4-parameter logistic fit.
Assay feasibility
To establish any new assay a so called “assay feasibility” is mandatory prior to any further steps. At the beginning of assay feasibility the read-out type is defined (e.g. absorbance, fluorescence) and the evaluation strategy of the measured values is set. The evaluation is typically done via assessment of dose-response curves fitted with a 4-parameter logistic function [
]. Several parameter describing the quality of the dose-response curve can be evaluated (e.g. signal/noise, for further details see “Selection of responses”).
In the next step, the assay developer should evaluate all existing information and, eventually, perform a small subset of experiments before going into the planning phase of our ixDoE. It is not necessary to perform a dedicated screening DoE as for bioassays typically enough information is available in advance. Possible sources of knowledge are: manufacturers information (assay kits and reagents), literature (publications of similar assays), expert knowledge (experience from related assays), and limits given by surrounding conditions (maximum number of cells available, time spans that fit into daily routine). In cases where the information is not sufficient, a small subset of experiments is needed. If too many potentially relevant factors were identified during planning phase, the factors which are unlikely to have effects or multiple interactions might be addressed during assay feasibility. Furthermore, experiments accounting the assay specificity should be also done within assay feasibility.
Based on the collected information and measured data, the assay developer can now select the experimental factors with the greatest potential and the design space of interest.
For the given example we addressed the following points during assay feasibility:
•
Testing of cell handling, growth behaviour in different media and comparison of continuous cell cultivation vs. ready-to-assay cells (R2A) (exemplary curves see
Fig. 2- Exemplary results of assay feasibility (A) Comparison of cultivated cells versus ready-to-assay cells (B) Comparison of ligands from different manufacturers.
Testing of assay conditions given by the manufacturer as a starting point and test of several assay conditions to avoid being outside the working range which would cause failed experiments in the later DoE.
•
Comparison of ligand from different suppliers to find the best material and to ensure ligand supply in case of delivery problems (exemplary curves see
Adaptation of pipetting steps and volumes to simplify routine procedure and to enable assay automation.
Integrated DoE (ixDoE)
Planning phase
The ixDoE starts with the planning phase (Fig. 1).
General considerations on assay mode of action (MoA)
Depending on the MoA of the target molecule there are two types of bioassays: a) the target molecule directly induces the response in the assay (i.e., activation or inhibition of the target receptor), b) a ligand (e.g. cytokine) induces a response which is inhibited by the target molecule by either blocking the receptor or the ligand. This kind of assay is used for e.g. monoclonal antibodies inactivating a specific ligand (like risankizumab targeting IL-23).
While for bioassays of type a) the usage of the target molecule in the assay is obviously mandatory, for bioassays of type b) we recommend performing the DoE with the ligand only i.e., without the target molecule. In the latter case, the presence of the target molecule prevents finding the true optimum of the assay (for details see supplement S1).
The exemplary DoE shown here is a bioassay type b) for an inactivating antibody.
Selection of factors
Typical relevant factors in bioassays are cells, ligand, substrate, medium and supplements to name a few. We will describe the most common factors here.
Cells
The DoE can be made with cells coming from continuous cultivation or R2A cells. Cells coming from continuous cultivation means less work initially as cells can be derived with a small lead time and constantly provided during DoE. However, there are some disadvantages due to variabilities which will likely affects the response of the cells e.g., cell cultivation, cell passage number. These disadvantages require more complex designs, leaving the developer with a choice between performing additional runs or putting up with a reduced statistical power. Using R2A cells means more work initially but overcomes the above-mentioned problems. We therefore recommend using R2A cells in DoEs if routine testing is planned to be done with R2A cells. To minimise variation, all DoE experiments shall be done with the same R2A cell bank.
The cultivation type (continuous cultivation vs. R2A) might also be used as separate categoric factor. In doing so a direct comparison of the results for continuous and R2A cells can be done. That can allow defining assay conditions that work well for both cell types and gives flexibility in later routine.The number of cells is one of the most important factors in bioassays and thus needs to be included. If two cell lines are used in the assay (e.g., for Antibody-dependent cytotoxicity assays), the cell number of both cells should be addressed as two separate factors in the DoE given that using a ratio would lead to a loss of information.
Other factors which are related to cells but less frequently used are e.g., pre-culture time (number of days after last sub-cultivation of the cells), pre-incubation time (the time the cells are allowed to attach in the well before the ligand is added), trypsination time or cell passage number .
If using adherent cells, the pre-incubation time (i.e., the time to allow the cells to attach to the microplate after they have been seeded prior to addition of the ligand/target molecule) should be tested in the DoE. Cells might show a different behaviour when not yet attached to the plate surface. For suspension cells this is usually not relevant except if the cells are known to be very sensitive, or if the cell preparation needs e.g., centrifugation or washing steps which stresses them.
In special cases other factors might be relevant e.g., trypsination time, if the receptor involved in the investigated signalling pathway is very sensitive to damage by trypsination. Factors like cell passage number might be recorded as uncontrolled factor to allow for correction of unexpected influence during data analysis.
Ligand, substrate, and other factors
In bioassays of type b) as described above, the ligand plays an important role in the DoE since the response is highly dependent on the ligand concentration. In these cases, the ligand is titrated to a dose-response curve and it is not a separate factor in the DoE. The incubation time of the ligand with the target cells is also highly relevant for the biological response. Depending on the complexity of the biological response it may be optimal after a few minutes or even several days.
To our experience, substrate concentrations given by the manufacturers are often higher than needed and reducing volumes can save money without losing assay performance. In addition, substrate incubation time strongly affects signal heights and should therefore be considered as a factor. Be careful that the substrate volume is not a limiting factor in the assay as this might cause artificial results.
Further factors which are less often used are the temperature of incubation steps like incubation of cells with ligand or substrate. These incubation steps are usually performed at 37°C in an incubator. However, there might be cases where incubation at room temperature or 4°C might lead to beneficial results. In this case temperatures of the respective incubation steps should be included as a categorical factor (e.g. 4°C, 37°C) or as a continuous factor (e.g. testing at 15°C, 20°C, 25°C) factor.
Furthermore, other materials might also be relevant for the assay performance such as different cell culture or assay media and supplements concentration (e.g., FCS).
As the developer chooses more factors to include in the DoE more runs are needed to achieve sufficient statistical power. Thus, it must be carefully assessed which factors are most likely to be relevant in the DoE. If too many factors (i.e., more than six) were identified during the planning phase, some of these factors might be addressed during assay feasibility if interaction effects with other factors are deemed unlikely.
In our example DoE R2A cells are used as these cells will also be used for routine testing. Cell number is used as one of the most relevant factors in a bioassay. The pre-incubation time is tested as a factor to challenge whether the preincubation is needed or not.
As the assay is a ligand assay, the ligand is titrated with different concentration on to the plate and it is not needed as factor. Ligand incubation time should always be used as a factor, so it was here. Different ligand suppliers were tested prior to the DoE as additional runs would be needed to achieve sufficient power. For the substrate, volume and incubation time were chosen as factors. Other materials like media or supplements were not used as factors in the DoE but were assessed during assay feasibility.
Definition of design space
After choosing the factors, the design space is chosen using knowledge acquired during the assay feasibility. A wider design space is preferred as this increases the chance to detect even small, but nonetheless relevant, effects. If one or more factors are not showing an effect and the model-fit is still valid, it can be concluded that the response is robust against these factors. In fact, these non-relevant factors improve the overall performance of the model by increasing the number of degrees of freedom and consequently increase the statistical power. To control the type-2 error (the chance of missing a significant effect) the design has to be planned using statistical metrics (see supplement S2). Furthermore, the design space should not be arbitrarily wide; this helps to avoid exceeding the “edge of failure” i.e., extreme combination of factors such as using many cells but very little ligand to avoid that the assay completely fails. For response surface models three levels of each factor should be chosen. The levels for the design space are typically linear and equidistant or, in special cases, set up as log scale (e.g., exponential behaviour of a signal-pathway).
For our example, the lower limit was chosen as half of the manufacturer's recommendation (i.e., 5,000 cells/well). The upper limit was chosen with 55,000 cells/well to have a large range with respect to typical cell densities in suspension. To ensure equidistant levels, 30,000 cells/well were chosen for the centre point. For the pre-incubation time 0 – 4 hours are recommended by the manufacturer. As the potential omission of the pre-incubation time would make assay preparation much easier, the factor was included in the DoE with the time spans 0/2/4 hours.
For the incubation time of the ligand with the cells 16 hours are recommended by the manufacturer. In the daily routine 16 hours means a typical overnight incubation and was thus used as the lower limit. To increase flexibility a time span of up to 24 hours was chosen for the ligand incubation time, thus levels are defined as 16/20/24 hours.
For the substrate a volume of 50 µl and a substrate incubation time of 3 hours are recommended by the manufacturer. For the volume the lower limit was chosen to be 25 µl since this is still easy to handle with precision. The middle level was chosen to be 50 µl and thus the upper limit was 75 µl to stay equidistant. For the substrate incubation time the limits were chosen to have as much flexibility as possible. During a normal working day time spans of up to 7 hours are desirable and therefore 7 hours was chosen as upper limit. To get equidistant levels a centre point of 4 hours and a lower limit of 1 hour were chosen.
Number of runs
The number of possible runs may be dependent on: length and complexity of the assay, project timelines, costs, availability of devices, and choice of design type etc. Typical considerations are: how many plates are possible per day, restrictions around number of daily cells availability, and the pricing of each plate.
For our example DoE strategy with up to 9 plates per day are possible. There are no special restrictions, however assay substrate is expensive and the minimum number of runs is strongly preferable.
Selection of design type and creation of run table
Many different design types are possible - however, we recommend using D- or I- optimal designs as response surface models with 3 levels per factor. These designs are flexible and require fewer runs as compared to full factorial designs. For full factorial designs all possible combinations of factors are performed. This results in Ln number of runs (L being the number of levels and n the number of factors) which would result in 81 runs for only 4 factors with 3 levels each. Often fractional factorial designs are used as only a subset of the full factorial design is needed allowing a reduced number of runs. But these designs either have a lower power/resolution or need a preselection which both bear the high risk to overlook existing effects. To determine the adequate number of runs in combination of an adequate design quality most software for planning of DoE (e.g., Design-Expert [
]) provide guidance (see Supplement S2 for further information).
With the results from the assay feasibility a D-optimal design was chosen as response surface model to assess main effects, quadratic and 2 factor-interactions. The design was a split-plot design with 3 levels on each factor contains 36 runs, distributed on 4 different blocks each represented by 9 assay plates performed on a single day. Blocking was done to consider variations between the different cell preparation used per day. The run table of the design is shown in Table 1.
Table 1Run table of the DoE with the respective factor levels.
The model is slightly underpowered (<80%) for delta=2 for quadratic effects, but sufficiently powered for delta=3 (data not shown). Furthermore, all factors show high orthogonality indicating a balanced design. The design presented here shows an average leverage of 0.19 and a maximum of 0.34 for two runs; we consider this low and therefore unproblematic. The highest correlation is between the quadratic term of pre-incubation time and substrate volume with -0.27. We consider this small and, given the high statistical power, not noteworthy.
Experimental phase
For performing the experiments, we suggest each run is performed on one plate. The whole dose-response curve should be used as this gives important information on the assay performance. Other approaches where only single points in the asymptotes are determined are likely to miss information (e.g., EC50, slope) or in the worst case even give misleading results (i.e., if dose response curve shifts and the points are not in the asymptotes anymore). In using the whole plate, up to 8 replicates for the dose response curve on the plate can be used to determine the assay variance in a precise way. A further benefit of using whole plates is the fact that it is easy to prepare and to handle – especially if using automated pipetting devices. In most cases these reasons justify the higher costs of reagents and plates.
To reduce variation the experiments should be done in a short time period and with little variation (e.g., experiments done by same analyst with same reagents and devices). Execution might be semi-automated or even fully automated to further increase precision by minimising pipetting variance and standardising assay performance.
During the experiments results should be evaluated immediately to check validity of the assays. If runs are invalid, the overall strategy is not lost. The design can be augmented and additional experiments can be added if necessary. As the ixDoE-approach is risk based, in very rare cases the design space might not been chosen adequately and the assay might not work. In this situation the data obtained within the first day of experiments are typically sufficient to identify the culprit, correct the range, and repeat this small set of runs.
Data analysis phase
For the analysis of the data, three consecutive steps are needed: selection of responses, modelling of responses, and numerical optimisation. These steps are followed by a robustness check.
Selection of responses
For analysis we evaluate several different parameters describing the dose-response curve. Depending on the aim of the DoE (e.g., optimise signal amplitude, decrease variance) we focus on those responses which address the respective aim best. In doing so, analysis and computation of pairwise correlations of all potentially relevant responses might be useful to decide on which parameters are most important. If responses have the same aim and are highly correlated, it is recommended to focus on the response which describes the quality of the curves in the best way. With respect to the assay type (i.e. with target molecule or with ligand only, as described above) there were no difference in the assessment of the responses as the parameters describing the assay performance are the same.
Two responses which are typically highly relevant for bioassays are described in the following.
•
Signal-to-noise (S/N) ratio: The S/N ratio assesses the variance of the measured replicates. We calculate it as the mean standard deviation (SD) of the highest concentration points divided by the signal amplitude. The smaller the variance is, the more precise are the fitted curves and thus the relative potency results. In our experience the S/N ratio is the most crucial assay parameter.
•
Dynamic range (DR): The DR describes the ratio between upper and lower asymptote. The DR is preferred over the separate assessment of upper or lower asymptote as it gives a ratio which is more meaningful than the absolute values. A high dynamic range is desirable as it gives a good assay window and therefore a robust assay.
In case of that the responses S/N ratio and dynamic range do not give meaningful models, alternative responses might be taken into consideration:
•
To assess assay variance, alternatives to S/N ratio can be the noise (the mean SD of the highest concentration points), R² value, residual sum of squares, and root mean square error (RMSE).
•
To address the assay window, alternatives to the dynamic range can be the amplitude of the upper or lower asymptote itself or the span (amplitude of the upper minus the lower amplitude).
Furthermore, bioassays can have special requirements, leading to additional parameters used in the optimisation:
•
Hill Slope: If the values of an assay are too high or too low, the hill slope should be used as additional response in the analysis of the DoE to obtain an optimal value. A too low value for the hill slope means very high serial dilution steps in the assay und therefore might cause a quite high variation within the relative potency values. A too high hill slope might show a high precision but a low assay range where only a low bandwidth of potency values can be measured (e.g. only 70 – 130 %).
•
EC50 value: Normally bioassays do not have to be optimized with respect to the EC50 value as the assay is a relative assay and absolute EC50 values are not relevant. However, when the assay needs very high doses of e.g. an antibody it might cause problems in the routine use. When only a very small predilution is needed there can be matrix effects from the formulation components which can have a negative impact on the assay (caused by e.g. Polysorbate). Furthermore, a large amount of antibody could be necessary to run an assay which might be a problem if the available samples volumes is very low. In these cases, a minimization of EC50 can help to overcome or at least reduce these problems.
•
OD: Restrictions in the devices might have to be considered, for example, readers for optical density might have a restricted linear working range (e.g., up to OD3). Thus, if OD values are too high it might be necessary to minimise the values to ensure staying in the linear range.
In our example DoE the signal-to-noise ratio was used to assess the variance of the assay and the dynamic range was chosen to assess the assay window.
Modelling of responses
Before modelling takes place, data should be checked for its distribution. Models are then fit and the quality of the fit is assessed using metrics such as R2adj and by graphical inspection. See supplement S3 for general recommendations on the modelling process.
After evaluation of the data and model quality, the data should be analysed with respect to the impact of the tested factors on the respective response. The coefficient estimates and their respective standard errors within the model are assessed on biological relevance. Not all terms which are included in the model are necessarily biologically relevant.
Response: signal-to-noise ratio
The S/N response was transformed using inverse square root transformation according to a box-cox plot and the transformed data is modelled as described above. The ANOVA table of the resulting model is given in Table 2.
Table 2- Analysis of variance table for selected model for signal/noise ratio.
Table 2 shows that, of the ten possible two-way interaction, five are retained in the model selection as well as two of the quadratic terms. The R2 was 0.8986 and the adjusted R2 was 0.8821. R2pred, lack-of-fit, and pure error is not computed in this case using Design-Expert [
An assessment of the residuals (including checks for normality, trends in prediction, and leverage) was performed. The results seen in Fig. 3 indicate good adherence of residuals to a normal distribution, no obvious bias in predicted values versus actual values, and no high leverage observational units.
Fig. 3- Assessment of residual in terms of normality (Upper), prediction error trends (Lower Left), and leverage of particular experiments using difference-in-fits (DFFITS) (Lower Right).
Fig. 4 shows the coefficient estimates and their respective standard errors determined by the chosen model in a Pareto plot. The plot shows all positive effects in red and all negative effects in blue in a descending order with respect to the % of the average signal size of the signal/noise ratio.
] via “numerical optimisation”. If several responses turned out to be relevant to describe the quality of the assay, an optimisation on the respective response models is recommended. A weighting on the responses is also possible if importance of the responses is different. The goal of the optimisation for each individual response needs to be defined (e.g., as high as possible, in the range of x-y). Visualisation of the optimisation can be performed with the help of ramp plots (see Fig. 5). Within Design-Expert [
] a desirability score is computed, and the best solutions are shown. It is recommended to look at several solutions to check for robustness of the optimisation (further information on robustness assessment see below).
Fig. 5- Numerical optimisation of the factors (red points) to obtain the optimal values for the responses (blue points) with respect to their prioritisation. The following settings are ideal: around 40.000 cells/well, approx. 0 hours of cell time, approx. 16 hours ligand time, approx. 50 µl of substrate volume and substrate time 7 hours.
The plausibility of the results from the numerical optimisation is assessed by visually evaluating whether the calculated optimum results in the best dose-response curves. If this is not the case, selection of responses needs to be re-considered.
For the given example, the optimisation for each response is performed as follows:
•
Signal/noise ratio: The goal was to maximise the signal/noise ratio, the importance was set to very high (“+++++”).
•
Dynamic range: The goal was to maximise the dynamic range. The importance was set to high (“++++”).
As already described above, restrictions for the factors can be necessary for reasons such as costs, time spans, and practicality. In our example one restriction was that the used substrate is very expensive and thus the substrate volume was set to 50 µL. This restriction led to lower values for the signal/noise ratio and dynamic range. However, this was accepted as being still sufficiently high considering the high costs.
Fig. 5 shows at which settings of the factors (ramps/boxes with red points) the values of the relevant responses (ramps with blue points) are as close as possible to their respective optimum given the prioritisation. The plots with grey points show factor settings for responses which were not included in the optimisation and are therefore for information only.
Robustness check
We assessed the assay robustness by using the data obtained from the DoE and using the desirability functions [
] of each relevant factor. To assess the robustness of the assay, the desirability plots show where the assay performs best and how changes in the settings of the factors affect the optimised assay performance responses.
The desirability plots can be used to define a range for the factors in which the responses are robust with respect to assay performance. Ranges are typically set for incubation times to allow a certain flexibility in the daily lab work. Slight changes in the desirability by using factor settings beside the optimum can be tolerated in most cases. Volumes and cell numbers do not need a range as they are used in fixed values. However, for cells it is recommended to compare a range around the optimum which represents the typical variability of cell counting. This makes sure that typical variance during cell counting in the daily routine does not negatively influence assay performance.
In the example, the desirability for the different factors was calculated based on the settings chosen for numerical optimisation by using the same responses signal/noise ratio and dynamic range (see above).
The Fig. 6 shows the desirability plots for the factors over the whole design space (set points for the best-optimised responses assay performance as red-crosses, compare with numerical optimisation above). The desirability plot for the substrate volume is 0 because of the restriction to 50 µL (see above). With the restriction the best option for substrate volume was excluded leading to a low desirability value. However, as discussed above, this was accepted as in summary desirability was still sufficiently high considering the high costs of the substrate.
Fig. 6- Desirability plots for the investigated factors. The defined ranges are coloured in blue.
Based on the plots, the following ranges for the factors are defined:
•
A fixed cell number of 40,000 cells/ml was chosen. Additionally, results were comparable for approx. +/- 20 % around the optimum.
•
For the pre-incubation time, a time span of 0 hours was optimal. The assay procedure was therefore adapted in a way that no time span for cell incubation time is needed.
•
The desirability for the ligand incubation time showed slightly smaller values at higher time spans with acceptable results for time spans up to 20 hours. Thus, a time span of 16 - 20 hours was defined.
•
For the substrate volume no range is defined and a fixed volume of 50 µl will be used.
•
The desirability for the substrate incubation time showed no relevant difference over the tested range. However, for practical reasons, a time span of 1 - 4 hours is defined.
Verification
The last step is the verification of the results. After the ixDoE is performed, experiments to confirm the detected optima from the ixDoE are recommended. To speed up assay development, it might be possible to spare verification experiments because experiments which are anyway performed (e.g., within method validation) can be used for this purpose. The verification runs are to be compared with the point-prediction computed on the underlying model from the DoE. Due to variance of methods and the resulting uncertainty of the model (a model will always contain a residual error), it is recommended to consider the prediction or tolerance interval for verification of the model.
Discussion
IxDoE, as we have outlined in this paper, is a cost and resource efficient tool to optimise cell-based bioassays whether they are used for drug development or in fields outside of the pharmaceutical industry. The goal of this article is to enable fellow researchers to quickly establish this novel ixDoE approach to easier develop more precise and robust bioassays.
A classical textbook (SCOV/SCOR) approach for the development of a method via DoE requires typically 3-4 full DoEs including a separate planning-, experimental-, and data analysis phase for each DoE (see Fig. 1). [
Besides the complexity of DoE, this substantial workload and material cost is likely to be the main factor causing reluctance in companies and institutions to move away from OFaT and its associated disadvantages [
]. In contrast to a classical DoE approach, ixDoE requires just a single DoE with one planning-, experimental-, and data analysis phase (Fig. 1). This significantly reduces not only the costs for staff and materials but also the development time. Thus, ixDoE should encourage more companies and institutions to move from OFaT towards DoE.
The ixDoE approach is valuable especially for bioassays because a screening DoE can be circumvented as potentially relevant factors can be already identified during assay feasibility phase and based on previous experience. Due to the complex interactions of cellular systems we have observed that often most factors are relevant. Thus, the idea of a pre-selection of factors via a screening DoE does often not bring the expected added value and even bears the risk that relevant factors might be wrongly identified as irrelevant due to the low resolution of screening DoEs. Furthermore, as absolute maxima are not necessarily needed for each response, the characterisation and optimisation stages of a traditional DoE are condensed into one step for ixDoE. Another benefit of the ixDoE approach is that for verification no additional DoE is needed as verification can be included in the experiments performed for assay validation. Finally, information about the robustness of the bioassay can be extracted out of the ixDoE, an important requirement by authorities for analytics performed under GMP [
. In preparation of product launch e.g. for clinical phase III, performance of analytical methods is typically reassessed as during the assay lifecycle adaptations might become necessary. These adaptations might require verification with an additional robustness DoE to ensure assay performance for later market supply. This, however, is also the case for the classical DoE approach.
As there is no single parameter which describes the overall assay performance, we suggest a systematic evaluation of a broad variety of responses (e.g. signal-to-noise, dynamic range etc.) via computation of pair-wise correlations. This guarantees that the most powerful and most relevant responses for assessment of assay performance are selected. In literature the usage of just a single parameter or a selection of several responses without any evaluation is often observed [
Putting a broad variety of responses into consideration has the advantage that all responses which describe the dose response curves best are included in the numerical optimisation and a “sweet spot” can be identified. Such a sweet spot does not necessarily need to be the global optimum for one parameter, but all relevant parameters. Before the verification of the ixDoE (Fig. 1), a compromise has to be found between assay performance and other external limitations like costs, handling time and material needed for the assay.
While ixDoE features many advantages, a main disadvantage of the approach is the initial higher investment to generate the know-how during assay feasibility and the planning phase. To plan and analyze ixDoE an advanced statistical expertise in conjunction with a deep knowledge about bioassays is necessary. Therefore, this paper shall give advice and support also novice scientists in acquiring this knowledge to perform an ixDoE.
Whereas DoE approaches are not new in areas like process development [
] there has been up to now only a scarce number of publications available which deal explicitly with the problems of DoE in the field of potency assays. For non-cell-based potency assays (e.g. ELISA, enzyme-based) there are a few detailed publications [
however, non-cell-based assays are less complex in assay procedure, less laborious with respect to wet-lab activities and planning of DoEs in comparison to cell-based assays. Therefore, the described concepts cannot be easily transferred to cell-based bioassays. Thus, for the more complex cell-based bioassay new concepts are needed to allow guidance on the usage of DoE. Only few existing publications are dealing with DoE for cell-based bioassays. These publications study either DoE only for a single step during development (e.g. assessing robustness [
Implementation of design of experiments (DOE) in the development and validation of a cell-based bioassay for the detection of anti-drug neutralizing antibodies in human serum.
Implementation of design of experiments (DOE) in the development and validation of a cell-based bioassay for the detection of anti-drug neutralizing antibodies in human serum.
which often does not sufficiently describe the quality of the dose-response curve (e.g. dynamic range only). Furthermore, responses like EC50 and relative potency as a final result do not provide information to relevantly assess the quality of the curve and thus should only be considered as additional responses.
] assessed several responses by using a graphical analysis but did not make a regression model. Thus, the advantages of a DoE have not been fully explored, i.e. they did not build a response surface model and did not use a numerical optimization to find an optimum in the multidimensional design space.
In conclusion, we have outlined a best practice for our ixDoE approach with a focus on cell-based bioassays. The approach is also beneficial for non-cell-based assay (e.g. ELISA, blood clotting, enzymatic assays). The presented strategy is a smart approach to obtain the needed information to develop a precise and robust assay. With a similar effort needed for an OFaT-approach, a manifold better assay performance can be achieved with ixDoE. The reason is not only the usage of prior information but also the risk-based principle which optimises only where necessary and beneficial. In addition, further guidance on how to improve DoE is presented, giving input to both statisticians and bioassay experts and allowing the development of better assays in a shorter time.
Author contributions
S.J., E.K. and K.B. conducted the experiments. S.J., E.K., K.B., B.H. designed the experiments, conceptualized and wrote the paper.
Declaration of Competing Interests
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:
Solzin Johannes, Eppler Karoline, Knapp Bettina and Bluhmki Erich are employees of Boehringer Ingelheim. Buchner Hannes is employee and shareholder of Staburo GmbH. No author has a financial conflict of interest related to this work.
Acknowledgments
Solzin Johannes, Eppler Karoline, Knapp Bettina and Bluhmki Erich are employees of Boehringer Ingelheim. Buchner Hannes is employee and shareholder of Staburo GmbH. No author has a financial conflict of interest related to this work.
We thank Anna Hertzog, Daniel Markx, Matthias Kania, Robert Reidy and Beate Presser for their careful review and their valuable input to the paper.
Implementation of design of experiments (DOE) in the development and validation of a cell-based bioassay for the detection of anti-drug neutralizing antibodies in human serum.
13705-6&id=gr1.jpg){kind=link}
13705-6&id=gr2.jpg){kind=link}
13705-6&id=gr3.jpg){kind=link}
13705-6&id=gr4.jpg){kind=link}
13705-6&id=gr5.jpg){kind=link}
13705-6&id=gr6.jpg){kind=link}