VNM : An R Package for Finding Multiple-Objective Optimal Designs for the 4-Parameter Logistic Model

A multiple-objective optimal design is useful for dose-response studies because it can incorporate several objectives at the design stage. Objectives can be of varying interests and a properly constructed multiple-objective optimal design can provide user-speciﬁed eﬃciencies, delivering higher eﬃciencies for the more important objectives. In this work, we introduce the VNM package written in R for ﬁnding 3-objective locally optimal designs for the 4-parameter logistic (4PL) model widely used in education, bioscience and in the manufacturing industry. The package implements the methodology to construct multiple-objective optimal designs in Hyun and Wong (2015). As illustrative examples, we focus on a biomedical application where our objectives are to estimate: (1) the shape of the dose-response curve, (2) the median eﬀective dose level (ED50) and (3) the minimum eﬀective dose level (MED) in the 4PL model. Our VNM package uses a state-of-the-art algorithm to generate multiple-objective optimal designs that meet the user-speciﬁed eﬃciency requirement for each objective, provides tools for calculating the eﬃciency of the generated design under each objective and also a plot for conﬁrming optimality of the VNM -generated design. The package can also be used to determine an optimal scheme for allocating subjects to the various doses when the number and doses of the drug are ﬁxed in advance.


Introduction
Optimal designs are getting more attention nowadays because of rising experimental cost and the desire to use minimum resources without sacrificing statistical precision in the inference making process. Traditionally, optimal designs are developed under one objective or criterion and they are found either analytically by theory or numerically via an algorithm. In real studies, there are usually more than one objective in a study and researchers often want to incorporate multiple objectives into a single study to save resources. The objectives typically have different levels of interest in the study and it is desirable to have a multiple-objective optimal design that provides user-specified efficiency under each criterion, with higher efficiencies for the more important objectives.
Numerous researches have attempted to tackle design problems with multiple objectives. Almost all focus on a two-objective design problem (Stigler 1971;Lauter 1974;Lee 1988;Dette 1992;Cook and Wong 1994;Huang and Wong 1998;Song and Wong 1999;Tsai and Zen 2004;Atkinson 2008;Leonov and Miller 2009;Padmanabhan and Dragalin 2010;Zhang, Wong, and Peng 2012). Some used ad-hoc or numerical methods and a few found dual-objective optimal designs using theory. Several researches mentioned that their method can be extended to three or more objectives but do not provide concrete examples. From a technical viewpoint, dual-objective optimal designs are relatively easier to find compared with finding an optimal design for three or more objectives. This may explain why the statistical literature has many dual-objective optimal design problems and virtually none on the construction of a three or more objective optimal design over a continuous design space with a concrete application. A limited number of 3-objective optimal designs has recently shown up in a non dose-response set up where the design space is discrete. One example is in determining an optimal adaptive allocation scheme for subjects to treatment groups under three or more criteria in a clinical trial; see, for example, Antognini and Zagoraiou (2012).
Currently, the availability of tools for finding multiple-objective optimal designs is limited. None of the commercial statistical software packages provides such capability. There is an R package DoseFinding (Bornkamp, Pinheiro, and Bretz 2016) that provides a function optDesign for searching dual-objective optimal designs when both objectives are equally important. One objective is to estimate model parameters and the other objective is to estimate the target dose, such as the ED100P or the MED. The ED100P is the dose that produces a response rate of 100P% and the MED is the minimum effective dose. The function searches the dual-objective optimal designs for a given set of dose-response models.
Multiple-objective optimal designs are especially appealing in a dose-response study. Researchers often want to estimate the shape of dose-response function and want to study selected properties of the agent at the same time. Given the multiple objectives, a multipleobjective optimal design can help to identify the optimal number of dose levels to use, where these optimal dose levels are and the optimal distribution of subjects over the selected dose levels for attaining the objectives most efficiently. Hyun and Wong (2015) recently constructed optimal designs for estimating three interesting features in a 4-parameter logistic model and each of these objectives may have different degrees of interest to the researcher. Their approach is systematic and can be directly applied to search for other types of multiple-objective optimal designs in other models.
We describe here our VNM package that implements the method from Hyun and Wong (2015) for generating one, two or three-objective optimal designs for the 4-parameter logistic (4PL) model. This package is available from the Comprehensive R Archive Network at https://CRAN.R-project.org/package=VNM. We chose the 4PL model because it is a model frequently used to describe sigmoidal curves in several disciplines such as in the pharmaceutical sciences, educational testings and in the manufacturing industries. Further, the 4PL model becomes the widely used 2-parameter logistic (2PL) model or 3-parameter logistic (3PL) model by setting some of the model parameters equal to specific values. We focus on designing an efficient dose-response trial to explore response relationship with the dose levels (Holford and Sheiner 1981;MacDougall 2006;Goutelle, Maurin, Rougier, Barbaut, Bourguignon, Ducher, and Maire 2008). After the user specifies the efficiencies sought for the various objectives, our VNM package provides functions to (i) search for the multipleobjective optimal designs, (ii) evaluate the efficiency of the multiple-objective optimal design under each objective and (iii) provide a graphical tool to confirm optimality of the generated design. In addition, when the dose levels in a dose-response study are fixed, our software can also determine the optimal number of subjects to assign to the various dose levels under one of the three criteria.
Section 2 describes the statistical background, model and optimality criteria. In Section 3, we describe our algorithm and show how our VNM package may be used to obtain the multipleobjective optimal designs. We also provide applications to find a multiple-objective optimal design for a dose-response study. Section 4 contains a summary.

Background
Suppose we have resources to take a fixed total number of n independent observations for the study. Assume further that Y i1 , . . . , Y in i are independent responses at dose level x i , i = 1, . . . , K, from the statistical model The function f (x i , Θ) is assumed to be known and differentiable, apart from the vector of unknown model parameters Θ. Furthermore, assume that σ 2 is finite and unknown. Design questions concern the choice of K, the number of different doses required, and the values of each x i and n i for i = 1, 2, . . . , K. Obviously, the optimal choices depend on the given design criterion, the statistical model and the range of plausible doses for the study.
The mean response in the 4PL model at x is where x is the dose in log scale selected from a given dose interval. A common interest in drug studies is to estimate the ED50 of the drug, which is the dose expected to produce an effect mid-way between the expected responses at the extreme doses. This model is a transformation of the 4-parameter Hill's model (Khinkis, Levasseur, Faessel, and Greco 2003) commonly used in biochemistry where x denotes the actual dose. The two models are essentially equivalent; if a researcher works with the log dose, the 4PL model is appropriate, otherwise, the 4parameter Hill model is more appropriate when x is the actual dose (Dinse 2011;Reeve and Turner 2013;Hyun and Wong 2015). The 4PL model has an advantage over the Hill model in that it yields more stable parameter estimates. We may interpret the parameters in (2) as follows: θ 1 is the difference between the expected responses at the two extreme doses (upper limit minus lower limit, sometimes called EMAX), θ 2 is the negative of the Hill's coefficient in the Hill's model that controls the rate of change in the response, θ 3 is −θ 2 log(ED50) and θ 4 is the lower limit of the response when θ 1 > 0, θ 2 = 0, and −∞ < ED50 < ∞. The model becomes the 2PL model when θ 1 = 1 and θ 4 = 0 or the 3PL model when θ 4 = 0.
In what is to follow, we work with continuous designs. Suppose n subjects are available for a dose response study with K dose levels at x i and n i subjects are assigned at x i , i = 1, . . . , K.
The continuous design is given by ξ = {(x i , w i ) } K 1 , where w i = n i /n and n 1 + . . . + n K = n. In practice, n is predetermined by either cost or the number of subjects available for the study. The continuous design is specified without regard to n and this simplification provides us a general framework to find an optimal continuous designs for a nonlinear model. The main advantages of working with continuous designs are that there are algorithms for finding several types of optimal continuous designs and there is a direct way of confirming if a continuous design is optimal or not using an equivalence theorem discussed below. Additional details are available in design monographs such as Atkinson, Donev, and Tobias (2007) and Fedorov and Leonov (2013).
Following convention, the worth of a design is measured by its Fisher information matrix. This matrix obtained from using the continuous design ξ is where 1 . This information matrix plays an important role in the formulation of the design criterion to search the multiple-objective optimal designs. We assume nominal values for Θ are available and so our constructed designs are locally optimal. For simplicity, we write g(x, Θ) = g(x).
To fix ideas, we suppose the three objectives in our dose response study are to estimate (i) the shape of the dose-response curve, (ii) the ED50, the dose that produces an expected response midway between the two extreme responses, and (iii) the minimum effective dose MED that would result in a user-specified change in the anticipated response. For the first objective, a D-optimal design is appropriate because such a design minimizes the volume of the confidence ellipsoid for the model parameters. For the second and third objectives, the quantities of interest are functions of the model parameters and so a c-optimal design that minimizes the asymptotic variance of each of the estimates is appropriate. Accordingly, we use c-optimal designs to estimate our two target doses, the ED50 and the MED, which, as we show below, are functions of the model parameters. In all cases, the minimization is over all designs on the given dose interval.
Under model (2), an explicit expression for the ED50 can be found directly from its definition: Since the explicit form of the ED50 does not contain θ 1 and θ 4 , the ED50 remains the same form under the 2PL or the 3PL models. For MED, we used the definition from Padmanabhan and Dragalin (2010) as the dose that produces a mean response of δ units better than the minimum dose. Here, δ is the clinically significant effect predetermined by the researcher. It follows that the explicit expression for MED in model (1) is given by The MED remains the same under the 3PL model but the MED under the 2PL model is different and can be obtained by setting θ 1 = 1 in the formula. If ED50 is the first derivative of ED50 with respect to Θ and a design ξ is used, the asymptotic variance of the estimated ED50 is proportional to . Similarly, if MED be the first derivative of MED with respect to Θ and a design ξ is used, the asymptotic variance of the estimated MED is proportional to Var( MED)| ξ = MED I − (ξ;Θ) [MED ] . Let ξ D be the D-optimal design for estimating Θ, let ξ ED50 be the c-optimal design for estimating the ED50 and let ξ MED be the c-optimal design for estimating the MED. All are locally optimal designs because they depend on the nominal values of Θ. The efficiencies of a design ξ under these criteria are respectively given by: We call the above efficiencies the D-efficiency, ED50-efficiency and MED-efficiency of the design ξ, respectively. Each efficiency is clearly between 0 and 1 and if e(ξ) = c, this means that the design ξ needs 100(1/c − 1)% more observations to provide the same accuracy for estimating the quantity of interest as the optimal design. For instance, if a design ξ is about as efficient as the D-optimal design ξ D for estimating Θ, then e 1 (ξ) is close to 1; if this number is one half, it means the design ξ has to be replicated twice to perform as well as the optimal design. In practice, designs with high efficiencies across several criteria are sought.
Because an optimal design under one objective can perform poorly under another objective, we need to compromise on the competing objectives and search for the multiple-objective optimal design that appropriately balances the various needs. One way to do this is to find an appropriate compound optimal design that optimizes a convex combination of the criteria (Cook and Wong 1994) and another way is to find a design that maximizes an appropriate product of the various efficiencies under the different criteria (Atkinson et al. 2007, Ch. 21). For the latter approach, the criterion is to find a design that maximizes Ψ = λ 1 log(e 1 (ξ)) + λ 2 log(e 2 (ξ)) + λ 3 log(e 3 (ξ)) where s is the number of parameters in the mean function and each λ 1 , λ 2 and λ 3 is preselected, non-negative and 3 i=1 λ i = 1. A larger value of λ i implies that the i-th criterion is relatively more important than the others. When λ i = 1, the problem becomes one for finding the optimal design for the i-th criterion only.
The general equivalence theorem is a powerful theoretical tool in optimal design theory that can be used to find and confirm the optimality of a design when the design criterion is a convex or concave function of the information matrix. To apply this theorem for our setup, we first calculate the directional derivative function of (4) evaluated at the generated design ξ in the direction of an arbitrary degenerate design at x to obtain the sensitivity function: Design ξ e 1 (ξ) e 2 (ξ) e 3 (ξ) The general equivalence theorem states that, for a given vector λ = (λ 1 , λ 2 , λ 3 ), the multipleobjective optimal design ξ M,λ optimizes (4) if and only if for all x in the dose interval, with equality at the dose levels of ξ M,λ . The sensitivity function contains a generalized inverse of the information matrix of ξ and plays an important role in the search and verification of the multiple-objective optimal design.
Example 1 We use the information from Miller, Guilbaud, and Dette (2007) and construct a 3 objective optimal design for the 4PL model using (4) and compare its performance with the locally D-optimal design and a dual-objective optimal design for estimating Θ and the ED50. These optimal designs are obtained using our VNM package by setting in (4) (i) λ 1 = 1 to obtain the single objective D-optimal design, (ii) λ 1 = λ 2 = 0.5 to obtain the dual-objective optimal design ξ Dual and (iii) λ 1 = λ 2 = λ 3 = 1/3 to obtain the 3-objective optimal design. The implementation details for our package are given in the next section and we note that the single and dual-objective optimal designs can also be obtained using the DoseFinding package in (Bornkamp et al. 2016). Table 1 shows the efficiencies of the 3 types of optimal designs under each of the 3 objectives. As expected, ξ D has D-efficiency equal to 1 but does not perform well for estimating the ED50 and MED, more so for the latter when it has only about 48% efficiency for estimating the MED. The dual-objective optimal design ξ Dual provides balanced and high efficiencies for estimating Θ and the ED50 but has an efficiency of only 37% for estimating the MED. The multiple-objective optimal design ξ M,λ compromises its overall efficiencies for the study by improving the efficiency for estimating the MED beyond that provided by the D-and dual-objective optimal designs by giving up slightly its efficiencies under the first two criteria. Further improvement in the desired efficiencies may be possible by an appropriate choice of the weight vector λ. This shows the flexibility and usefulness of a multiple-objective optimal design.

The VNM package
In this section, we describe the VNM package for finding 3-objective optimal designs to estimate the shape of dose-response, the ED50 and the MED for the 4PL model. The acronym VNM stands for V-algorithm using Newton Raphson method to search multiple-objective optimal design. The package contains several functions useful for designing a dose-response study using the 4PL model. The package also can be used for the 2PL or the 3PL models by assigning specific values to some of the model parameters in the 4PL model. The function MOPT finds and verifies that the generated design is the multiple-objective optimal design. The functions Deff, ceff1 and ceff2 compute efficiencies of the generated design under the three objectives and the function S.Weight computes the optimal proportions of subjects for problems when the number and dose levels are fixed. The latter is frequently used in situations when the drug under investigation is only available in few fixed dose levels and the problem then is how to assign the subjects optimally to the dose levels under a given design criterion.
Because there are currently no effective algorithms for generating multiple-objective optimal designs, we developed the VNM package using a state-of-the-art algorithm for finding them. Specifically, we first obtained the SAS/IML (SAS Institute Inc. 2004) code from Yang, Biedermann, and Tang (2014), who had recently shown their algorithm can theoretically converge to many types of single-objective optimal designs and also outperform current algorithms for finding single-objective optimal designs. We recoded their SAS/IML codes into R codes because R is open source and the research community at large can use it freely to generate various types of multiple-objective optimal designs. Finding multiple-objective optimal designs is a much harder task than finding single-objective optimal designs because more variables have to be optimized and the criterion is more complex. Our carefully modified code into R code based on the Yang's algorithm was able to find the multiple-objective optimal designs quickly but on rare occasions, it was unable to do so. To overcome the occasional problem, we innovate an extra step A1 in our algorithm to provide a better initial design for the search and this seems to have resolved the issue. The VNM package also includes functions to evaluate the efficiencies of the generated design under the various single-objective criteria and provide plots to confirm optimality of the generated designs. Additionally, the package is also able to determine the optimal proportions of observations at a given number of doses. Currently, our package is limited to search and evaluate multiple-objective optimal designs under a user-specified model with three specific objectives but the package can be easily modified to search for multiple-objective optimal designs for other models and have a different number of user specified objectives.

A modified new algorithm for the multiple-objective optimal design
The VNM package requires to discretize the dose interval first. The default is a grid density with points 0.01 units apart, but other choices are allowed by supplying a different value in the argument of the function used to find the multiple optimal design. We denote the set of generated dose levels at the tth iteration by X t and the unknown proportions of subjects at the first K − 1 dose levels by ω t , respectively, where ω = (w 1 , w 2 , · · · , w K−1 ) , w i ≥ 0 and K−1 i=1 w i ≤ 1. We also denote the generated design at the tth iteration by ξ t , the optimality criterion (4) and the sensitivity function (5) evaluated at ξ t by Ψ(ξ t ) and d(x, ξ t ), respectively. The steps in the main algorithm below are prefixed by the letter A and the steps for a component of the algorithm performed by the Newton Raphson method are prefixed by the letter N for improved clarity.
First, nominal values for the model parameters in Θ are required to initiate the code. These may be available from a pilot study or using information from similar drugs. Alterna-tively, information on the value of a related vector of parameters is first sought before backtransforming them to the actual nominal value for each parameter. For example, Miller et al. (2007) and Bretz, Dette, and Pinheiro (2010) both used the 4-parameter logistic model to study responses from their studies and solicited information on nominal values for the vector P = (p 1 , p 2 , p 3 , p 4 ), where p 1 is the lower limit of the response, p 2 is EMAX, p 3 is the ED50 and p 4 is the slope at the ED50. The rationale is that this can be a more practical way to elicit likely values for the transformed parameters since they are more meaningful and more easily available than the value of the parameter vector Θ.
The algorithm below has several tuning parameters to refine the search or change various stopping rules. For instance, different grid sizes on the dose space may be needed (i.e., setting the grid size equal to 0.001 or 0.0001), different values of r may be needed to select the initial dose levels in (A1), or different values of ε and ε ω to stop the whole algorithm in (A5) or just terminate one component of the algorithm performed by the Newton Raphson procedure in (N4). Input values for these parameters are required to start the algorithm. In the VNM package, the grid size, r, ε and ε ω are denoted by grid, r, epsilon and epsilon_w respectively. For convenience, we provide default values for these parameters: grid = 0.01, r = 10, epsilon = 10 −3 , and epsilon_w = 10 −6 . These default values should be changed when they do not produce the optimal design. For example, our experience is that the algorithm may sometimes fail to find the multiple-objective optimal design with r = 10, where r is the number of iterations to select the initial dose level to run the new algorithm. For these cases, we increased its value to r = 30 or r = 50 and the algorithm was able to generate the multiple-objective optimal design.
The algorithm includes the V-algorithm by Fedorov, Klimko, and Studden (1972) as a component and runs sequentially as follows after the above inputs are supplied: (A1) Run the V-algorithm using the sensitivity function (5) r times and select the last s + 1 generated dose levels as the initial dose levels. The selected initial dose levels become X 1 .
(A2) Apply the Newton Raphson method to determine the optimal proportions of subjects ω t for X t . For X t , assign initial proportions of subjects ω t 0 randomly. Then, the proportions of subjects at the lth iteration ω t l are updated as follows (starting with τ = 0.2): (N3) Reduce τ to τ /2 and repeat steps N1 and N2 until τ reaches the small value 10 −4 . If there are still nonpositive proportions of subjects in ω t l , remove the dose level with the smallest proportion of subjects. Then go to Step N1 with the new design (i.e., the new design is the one that removes the dose level with the smallest proportion of observations from ξ t ).
(N4) For a pre-selected small positive value ε ω , if ||ω t l − ω t l−1 || < ε ω , ω t l becomes the optimal proportions of subjects ω t for the selected dose levels X t . Otherwise, go to Step N1 and update the proportions of subjects for the next iteration.
(A3) Evaluate d(x, ξ t ) based on all dose levels in the give dose interval and select the dose x * t that maximizes d(x, ξ t ).
(A6) Otherwise, set X t+1 = X t ∪ {x * t } and repeat steps A2 to A5 after setting t = t + 1. In Step A1, we have modified one part of the algorithm proposed in Yang et al. (2014). In their algorithm, a set of s + 1 uniform dose levels is used as the initial dose levels but we modify this part to select better initial dose levels through the V-algorithm. If the selected initial dose levels are far from the optimal dose levels, the algorithm will take longer to find the optimal design. Sometimes it fails to find the optimal designs, especially when a small weight is assigned to the first criterion (i.e., λ 1 = 0.05 or λ 1 = 0.1) in searching for the multiple-objective optimal designs. The modified part (A1) in our algorithm solves this problem. For finding multiple-objective optimal designs for different objectives under different dose-response model, the Fisher information matrix (3) and the sensitivity function (5) need to be modified. As long as the dose response model is a non-linear function of parameters and the optimality criteria for the objectives are the convex functions, the above algorithm can be easily modified and works well for searching for the multiple-objective optimal designs.

The functions in the VNM package
The VNM package can be obtained and loaded in R using the following commands: R> install.packages("VNM") R> library("VNM") In this section, we use two real-data examples to demonstrate how to use the key functions in the VNM package to generate multiple-objective optimal designs using the modified new algorithm. The reference manual for the package provides the detailed descriptions of the arguments required to run the functions.
Our first example is taken from Bretz et al. (2010) and the second is taken from Miller et al. (2007). Bretz et al. (2010) constructed efficient designs to estimate target doses of interest such as MED and the ED100P. Several dose response models were considered for a real Phase II clinical study for asthma indication, and one of the models is the 3-parameter EMAX model. This model is the same model as (2) when we set θ 2 = −1. The other example comes from Miller et al. (2007), who explored various design options for a phase IIB trial conducted by AstraZeneca. The 4PL model (called EMAX-sigmoid model in their paper) was the model of choice because of its ability to accommodate different dose-response patterns.
In the VNM package, different argument length of P will make all the functions run different models. The functions run the 4PL model when all four nominal values in P are specified: P = (p 1 , p 2 , p 3 , p 4 ). For the 3PL model, the user needs to specify only three nominal values, Emax, ED50, and the slope: P = (p 2 , p 3 , p 4 ). For the 2PL model, the user needs to specify only two nominal values, ED50 and the slope: P = (p 3 , p 4 ).

The function MOPT
The MOPT function maximizes the optimality criterion (4) and verifies the optimality of the generated design using the general equivalence theorem by producing a graphical plot of the function in (5). The user can then determine from the plot whether conditions in the equivalence theorem are satisfied or not. If they are, the generated design is optimal, otherwise it is not. The synopsis of the MOPT function is: MOPT(LB, UB, P, lambda, delta, r = 10, grid = 0.01, epsilon = 1e-3, epsilon_w = 1e-6, verbose = FALSE) To run this function, the following minimum number of arguments are required: the lower and the upper bounds of the dose interval in log scale, [LB, UB]; nominal values for the parameters P; the user-selected weights for the first two objectives, lambda = (λ 1 , λ 2 ); and the clinically significant effect defined in the MED, delta = δ. Running the MOPT function provides the computed maximum value of |d(x, ξ t ) − 1| at each iteration from the algorithm, which stops if the maximum value of |d(x, ξ t ) − 1| is less than ε at the tth iteration. The obtained multipleobjective optimal design ξ M and the plot of the sensitivity function d(x, ξ M ) over the given dose interval that confirms the optimality of the obtained design ξ M are provided by using the summary and the plot functions, respectively.
As an illustrative example, suppose we want to search the 3-objective optimal design for the 4PL model and the objectives are as previously mentioned. The first step is to input the design parameters as ordered arguments into the function to search for the multiple-objective optimal design. In Example 2, we can see that |d(x, ξ 3 ) − 1| < ε = 10 −3 is achieved within three iterations. Thus, ξ 3 is the multiple-objective optimal design that maximizes the product of efficiencies for the three objectives when λ = (1/3, 1/3, 1/3). Similarly, Example 3 shows that two iterations were required to determine that ξ 3 is the multiple-objective optimal design when λ = (1/3, 1/3, 1/3). The summary function shows the multiple-objective optimal design ξ M with its dose levels displayed in the first row after the label X and the corresponding proportions of subjects displayed in the second row after the label W. The plot function shows the verification plot (Figure 1). The plot is bounded above by unity over the given dose interval with equality at the dose levels of the ξ M and this confirms optimality of the design. We note that if convergence did not occur, we would increase the number of iterations allowed by increasing the value of r in the argument. Sometimes convergence can be achieved by reducing the default values for the grid density through changing grid. In practice it may be difficult to supply suitable values for the weights λ 1 and λ 2 to be used in the combined design criterion. The next function can be used to provide informed values of λ 1 and λ 2 to use in the function by computing the resulting efficiencies of the generated design under the three criteria.

The functions Deff, ceff1 and ceff2
The functions Deff, ceff1, and ceff2 evaluate the efficiencies of the generated design ξ for estimating the shape of dose-response, ED50 and MED respectively. The Deff function computes e 1 (ξ), the efficiency of ξ for estimating the model parameters relative to the locally D-optimal design. Similarly, the function ceff1 computes e 2 (ξ), the efficiency of ξ for estimating the ED50 relative to the c-optimal design for the ED50, and the function ceff2 computes e 3 (ξ), the efficiency of ξ for estimating the sought MED relative to the c-optimal design for the MED. These efficiencies enable the user to increase or decrease the value for λ 1 or λ 2 to obtain the multiple-objective optimal design that meets the user specified efficiency requirements. The synopses are: Deff(weight, P, dose, LB, UB, r = 10, grid = 0.01, epsilon = 1e-3, epsilon_w = 1e-6) ceff1(weight, P, dose, LB, UB, r = 10, grid = 0.01, epsilon = 1e-3, epsilon_w = 1e-6) ceff2(weight, P, dose, LB, UB, delta, r = 10, grid = 0.01, epsilon = 1e-3, epsilon_w = 1e-6) As before, the user needs to input the design parameters as ordered arguments into these functions. For example, to check whether the design found from the function MOPT is the sought multiple-objective optimal design, we input the selected dose levels, the corresponding proportions of subjects in ξ M , dose and weight, and the values for [LB,UB], P, and delta in the function ceff2 for the desired MED. Running these 3 functions now produces the 3 types of efficiencies of the design found from the function MOPT: Example 2 (continued) The three implemented functions use the multiple-objective optimal design found in Example 2 and produce the following results: R> Res. 0.156,0.378,0.195 -ceff1(weight = c(0.271, 0.156, 0.378, 0.195 -ceff2(weight = c(0.271, 0.156, 0.378, 0.195 (100) The above two examples show that the functions Deff, ceff1 and ceff2, along with the summary function, provide the single-objective optimal design and the efficiency of the entered design under each criterion. For instance, in the first example, summary(Res.Deff) shows the D-efficiency of the generated multiple-objective design ξ M for estimating the model parameters is 0.917. Likewise, the results from the functions ceff1 and ceff2 can be similarly interpreted.
In practice, the values for λ 1 and λ 2 are chosen so that the efficiencies of the generated design meet the user-specified efficiencies for the 3 objectives. In general, we expect a larger value of the weight, say λ 1 , used in the combined criterion will result in the generated design having a higher D-efficiency and conversely. To meet the pre-specified efficiency requirement for each criterion, the desired multiple-objective optimal design can be found by choosing the weights λ 1 and λ 2 iteratively until the target is met. In particular, the function MOPT generates the multiple-objective optimal design and the functions Deff, ceff1 and ceff2 determine their efficiencies under the three objectives.
The next function is useful when the number of doses and the levels of the dosage are fixed in a dose-response study using the 4PL model. Given fixed dose levels, the optimal allocation of subjects to each dose for studying the three objectives are obtained.

The function S.Weight
The S.Weight function finds the optimal proportions of subjects at the given doses to estimate the shape of dose-response, ED50, and the sought MED simultaneously. The function does so by maximizing the optimality criterion (4) at the given dose levels using the Newton Raphson algorithm described in the previous section. Specifically, the S.Weight function determines the optimal proportions of subjects (w * 1 , w * 2 , . . . , w * k ) for the fixed dose levels (x 1 , x 2 , . . . , x k ), and implements the design by rounding each w * i n to the nearest integer n i and assigning them to log dose x i subject to k i=1 n i = n. The synopsis of the S.Weight function is: S.Weight(X, P, lambda, delta, epsilon_w = 1e-6) The function S.Weight requires the following inputs to start the calculation: the fixed dose levels X = (x 1 , x 2 , . . . , x k ), P, lambda and delta. As mentioned earlier, ε ω = 10 −6 is the default value for the stopping criteria but the user can change it. For given user-selected weights for the three objectives, S.Weight identifies the optimal proportions of subjects for the given dose levels that maximizes the product of the three efficiencies, and verifies the result.
Example 4 Suppose we wish to determine the optimal distribution of observations or subjects at a fixed set of dose levels given by X = (−6.91, 2.99, 4.96, 6.21).

R>
The first derivative of the criterion: 3.117397e-05 -8.224572e-07 -5.821497e-07 The second derivative of the criterion: -12. 70619 -12.2149 -11.4482 As shown in the above example, the summary function of the S.Weight result gives two sets of output: the first set shows the generated proportions of subjects for the multiple objective optimal design at the given set of doses when we weight the three objectives equally (i.e., λ 1 = λ 2 = 1/3). We note that these optimal proportions of subjects are identical to the ones obtained from the MOPT function since they have the same dose levels. The second part of the output verifies optimality by checking whether the equivalence theorem is satisfied. Specifically, it shows the generated proportions of subjects that maximize the criterion (4) by reporting the first derivative of the criterion ∂Ψ(ξ) ∂ω i at the dose levels (should be close to zero), and the second derivative of the criterion ∂ 2 Ψ(ξ) ∂ω i ω j at the given dose levels (should be negative).

Summary
We develop the user-friendly R package VNM that uses a state-of-the-art algorithm for generating multiple-objective optimal designs to estimate (1) the shape of the dose-response curve, (2) the median effective dose ED50 and (3) the minimum effective dose level MED in the 4PL model. The package also determines the optimal proportions of subjects to be assigned to the given set of dose levels for the three criteria of interest. We hope that this package helps practitioners find a multiple-objective optimal design to balance the competing demands in the objectives and implement a more informed design.
In practice, practitioners first prioritize the importance of the various objectives and specify efficiencies that the implemented design should deliver under the various criteria. Naturally, the sought design should yield higher efficiencies for the more important objectives. For example, the user may want the design to provide at least 80% efficiencies for estimating both the ED50 and the MED (signifying equal interest in the two objectives), and subject to these requirements, does as good as possible in estimating the model parameters. Second, it must be determined which set of weights generates the design that has the specified efficiencies for the 3 objectives. To do this, one can repeatedly apply the algorithm with different sets of weights and find a generated design that meets the efficiency requirements for all the objectives. This is generally a laborious task even though the time required to determine the sought multiple-objective optimal design is now much reduced using our current package. An alternative and likely more effective way of using this package to generate multiple-objective optimal designs is described in Hyun and Wong (2015). Of course, if the demands are too stringent, a multiple objective optimal design may not be found.
The codes we provide here are specific for the 2PL, 3PL and 4PL models but that they can be readily modified to find other types of optimal designs using different statistical models for other applications. It can also be directly modified to find a multiple-objective design when there are four or more criteria. Similarly, Bayesian optimal designs can also be found directly from our approach once a routine for computing the integration is included. In either case, the directional directive of the compound criterion can be similarly calculated and evaluated at the generated design to determine its optimality via the general equivalence theorem.
Zhang's help to build the package. We are grateful to the editors as well as the referees for their helpful comments and suggestions.