# FIT_METHODS

A required section that defines how PoPy estimates the `f[X]` and `r[X]` model parameters given a data file.

The FIT_METHODS section is required in the following scripts:-

- Tut Script
- Fit Script
- MTut Script
- MFit Script

i.e. Any script that estimates PK/PD model parameters is required to have a FIT_METHODS section.

PoPy currently has the following fitting methods:-

- JOE Fitting Method
- FOCE Fitting Method
- ND Fitting Method

Which can be combined using a simple list, see FIT_METHODS Structure.

## FIT_METHODS Structure

The FIT_METHODS section is a list of fitting methods to be applied in order to estimate the `f[X]` and `r[X]` variables of a PK/PD model, for example:-

    FIT_METHODS:
        - <fitter1>: {}
        - <fitter2>: {}
        - <fitter3>: {}

This allows in this case <fitter1> to initialise the `f[X]` for <fitter2> and for <fitter2> to then initialise the `f[X]` for <fitter3> etc..

For example it is recommended that with PoPy, you first apply the JOE fitter method, followed by the FOCE method.

## JOE Fitting Method

JOE stands for Joint Optimisation and Estimation and is PoPy’s approximate equivalent of Nonmem’s ITS (Iterative Two Stage) method. The methods differ in how they update the `f[X]` provided in a Fit Script by the modeller to estimate the final optimised `f[X]`.

However both JOE and ITS both attempt to separately optimise the main `f[X]`, variance `f[X]`, noise `f[X]` and `r[X]` parameters to arrive at a reasonable overall fitted `f[X]` vector. This is in contrast to the FOCE approach which applies a quasi-newton optimisor to the combined `f[X]` vector.

In general JOE is more capable of finding sensible fitted `f[X]` parameters when initialised further away from the true `f[X]` compared to FOCE. However, FOCE is often more accurate when started close to the true solution.

## FOCE Fitting Method

The FOCE and JOE methods both use a first order version of the Laplace approximation to make the computation of the likelihood function tractable. See [Wang2007] for details. We refer to this likelihood as the FOCE ObjV, which was first described in [Lindstrom1990], but is now most commonly associated with the popular FOCE fitting method in Nonmem.

The FOCE method uses a quasi-newton optimisor inter-leaved with a `r[X]` optimiser for each individual. It is the most common fitting method used in PK/PD today. The PoPy implementation of FOCE is similar to Nonmem, but it is not 100% the same.

Note, given the same PK/PD model, data file and same `f[X]` and `r[X]` variables. JOE and FOCE will return the same FOCE ObjV in PoPy. The fitting results are only likely to agree exactly for very simple PK models, but should be similar for more complex cases.

Note FOCE is generally more accurate than JOE when initialised close to the true minima. However it is also capable of falling into false minima away from the global minima. For this reason we recommend using JOE then FOCE when using PoPy, see FIT_METHODS Examples.

The type of optimisation methods used by PoPy are described in references such as [DennisSchnabel1987] [NocedalWright2006].

## ND Fitting Method

The recommended fitting method for PoPy version 1.0.5 is the ND method.

The ND method uses the same ObjV as FOCE. This new approach uses FOCE to estimate the `f[X]` parameters. However when the gradient based FOCE algorithm converges ND switches to using the non-gradient BOBYQA optimisation method. When BOBYQA converges it switches back to the FOCE, until both methods have converged. The ND method is therefore very robust as it utilises two different fitting strategies.

However the ND method will always be slower than FOCE, so for simple cases, or where a quick result, but not necessarily optimal solution is required, it may be preferable to use JOE or FOCE in preference to ND.

In the more common case, where accuracy is required it is recommended to use ND.

## Other Fitting Methods

Note JOE, ITS and FOCE fitting methods are deterministic and fundamentally different from the stochastic sampling methods such as SAEM and IMP. SAEM and IMP that also estimate `f[X]` for PK/PD models, but the stochastic Objective Value is not based on the Laplace approximation used in the FOCE ObjV.

## FIT_METHODS Examples

### Fit using ND

The recommended fitting method for PoPy version 1.0.5 is the ND method.

The ND method can be implemented as follows:-

    FIT_METHODS: [ND:{}]

Or alternatively:-

    FIT_METHODS:
        - ND: {max_n_main_iterations: 30}

### Fit using JOE and FOCE

Another way to fit PK/PD models in PoPy is to use JOE followed by FOCE in a Fit Script is as follows:-

    FIT_METHODS: [JOE:{}, FOCE:{}]

This single line, runs the JOE fitting method followed by FOCE with the default settings. Note that the square bracket notation ‘[]’ is required because ‘FIT_METHODS’ expects a list. Or equivalently:-

    FIT_METHODS:
        - JOE: {}
        - FOCE: {}

Where the ‘-‘ is YAML notation for a list item.

One of the simplest JOE and FOCE parameters is ‘max_n_main_iterations’:-

    FIT_METHODS:
        - JOE: {max_n_main_iterations: 30}
        - FOCE: {max_n_main_iterations: 30}

Which sets the number of times the `f[X]` parameters are updated. The JOE fitting proceeds iteratively with interleaved `f[X]` and `r[X]` updates, followed by FOCE starting from the final `f[X]` result returned by JOE. Setting this limit forces PoPy to stop after a finite number of iterations.

If you miss out ‘max_n_main_iterations’ it defaults to 50. Another option is as follows:-

    FIT_METHODS:
        - JOE: {max_n_main_iterations: 0}

This only updates the `r[X]` parameters and leaves the `f[X]` fixed.

## Fitting Convergence Criteria

Another common setting is the ‘CONVERGER’ field. There are three possible settings:-

- NONE - no convergence termination - estimation stops when ‘max_n_main_iterations’ is reached
- OBJ_INC - terminate convergence if the ObjV increases
- OBJ_SIM - terminate convergence if the ObjV is similar between iterations

These settings decide when a Fit Script will return the final `f[X]` estimates.

The default setting for JOE and FOCE fitting is ‘OBJ_INC’, hence:-

    FIT_METHODS:
        - JOE:
            max_n_main_iterations: 100
        - FOCE:
            max_n_main_iterations: 100

Is the same as:-

    FIT_METHODS:
        - JOE:
            max_n_main_iterations: 100
            CONVERGER: {OBJ_INC: {}}
        - FOCE:
            max_n_main_iterations: 100
            CONVERGER: {OBJ_INC: {}}

And the fit estimation will stop as soon as any iteration increases the ObjV. To allow the ObjV to increase during fitting, i.e. if the likelihood gets worse between iterations, but you wish the search to continue, use:-

    FIT_METHODS:
        - JOE:
            max_n_main_iterations: 100
            CONVERGER: {OBJ_SIM: {}}
        - FOCE:
            max_n_main_iterations: 100
            CONVERGER: {OBJ_SIM: {}}

Here ‘OBJ_SIM’ will terminate the estimation when two consecutive iterations return similar objective values (within a relative tolerance of 1e-06).

Note that when fitting complex models the ObjV usually decreases at each iteration, but occasionally increases. This is due to fact that `f[X]` and `r[X]` are optimised separately. Separate optimisation is necessary for computational efficiency. However optimising components of the likelihood independently can mean the combined ObjV increases. The ‘OBJ_SIM’ option above is designed to allow the search to continue in these cases.