.. _fit_methods:

FIT_METHODS
##############

A required section that defines how |popy| estimates the |fx| and |rx| model parameters given a |data_file|. 

The |fit_methods| section is required in the following scripts:-

* :ref:`tut_script`
* :ref:`fit_script`
* :ref:`mtut_script`
* :ref:`mfit_script`

|ie| Any script that estimates |pkpd| model parameters is required to have a |fit_methods| section.

|popy| currently has the following fitting methods:-

* :ref:`fit_methods_joe`
* :ref:`fit_methods_foce`
* :ref:`fit_methods_nd`

Which can be combined using a simple list, see :ref:`fit_methods_structure`.

.. _fit_methods_structure:

FIT_METHODS Structure
=======================

The |fit_methods| section is a list of fitting methods to be applied in order to estimate the |fx| and |rx| variables of a |pkpd| model, for example:-

.. code-block:: pyml

    FIT_METHODS:
        - <fitter1>: {}
        - <fitter2>: {}
        - <fitter3>: {}

This allows in this case <fitter1> to initialise the |fx| for <fitter2> and for <fitter2> to then initialise the |fx| for  <fitter3> |etc|.

For example it is recommended that with |popy|, you first apply the |joe| fitter method, followed by the |foce| method.

.. _fit_methods_joe:

JOE Fitting Method
=====================

|joe| stands for |joe_long| and is |popy|'s approximate equivalent of |nonmem|'s |its| (|its_long|) method. The methods differ in how they update the |fx| provided in a :ref:`fit_script` by the modeller to estimate the final optimised |fx|. 

However both |joe| and |its| both attempt to separately optimise the main |fx|, variance |fx|, noise |fx| and |rx| parameters to arrive at a reasonable overall fitted |fx| vector. This is in contrast to the |foce| approach which applies a quasi-newton optimisor to the combined |fx| vector.
  
In general |joe| is more capable of finding sensible fitted |fx| parameters when initialised further away from the true |fx| compared to |foce|. However, |foce| is often more accurate when started close to the true solution.
  
.. _fit_methods_foce:

FOCE Fitting Method
==========================

The |foce| and |joe| methods both use a first order version of the Laplace approximation to make the computation of the likelihood function tractable. See [Wang2007]_ for details. We refer to this likelihood as the |foce| |objv|, which was first described in [Lindstrom1990]_, but is now most commonly associated with the popular |foce| fitting method in |nonmem|.

The |foce| method uses a quasi-newton optimisor inter-leaved with a |rx| optimiser for each individual. It is the most common fitting method used in |pkpd| today. The |popy| implementation of |foce| is similar to |nonmem|, but it is not 100% the same.

Note, given the same |pkpd| model, |data_file| and same |fx| and |rx| variables. |joe| and |foce| will return the same |foce| |objv| in |popy|. The fitting results are only likely to agree exactly for very simple |pk| models, but should be similar for more complex cases. 

Note |foce| is generally more accurate than |joe| when initialised close to the true minima. However it is also capable of falling into false minima away from the global minima. For this reason we recommend using |joe| then |foce| when using |popy|, see :ref:`fit_methods_examples`.

The type of optimisation methods used by |popy| are described in references such as [DennisSchnabel1987]_ [NocedalWright2006]_.

.. _fit_methods_nd:

ND Fitting Method
==========================

The recommended fitting method for |popy| version 1.0.5 is the |nd| method. 

The |nd| method uses the same |objv| as |foce|. This new approach uses |foce| to estimate the |fx| parameters. However when the gradient based |foce| algorithm converges |nd| switches to using the non-gradient |bobyqa| optimisation method. When |bobyqa| converges it switches back to the |foce|, until both methods have converged. The |nd| method is therefore very robust as it utilises two different fitting strategies.

However the |nd| method will always be slower than |foce|, so for simple cases, or where a quick result, but not necessarily optimal solution is required, it may be preferable to use |joe| or |foce| in preference to |nd|.

In the more common case, where accuracy is required it is recommended to use |nd|.

.. _fit_methods_other:

Other Fitting Methods
======================

Note |joe|, |its| and |foce| fitting methods are deterministic and fundamentally different from the stochastic sampling methods such as |saem| and |imp|. |saem| and |imp| that also estimate |fx| for |pkpd| models, but the stochastic |objv_long| is |not| based on the Laplace approximation used in the |foce| |objv|.


.. _fit_methods_examples:

FIT_METHODS Examples
======================

.. _fit_using_nd:

Fit using ND 
-------------

The recommended fitting method for |popy| version 1.0.5 is the |nd| method. 

The |nd| method can be implemented as follows:-

.. code-block:: pyml

    FIT_METHODS: [ND:{}]

Or alternatively:-

.. code-block:: pyml

    FIT_METHODS: 
        - ND: {max_n_main_iterations: 30}
        
        
.. _fit_using_joe_and_foce:

Fit using JOE and FOCE
------------------------

Another way to fit |pkpd| models in |popy| is to use |joe| followed by |foce| in a :ref:`fit_script` is as follows:-

.. code-block:: pyml

    FIT_METHODS: [JOE:{}, FOCE:{}]

This single line, runs the |joe| fitting method followed by |foce| with the default settings. Note that the square bracket notation '[]' is required because 'FIT_METHODS' expects a list. Or equivalently:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: {}
        - FOCE: {}

Where the '-' is :term:`YAML` notation for a list item.
        
One of the simplest |joe| and |foce| parameters is 'max_n_main_iterations':-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: {max_n_main_iterations: 30}
        - FOCE: {max_n_main_iterations: 30}
    
Which sets the number of times the |fx| parameters are updated. The |joe| fitting proceeds iteratively with interleaved |fx| and |rx| updates, followed by |foce| starting from the final |fx| result returned by |joe|. Setting this limit forces |popy| to stop after a finite number of iterations.

If you miss out 'max_n_main_iterations' it defaults to 50. Another option is as follows:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: {max_n_main_iterations: 0}
            
This only updates the |rx| parameters and leaves the |fx| fixed. 


.. _fitting_convergence_criteria:

Fitting Convergence Criteria
=============================

Another common setting is the 'CONVERGER' field. There are three possible settings:-

* NONE - no convergence termination - estimation stops when 'max_n_main_iterations' is reached
* OBJ_INC - terminate convergence if the |objv| increases 
* OBJ_SIM - terminate convergence if the |objv| is similar between iterations

These settings decide when a :ref:`fit_script` will return the final |fx| estimates.

The default setting for |joe| and |foce| fitting is 'OBJ_INC', hence:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100
        - FOCE: 
            max_n_main_iterations: 100
        
Is the same as:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_INC: {}}
        - FOCE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_INC: {}}
            
And the fit estimation will stop as soon as any iteration increases the |objv|. To allow the |objv| to increase during fitting, |ie| if the likelihood gets worse between iterations, but you wish the search to continue, use:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_SIM: {}}
        - FOCE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_SIM: {}}
  
Here 'OBJ_SIM' will terminate the estimation when two consecutive iterations return similar objective values (within a relative tolerance of 1e-06).
  
Note that when fitting complex models the |objv| usually decreases at each iteration, but occasionally increases. This is due to fact that |fx| and |rx| are optimised separately. Separate optimisation is necessary for computational efficiency. However optimising components of the likelihood independently can mean the combined |objv| increases. The 'OBJ_SIM' option above is designed to allow the search to continue in these cases.


