Maximum Likelihood Parameters: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 6: | Line 6: | ||
== Parameters == | == Parameters == | ||
$n$: number of observations in sample | |||
$r$: number of parameters + latent variables | |||
== Table of Algorithms == | == Table of Algorithms == | ||
| Line 22: | Line 24: | ||
| [[Parameter-expanded expectation maximization (PX-EM) algorithm ( Maximum Likelihood Parameters)|Parameter-expanded expectation maximization (PX-EM) algorithm]] || 1998 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://www.jstor.org/stable/2337481 Time] | | [[Parameter-expanded expectation maximization (PX-EM) algorithm ( Maximum Likelihood Parameters)|Parameter-expanded expectation maximization (PX-EM) algorithm]] || 1998 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://www.jstor.org/stable/2337481 Time] | ||
|- | |- | ||
| [[Expectation conditional maximization (ECM) ( Maximum Likelihood Parameters)|Expectation conditional maximization (ECM)]] || 2017 || $O(n^{2} | | [[Expectation conditional maximization (ECM) ( Maximum Likelihood Parameters)|Expectation conditional maximization (ECM)]] || 2017 || $O(n^{2} \log n)$ || $O(n+r)$? || Exact || Deterministic || [https://arxiv.org/abs/1709.06970 Time] | ||
|- | |- | ||
| [[Generalized expectation maximization (GEM) algorithm ( Maximum Likelihood Parameters)|Generalized expectation maximization (GEM) algorithm]] || 1994 || $O(n^{4} log^{0. | | [[Generalized expectation maximization (GEM) algorithm ( Maximum Likelihood Parameters)|Generalized expectation maximization (GEM) algorithm]] || 1994 || $O(n^{4} \log^{0.1}(.{5}n)$) || $O(n+r)$? || Exact || Deterministic || [https://web.eecs.umich.edu/~fessler/papers/files/jour/94/web/fessler-94-sag.pdf Time] | ||
|- | |- | ||
| [[α-EM algorithm ( Maximum Likelihood Parameters)|α-EM algorithm]] || 2003 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://dl.acm.org/doi/10.1109/TIT.2002.808105 Time] | | [[α-EM algorithm ( Maximum Likelihood Parameters)|α-EM algorithm]] || 2003 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://dl.acm.org/doi/10.1109/TIT.2002.808105 Time] | ||
| Line 33: | Line 35: | ||
[[File:Maximum Likelihood Parameters - Time.png|1000px]] | [[File:Maximum Likelihood Parameters - Time.png|1000px]] | ||
Latest revision as of 09:08, 28 April 2023
Description
In these algorithms, the goal is to estimate hyperparameters using maximum likelihood.
Parameters
$n$: number of observations in sample
$r$: number of parameters + latent variables
Table of Algorithms
| Name | Year | Time | Space | Approximation Factor | Model | Reference |
|---|---|---|---|---|---|---|
| Expectation–maximization (EM) algorithm | 1977 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| Newton–Raphson algorithm | 1685 | $O(n^{3})$ | $O(n+r^{2})$? | Exact | Deterministic | |
| Parameter-expanded expectation maximization (PX-EM) algorithm | 1998 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| Expectation conditional maximization (ECM) | 2017 | $O(n^{2} \log n)$ | $O(n+r)$? | Exact | Deterministic | Time |
| Generalized expectation maximization (GEM) algorithm | 1994 | $O(n^{4} \log^{0.1}(.{5}n)$) | $O(n+r)$? | Exact | Deterministic | Time |
| α-EM algorithm | 2003 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
Time Complexity Graph
