Maximum Likelihood Methods in Unknown Latent Variables: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 6: | Line 6: | ||
== Parameters == | == Parameters == | ||
$n$: number of observations in sample | |||
$r$: number of parameters + latent variables | |||
== Table of Algorithms == | == Table of Algorithms == | ||
| Line 18: | Line 20: | ||
| [[Expectation-Maximization (EM) algorithm (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|Expectation-Maximization (EM) algorithm]] || 1977 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://www.jstor.org/stable/2984875 Time] | | [[Expectation-Maximization (EM) algorithm (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|Expectation-Maximization (EM) algorithm]] || 1977 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://www.jstor.org/stable/2984875 Time] | ||
|- | |- | ||
| [[EM with Quasi-Newton Methods (Jamshidian; Mortaza; Jennrich; Robert I.) (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|EM with Quasi-Newton Methods (Jamshidian; Mortaza; Jennrich; Robert I.)]] || 1997 || $O(n^{2} log^{3} n)$ || $O(n+r^{2})$? || Exact || Deterministic || [https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/1467-9868.00083 Time] | | [[EM with Quasi-Newton Methods (Jamshidian; Mortaza; Jennrich; Robert I.) (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|EM with Quasi-Newton Methods (Jamshidian; Mortaza; Jennrich; Robert I.)]] || 1997 || $O(n^{2} \log^{3} n)$ || $O(n+r^{2})$? || Exact || Deterministic || [https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/1467-9868.00083 Time] | ||
|- | |- | ||
| [[Parameter-expanded expectation maximization (PX-EM) (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|Parameter-expanded expectation maximization (PX-EM)]] || 1998 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [http://www.stat.ucla.edu/~ywu/research/papers/PXEM.pdf Time] | | [[Parameter-expanded expectation maximization (PX-EM) (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|Parameter-expanded expectation maximization (PX-EM)]] || 1998 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [http://www.stat.ucla.edu/~ywu/research/papers/PXEM.pdf Time] | ||
| Line 28: | Line 30: | ||
| [[α-EM Algorithm (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|α-EM Algorithm]] || 2003 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://waseda.pure.elsevier.com/en/publications/the-%CE%B1-em-algorithm-surrogate-likelihood-maximization-using-%CE%B1-loga Time] | | [[α-EM Algorithm (Maximum Likelihood Methods in Unknown Latent Variables Maximum Likelihood Methods in Unknown Latent Variables)|α-EM Algorithm]] || 2003 || $O(n^{3})$ || $O(n+r)$? || Exact || Deterministic || [https://waseda.pure.elsevier.com/en/publications/the-%CE%B1-em-algorithm-surrogate-likelihood-maximization-using-%CE%B1-loga Time] | ||
|- | |- | ||
| [[Shaban; Amirreza; Mehrdad; Farajtabar (Maximum Likelihood Methods in Unknown Latent Variables; multi-view model, discrete observations Maximum Likelihood Methods in Unknown Latent Variables)|Shaban; Amirreza; Mehrdad; Farajtabar]] || 2015 || $O(n^{2} log^{2} n)$ || $O(kd+d^{3})$?? || Exact || Deterministic || [https://faculty.cc.gatech.edu/~bboots3/files/SpectralExteriorPoint-NIPSWorkshop.pdf Time] | | [[Shaban; Amirreza; Mehrdad; Farajtabar (Maximum Likelihood Methods in Unknown Latent Variables; multi-view model, discrete observations Maximum Likelihood Methods in Unknown Latent Variables)|Shaban; Amirreza; Mehrdad; Farajtabar]] || 2015 || $O(n^{2} \log^{2} n)$ || $O(kd+d^{3})$?? || Exact || Deterministic || [https://faculty.cc.gatech.edu/~bboots3/files/SpectralExteriorPoint-NIPSWorkshop.pdf Time] | ||
|- | |- | ||
| [[alpha-HMM (Matsuyama, Yasuo) (Maximum Likelihood Methods in Unknown Latent Variables, Hidden Markov Models Maximum Likelihood Methods in Unknown Latent Variables)|alpha-HMM (Matsuyama, Yasuo)]] || 2011 || $O(n^{2} log^{2} n)$ || || Exact || Deterministic || [https://ieeexplore.ieee.org/abstract/document/7895145 Time] | | [[alpha-HMM (Matsuyama, Yasuo) (Maximum Likelihood Methods in Unknown Latent Variables, Hidden Markov Models Maximum Likelihood Methods in Unknown Latent Variables)|alpha-HMM (Matsuyama, Yasuo)]] || 2011 || $O(n^{2} \log^{2} n)$ || || Exact || Deterministic || [https://ieeexplore.ieee.org/abstract/document/7895145 Time] | ||
|- | |- | ||
|} | |} | ||
Revision as of 08:23, 10 April 2023
Description
In this problem, the goal is to compute maximum-likelihood estimates when the observations can be viewed as incomplete data.
Parameters
$n$: number of observations in sample
$r$: number of parameters + latent variables
Table of Algorithms
| Name | Year | Time | Space | Approximation Factor | Model | Reference |
|---|---|---|---|---|---|---|
| Expectation-Maximization (EM) algorithm | 1977 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| EM with Quasi-Newton Methods (Jamshidian; Mortaza; Jennrich; Robert I.) | 1997 | $O(n^{2} \log^{3} n)$ | $O(n+r^{2})$? | Exact | Deterministic | Time |
| Parameter-expanded expectation maximization (PX-EM) | 1998 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| Expectation conditional maximization (ECM) | 1993 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| Expectation conditional maximization either (ECME) (Liu; Chuanhai; Rubin; Donald B) | 1994 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| α-EM Algorithm | 2003 | $O(n^{3})$ | $O(n+r)$? | Exact | Deterministic | Time |
| Shaban; Amirreza; Mehrdad; Farajtabar | 2015 | $O(n^{2} \log^{2} n)$ | $O(kd+d^{3})$?? | Exact | Deterministic | Time |
| alpha-HMM (Matsuyama, Yasuo) | 2011 | $O(n^{2} \log^{2} n)$ | Exact | Deterministic | Time |
Time Complexity Graph
Space Complexity Graph
Time-Space Tradeoff
