HW4 FAQs

  • 1st derivative of log-gamma function $\ln \Gamma (x)$ is digamma function $\Psi(x)$.

  • 2nd derivative of log-gamma function is trigamma function $\Psi’(x)$.

  • In Julia 
    using SpecialFunctions
polygamma(0, 0.5) # digamma(0.5)
polygamma(1, 0.5) # trigamma(0.5)

  • Q7, method of moment estimator for $\alpha$.

    If $P=(P_1, \ldots, P_d)$ is from Dirichlet distribution with parameter $\alpha = (\alpha_1, \ldots, \alpha_d)$, then $E(P_j) = \alpha_j / (\sum_{j’} \alpha_{j’})$ and $E(P_j^2) = \alpha_j (\alpha_j + 1) / (\sum_{j’} \alpha_{j’}) / ((\sum_{j’} \alpha_{j’}) + 1)$. We can estimate $E(P_j)$ and $E(P_j^2)$ from Dirichlet-Multinomial sample and then solve for $\alpha_j$.

  • In training data, there are columns that are all zeros. It’s obvious that the corresponding $\alpha_j$ in MLE should be 0. So we may remove those zero columns before running the Newton’s algorithm to avoid some idiosyncrasies.

Today