TY - JOUR
T1 - Learning to Sample with Adversarially Learned Likelihood-Ratio
AU - Li, Chunyuan
AU - Li, Jianqiao
AU - Wang, Guoyin
AU - Carin, Lawrence
PY - 2018
Y1 - 2018
N2 - We link the reverse KL divergence with adversarial learning. This insight enables learning to synthesize realistic samples in two settings: (i) Given a set of samples from the true distribution, an adversarially learned likelihood-ratio and a new entropy bound are used to learn a GAN model, that improves synthesized sample quality relative to previous GAN variants. (ii) Given an unnormalized distribution, a reference-based framework is proposed to learn to draw samples, naturally yielding an adversarial scheme to amortize MCMC/SVGD samples. Experimental results show the improved performance of the derived algorithms. 1 BACKGROUND ON THE REVERSE KL DIVERGENCE Target Distribution Assume we are given a set of samples D = {x i } i=1,N , with each sample assumed drawn iid from an unknown distribution q(x). For x ∈ X , let S q ⊂ X represent the support of q, implying that S q is the smallest subset of X for which Sq q(x)dx = 1 (or Sq q(x)dx = 1 − , for → 0 +). Let S o q represent the complement set of S q , i.e., S q ∪ S o q = X and S q ∩ S o q = ∅.
AB - We link the reverse KL divergence with adversarial learning. This insight enables learning to synthesize realistic samples in two settings: (i) Given a set of samples from the true distribution, an adversarially learned likelihood-ratio and a new entropy bound are used to learn a GAN model, that improves synthesized sample quality relative to previous GAN variants. (ii) Given an unnormalized distribution, a reference-based framework is proposed to learn to draw samples, naturally yielding an adversarial scheme to amortize MCMC/SVGD samples. Experimental results show the improved performance of the derived algorithms. 1 BACKGROUND ON THE REVERSE KL DIVERGENCE Target Distribution Assume we are given a set of samples D = {x i } i=1,N , with each sample assumed drawn iid from an unknown distribution q(x). For x ∈ X , let S q ⊂ X represent the support of q, implying that S q is the smallest subset of X for which Sq q(x)dx = 1 (or Sq q(x)dx = 1 − , for → 0 +). Let S o q represent the complement set of S q , i.e., S q ∪ S o q = X and S q ∩ S o q = ∅.
UR - https://openreview.net/forum?id=S1eZGHkDM
UR - https://www.mendeley.com/catalogue/385dcfb3-1bf3-3fc4-9211-3561a398d27e/
M3 - Article
SP - 1
EP - 6
JO - Iclr 2018
JF - Iclr 2018
IS - 2
ER -