Neural Processes for Image Completion πŸŽ¨πŸ–Œ πŸ”—

Presented by Christabella Irwanto

Agenda πŸ—’ πŸ”—

MotivationπŸ’‘ πŸ”—

Deep neural networks (NN) πŸ”—

Gaussian processes (GP) πŸ”—

Best of both worlds πŸ™Œ πŸ”—

Meta-learning πŸ”—

Image completion task πŸ”—

Figure 5 from (Garnelo et al., 2018)

Neural process variants πŸ”—

Conditional neural process (CNP) πŸ”—

Architecture of CNP from (DeepMind, 2018)

Training of CNP πŸ”—

\[ \mathcal { L } ( \theta ) = - \mathbb { E } _ { f \sim P } \left[ \mathbb { E } _ { N } \left[ \log Q _ { \theta } \left( \left\{ y _ { i } \right\} _ { i = 0 } ^ { n - 1 } | O _ { N } , \left\{ x _ { i } \right\} _ { i = 0 } ^ { n - 1 } \right) \right] \right] \] (Eq. 4 in (Garnelo et al., 2018))

# Latent encoder outputs mu and sigma.
dist = tf.contrib.distributions.MultivariateNormalDiag(
	loc=mu, scale_diag=sigma)
log_p = dist.log_prob(target_y)
loss = -tf.reduce_mean(log_prob)
train_step = optimizer.minimize(loss)

CNP paper results πŸ”—

Figure 3a from CNP (Garnelo et al., 2018)

Latent formulation of CNP πŸ”—

Architecture of CNP latent variable model, modified from (DeepMind, 2018)

Implications for image completion πŸ”—

Figure 6 from (Garnelo et al., 2018)

Training of latent CNP πŸ”—

\[\text {ELBO} = \mathbb{E}_{q(z | C, T)} \left[ \sum_{t=1}^T \log p(y_t^{\ast} | z, x_t^{\ast}) + \log \frac{q(z | C)}{q(z | C, T)} \right]\]

Neural process (NP) πŸ”—

NP diagram from (DeepMind, 2018)

NPs underfit πŸ”—

Figure 1 from (Kim et al., 2019)

Attentive neural process (ANP) πŸ”—

Modified from Figure 2 in (Garnelo et al., 2018)

Experiments πŸ”—

(e_1): Linear(in_features=3, out_features=400, bias=True)
(e_2): Linear(in_features=400, out_features=400, bias=True)
(e_3): Linear(in_features=400, out_features=128, bias=True)
(r_to_z_mean): Linear(in_features=128, out_features=128, bias=True)
(r_to_z_logvar): Linear(in_features=128, out_features=128, bias=True)
(d_1): Linear(in_features=130, out_features=400, bias=True)
(d_2): Linear(in_features=400, out_features=400, bias=True)
(d_3): Linear(in_features=400, out_features=400, bias=True)
(d_4): Linear(in_features=400, out_features=400, bias=True)
(d_5): Linear(in_features=400, out_features=1, bias=True)

Training πŸ”—

400 epochs took 7 hours on NVIDIA Quadro P5000 GPU (Paniikki computer)

Results πŸ”—

10 and 100 context points at epoch 400 πŸ”—

300 context points πŸ”—

784 (100%) context points πŸ”—

What about those nice celebrity faces? πŸ”—

Key learnings and findings πŸ”—

Conclusion ✨ πŸ”—

Lingering doubts πŸ€” πŸ”—

Dropout as a Bayesian approximation, Weight uncertainty in neural networks, Bayesian compression for DL; Multiplicative normalizing flows for variational bayesian neural networks

Inference in NP πŸ”—

Bibliography

Wikipedia contributors (2019). Gaussian process — Wikipedia, the free encyclopedia. Retrieved from https://en.wikipedia.org/w/index.php?title=Gaussian_process&oldid=896566703. [Online; accessed 16-May-2019]. ↩

Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S. M. A., & Teh, Y. W. (2018), Neural Processes, CoRR. ↩

Garnelo, M., Rosenbaum, D., Maddison, C. J., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., … (2018), Conditional Neural Processes, CoRR. ↩

DeepMind, (2018). Open-source implementations of neural process variants. Retrieved from https://github.com/deepmind/neural-processes. (Accessed on 05/08/2019). ↩

Kim, H., Mnih, A., Schwarz, J., Garnelo, M., Eslami, A., Rosenbaum, D., Vinyals, O., … (2019), Attentive Neural Processes, CoRR. ↩

geniki, (2018). Pytorch implementation of neural processes. Retrieved from https://github.com/geniki/neural-processes/. (Accessed on 05/17/2019). ↩

MΓ€rtens, K. (2018). Neural processes as distributions over functions | kaspar mΓ€rtens | phd student in statistical machine learning. Retrieved from https://kasparmartens.rbind.io/post/np/. (Accessed on 05/17/2019). ↩