Sampling or otherwise summarizing complex probability distributions is a central task in applied mathematics, statistics, and machine learning. Many modern algorithms for this task introduce dynamics in the space of probability measures, designing these dynamics to achieve good practical performance.
We will discuss several aspects of this broad design endeavor. First is the problem of optimal scheduling of dynamic transport, i.e., with what speed should one proceed along a prescribed path of probability measures? Though many popular methods seek “straight line” trajectories, i.e., trajectories with zero acceleration in a Lagrangian frame, we show how a specific class of “curved” trajectories can improve approximation and learning. We will then discuss extensions of this idea which seek not only schedules but paths that improve spatial regularity of the velocity.
Second, we discuss the problem of weighted quantization, i.e., summarizing a complex distribution with a small set of weighted Dirac measures. We study this problem from the perspective of minimizing maximum mean discrepancy via gradient flow in the Wasserstein-Fisher-Rao (WFR) geometry. This gradient flow yields an ODE system from which we derive a fixed-point algorithm called mean shift interacting particles (MSIP). We show that MSIP extends the classical mean shift algorithm, used for identifying modes in kernel density estimates, and that it outperforms state-of-the-art methods for quantization. We also describe how MSIP can be used not only to quantize an empirical measure, but to generate good particle approximations given only an unnormalized density.