Abstract

An applied compact introductory survey of Markov stochastic processes and control in continuous time is presented. The presentation is in tutorial stages, beginning with deterministic dynamical systems for contrast and continuing on to perturbing the deterministic model with diffusions using Wiener processes. Then jump perturbations are added using simple Poisson processes constructing the theory of simple jump-diffusions. Next, marked-jump-diffusions are treated using compound Poisson processes to include random marked jump-amplitudes in parallel with the equivalent Poisson random measure formulation. Otherwise, the approach is quite applied, using basic principles with no abstractions beyond Poisson random measure. This treatment is suitable for those in classical applied mathematics, physical sciences, quantitative finance and engineering, but have trouble getting started with the abstract measure-theoretic literature. The approach here builds upon the treatment of continuous functions in the regular calculus and associated ordinary differential equations by adding non-smooth and jump discontinuities to the model. Finally, the stochastic optimal control of marked-jump-diffusions is developed, emphasizing the underlying assumptions. The survey concludes with applications in biology and finance, some of which are canonical, dimension reducible problems and others are genuine nonlinear problems.