Discrete time markov chains: Random processes and Markov property. Markov chain theory. Connections to graph theory and linear algebra. Basic structure of a chain. Times of first passage ant first return. Recurrent and transient states. Infinitely many visits of a state. Ergodic behaviour of a chain. Limit theorems. Specific results for the case of finite number of states.

Continuous time markov chains: Poisson flow and Poisson process. Birth processes: solving Kolmogorov equations. Continuous time Markov property. Forward and backward Kolmogorov equations and their solutions. Stacionary distribution. Reverse approach. Stability and explosions. Diferential equations and generator of a one-parameter semigroup.

Applications of markov chains: Waiting queue systems (birth&death system, M/M/1, introduction into the general theory,some important cases of waiting queue systems). Monte Carlo markov chains (Bayesian statistics and Monte Carlo simulations, Gibbs sampler and Metropolis-Hastings algorithm, convergence of MCMC algorithms, applications in Financial Mathematics).

# Probability 2

G. Grimmett, D. Stirzaker: Probability and Random Processes, 3rd edition, Oxford Univ. Press, Oxford, 2001.

D. Williams: Probability with Martingales, Cambridge Univ. Press, Cambridge, 1995.

L. C. G. Rogers, D. Williams: Diffusions, Markov Processes, and Martingales I : Foundations, 2nd edition, Cambridge Univ. Press, Cambridge, 2000.

J. R. Norris: Markov Chains, Cambridge Univ. Press, Cambridge, 1999.

S. I. Resnick: Adventures in Stochastic Processes, Birkhäuser, Boston, 1992.

The course provides a certain number of probability themes that do not need deep theoretical knowledge. However they are important in view of applications. The emphasys is on ergodic theory, both in discrete and continuous time. Appliacations include waiting queue systems and MCMC methods.

Knowledge and understanding:

The knoledge of some of the most important applications of probability is acquired.

Lectures, exercises, homeworks, consultations

Type (examination, oral, coursework, project):

written exam or 2 midterm type exams

oral exam that can be partially replaced by theoretical tests

grading: 1-5 (fail), 6-10 (pass) (according to the Statute of UL)

Mihael Perman:

PERMAN, Mihael, PITMAN, Jim, YOR, Marc. Size-biased sampling of Poisson processes and excursions. Probability theory and related fields, ISSN 0178-8051, 1992, 92, no. 1, str. 21-39. [COBISS-SI-ID 12236377]

PERMAN, Mihael, WELLNER, Jon A. On the distribution of Brownian areas. Annals of applied probability, ISSN 1050-5164, 1996, let. 6, št. 4, str. 1091-1111. [COBISS-SI-ID 7101017]

PERMAN, Mihael. An excursion approach to Ray-Knight theorems for perturbed Brownian motion. Stochastic Processes and their Applications, ISSN 0304-4149. [Print ed.], 1996, let. 63, str. 67-74. [COBISS-SI-ID 7621465]

Janez BERNIK:

1) BERNIK, Janez, RADJAVI, Heydar. Invariant and almost-invariant subspaces for pairs of idempotents. Integral equations and operator theory,ISSN 0378-620X, 2016, vol. 84, iss. 2, str. 283-288. http://dx.doi.org/10.1007/s00020-015-2260-3, doi: 10.1007/s00020-015-2260-3.

2.) BERNIK, Janez, POPOV, Alexey I. Obstructions for semigroups of partial isometries to be self-adjoint. Mathematical proceedings of the Cambridge Philosophical Society, ISSN 0305-0041, 2016, vol. 161, iss. 1, str. 107-116. http://dx.doi.org/10.1017/S0305004116000141.

3. ) BERNIK, Janez, MARCOUX, Laurent W., POPOV, Alexey I., RADJAVI, Heydar. On selfadjoint extensions of semigroups of partial isometries. Transactions of the American Mathematical Society, ISSN 0002-9947, 2016, vol. 368, no. 11, str. 7681-7702. http://dx.doi.org/10.1090/tran/6619.