We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

Two-state imprecise Markov chains for statistical modelling of two-state non-Markovian processes.

Troffaes, Matthias C. M. and Krak, Thomas and Bains, Henna (2019) 'Two-state imprecise Markov chains for statistical modelling of two-state non-Markovian processes.', in Proceedings of the Eleventh International Symposium on Imprecise Probabilities : Theories and Applications. , pp. 394-403. Proceedings of machine learning research. (103).


This paper proposes a method for fitting a two-state imprecise Markov chain to time series data from a twostate non-Markovian process. Such non-Markovian processes are common in practical applications. We focus on how to fit modelling parameters based on data from a process where time to transition is not exponentially distributed, thereby violating the Markov assumption. We do so by first fitting a many-state (i.e. having more than two states) Markov chain to the data, through its associated phase-type distribution. Then, we lump the process to a two-state imprecise Markov chain. In practical applications, a two-state imprecise Markov chain might be more convenient than a many-state Markov chain, as we have closed analytic expressions for typical quantities of interest (including the lower and upper expectation of any function of the state at any point in time). A numerical example demonstrates how the entire inference process (fitting and prediction) can be done using Markov chain Monte Carlo, for a given set of prior distributions on the parameters. In particular, we numerically identify the set of posterior densities and posterior lower and upper expectations on all model parameters and predictive quantities. We compare our inferences under a range of sample sizes and model assumptions. Keywords: imprecise Markov chain, estimation, reliability, Markov assumption, MCMC

Item Type:Book chapter
Full text:(AM) Accepted Manuscript
Available under License - Creative Commons Attribution.
Download PDF
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution.
Download PDF
Publisher Web site:
Publisher statement:This paper has been published under a Creative Commons Attribution 4.0 International License specified at (human readable summary at
Date accepted:17 May 2019
Date deposited:18 June 2019
Date of first online publication:2019
Date first made open access:No date available

Save or Share this output

Look up in GoogleScholar