Control of a queuing system with hidden Markov state

Abstract

A single-server finite-buffer queuing system is considered on a fixed time interval. The server accepts a non-stationary Poisson stream of incoming packets for further transmission through a communication channel governed by a hidden Markov chain. Round-trip times for sent packets are described by the Markov counting process which is observed directly. The service rate is proportional to the transmission rate with a channel-dependent factor. The transmission rate is to be optimized within the class of feedback control policies given two performance characteristics: the average number of lost packets and the mean level of energy consumption. The approach proposed for control optimization is based on the optimal filter equations, the complete-information control algorithm, and Monte Carlo simulation.