ResearchPad - markov-processes https://www.researchpad.co Default RSS Feed en-us © 2020 Newgen KnowledgeWorks <![CDATA[ToyArchitecture: Unsupervised learning of interpretable models of the environment]]> https://www.researchpad.co/article/elastic_article_15730 Research in Artificial Intelligence (AI) has focused mostly on two extremes: either on small improvements in narrow AI domains, or on universal theoretical frameworks which are often uncomputable, or lack practical implementations. In this paper we attempt to follow a big picture view while also providing a particular theory and its implementation to present a novel, purposely simple, and interpretable hierarchical architecture. This architecture incorporates the unsupervised learning of a model of the environment, learning the influence of one’s own actions, model-based reinforcement learning, hierarchical planning, and symbolic/sub-symbolic integration in general. The learned model is stored in the form of hierarchical representations which are increasingly more abstract, but can retain details when needed. We demonstrate the universality of the architecture by testing it on a series of diverse environments ranging from audio/visual compression to discrete and continuous action spaces, to learning disentangled representations.

]]>
<![CDATA[Exploring chromatin hierarchical organization via Markov State Modelling]]> https://www.researchpad.co/article/5c79b497d5eed0c4841ea6b0

We propose a new computational method for exploring chromatin structural organization based on Markov State Modelling of Hi-C data represented as an interaction network between genomic loci. A Markov process describes the random walk of a traveling probe in the corresponding energy landscape, mimicking the motion of a biomolecule involved in chromatin function. By studying the metastability of the associated Markov State Model upon annealing, the hierarchical structure of individual chromosomes is observed, and corresponding set of structural partitions is identified at each level of hierarchy. Then, the notion of effective interaction between partitions is derived, delineating the overall topology and architecture of chromosomes. Mapping epigenetic data on the graphs of intra-chromosomal effective interactions helps in understanding how chromosome organization facilitates its function. A sketch of whole-genome interactions obtained from the analysis of 539 partitions from all 23 chromosomes, complemented by distributions of gene expression regulators and epigenetic factors, sheds light on the structure-function relationships in chromatin, delineating chromosomal territories, as well as structural partitions analogous to topologically associating domains and active / passive epigenomic compartments. In addition to the overall genome architecture shown by effective interactions, the affinity between partitions of different chromosomes was analyzed as an indicator of the degree of association between partitions in functionally relevant genomic interactions. The overall static picture of whole-genome interactions obtained with the method presented in this work provides a foundation for chromatin structural reconstruction, for the modelling of chromatin dynamics, and for exploring the regulation of genome function. The algorithms used in this study are implemented in a freely available Python package ChromaWalker (https://bitbucket.org/ZhenWahTan/chromawalker).

]]>
<![CDATA[An evolutionary game perspective on quantised consensus in opinion dynamics]]> https://www.researchpad.co/article/5c390ba9d5eed0c48491dbf3

Quantised consensus has been used in the context of opinion dynamics. In this context agents interact with their neighbours and they change their opinion according to their interests and the opinions of their neighbours. We consider various quantised consensus models, where agents have different levels of susceptibility to the inputs received from their neighbours. The provided models share similarities with collective decision making models inspired by honeybees and evolutionary games. As first contribution, we develop an evolutionary game-theoretic model that accommodates the different consensus dynamics in a unified framework. As second contribution, we study equilibrium points and extend such study to the symmetric case where the transition probabilities of the evolutionary game dynamics are symmetric. Symmetry is associated with the case of equally favourable options. As third contribution, we study stability of the equilibrium points for the different cases. We corroborate the theoretical results with some simulations to study the outcomes of the various models.

]]>
<![CDATA[A Markovian Entropy Measure for the Analysis of Calcium Activity Time Series]]> https://www.researchpad.co/article/5989d9f2ab0ee8fa60b6ef3c

Methods to analyze the dynamics of calcium activity often rely on visually distinguishable features in time series data such as spikes, waves, or oscillations. However, systems such as the developing nervous system display a complex, irregular type of calcium activity which makes the use of such methods less appropriate. Instead, for such systems there exists a class of methods (including information theoretic, power spectral, and fractal analysis approaches) which use more fundamental properties of the time series to analyze the observed calcium dynamics. We present a new analysis method in this class, the Markovian Entropy measure, which is an easily implementable calcium time series analysis method which represents the observed calcium activity as a realization of a Markov Process and describes its dynamics in terms of the level of predictability underlying the transitions between the states of the process. We applied our and other commonly used calcium analysis methods on a dataset from Xenopus laevis neural progenitors which displays irregular calcium activity and a dataset from murine synaptic neurons which displays activity time series that are well-described by visually-distinguishable features. We find that the Markovian Entropy measure is able to distinguish between biologically distinct populations in both datasets, and that it can separate biologically distinct populations to a greater extent than other methods in the dataset exhibiting irregular calcium activity. These results support the benefit of using the Markovian Entropy measure to analyze calcium dynamics, particularly for studies using time series data which do not exhibit easily distinguishable features.

]]>
<![CDATA[Degradation Parameters from Pulse-Chase Experiments]]> https://www.researchpad.co/article/5989dab1ab0ee8fa60bab897

Pulse-chase experiments are often used to study the degradation of macromolecules such as proteins or mRNA. Considerations for the choice of pulse length include the toxicity of the pulse to the cell and maximization of labeling. In the general case of non-exponential decay, varying the length of the pulse results in decay patterns that look different. Analysis of these patterns without consideration to pulse length would yield incorrect degradation parameters. Here we propose a method that constructively includes pulse length in the analysis of decay patterns and extracts the parameters of the underlying degradation process. We also show how to extract decay parameters reliably from measurements taken during the pulse phase.

]]>
<![CDATA[A rough set approach for determining weights of decision makers in group decision making]]> https://www.researchpad.co/article/5989db50ab0ee8fa60bdbd78

This study aims to present a novel approach for determining the weights of decision makers (DMs) based on rough group decision in multiple attribute group decision-making (MAGDM) problems. First, we construct a rough group decision matrix from all DMs’ decision matrixes on the basis of rough set theory. After that, we derive a positive ideal solution (PIS) founded on the average matrix of rough group decision, and negative ideal solutions (NISs) founded on the lower and upper limit matrixes of rough group decision. Then, we obtain the weight of each group member and priority order of alternatives by using relative closeness method, which depends on the distances from each individual group member’ decision to the PIS and NISs. Through comparisons with existing methods and an on-line business manager selection example, the proposed method show that it can provide more insights into the subjectivity and vagueness of DMs’ evaluations and selections.

]]>
<![CDATA[Detection of statistical asymmetries in non-stationary sign time series: Analysis of foreign exchange data]]> https://www.researchpad.co/article/5989db5cab0ee8fa60bdfeef

We extend the concept of statistical symmetry as the invariance of a probability distribution under transformation to analyze binary sign time series data of price difference from the foreign exchange market. We model segments of the sign time series as Markov sequences and apply a local hypothesis test to evaluate the symmetries of independence and time reversion in different periods of the market. For the test, we derive the probability of a binary Markov process to generate a given set of number of symbol pairs. Using such analysis, we could not only segment the time series according the different behaviors but also characterize the segments in terms of statistical symmetries. As a particular result, we find that the foreign exchange market is essentially time reversible but this symmetry is broken when there is a strong external influence.

]]>
<![CDATA[Location Prediction Based on Transition Probability Matrices Constructing from Sequential Rules for Spatial-Temporal K-Anonymity Dataset]]> https://www.researchpad.co/article/5989da5cab0ee8fa60b90265

Spatial-temporal k-anonymity has become a mainstream approach among techniques for protection of users’ privacy in location-based services (LBS) applications, and has been applied to several variants such as LBS snapshot queries and continuous queries. Analyzing large-scale spatial-temporal anonymity sets may benefit several LBS applications. In this paper, we propose two location prediction methods based on transition probability matrices constructing from sequential rules for spatial-temporal k-anonymity dataset. First, we define single-step sequential rules mined from sequential spatial-temporal k-anonymity datasets generated from continuous LBS queries for multiple users. We then construct transition probability matrices from mined single-step sequential rules, and normalize the transition probabilities in the transition matrices. Next, we regard a mobility model for an LBS requester as a stationary stochastic process and compute the n-step transition probability matrices by raising the normalized transition probability matrices to the power n. Furthermore, we propose two location prediction methods: rough prediction and accurate prediction. The former achieves the probabilities of arriving at target locations along simple paths those include only current locations, target locations and transition steps. By iteratively combining the probabilities for simple paths with n steps and the probabilities for detailed paths with n-1 steps, the latter method calculates transition probabilities for detailed paths with n steps from current locations to target locations. Finally, we conduct extensive experiments, and correctness and flexibility of our proposed algorithm have been verified.

]]>
<![CDATA[Improved Measures of Integrated Information]]> https://www.researchpad.co/article/5989d9f1ab0ee8fa60b6e930

Although there is growing interest in measuring integrated information in computational and cognitive systems, current methods for doing so in practice are computationally unfeasible. Existing and novel integration measures are investigated and classified by various desirable properties. A simple taxonomy of Φ-measures is presented where they are each characterized by their choice of factorization method (5 options), choice of probability distributions to compare (3 × 4 options) and choice of measure for comparing probability distributions (7 options). When requiring the Φ-measures to satisfy a minimum of attractive properties, these hundreds of options reduce to a mere handful, some of which turn out to be identical. Useful exact and approximate formulas are derived that can be applied to real-world data from laboratory experiments without posing unreasonable computational demands.

]]>
<![CDATA[Queues with Dropping Functions and General Arrival Processes]]> https://www.researchpad.co/article/5989da7fab0ee8fa60b99e01

In a queueing system with the dropping function the arriving customer can be denied service (dropped) with the probability that is a function of the queue length at the time of arrival of this customer. The potential applicability of such mechanism is very wide due to the fact that by choosing the shape of this function one can easily manipulate several performance characteristics of the queueing system. In this paper we carry out analysis of the queueing system with the dropping function and a very general model of arrival process—the model which includes batch arrivals and the interarrival time autocorrelation, and allows for fitting the actual shape of the interarrival time distribution and its moments. For such a system we obtain formulas for the distribution of the queue length and the overall customer loss ratio. The analytical results are accompanied with numerical examples computed for several dropping functions.

]]>
<![CDATA[Network switching strategy for energy conservation in heterogeneous networks]]> https://www.researchpad.co/article/5989db50ab0ee8fa60bdbcf0

In heterogeneous networks (HetNets), the large-scale deployment of small base stations (BSs) together with traditional macro BSs is an economical and efficient solution that is employed to address the exponential growth in mobile data traffic. In dense HetNets, network switching, i.e., handovers, plays a critical role in connecting a mobile terminal (MT) to the best of all accessible networks. In the existing literature, a handover decision is made using various handover metrics such as the signal-to-noise ratio, data rate, and movement speed. However, there are few studies on handovers that focus on energy efficiency in HetNets. In this paper, we propose a handover strategy that helps to minimize energy consumption at BSs in HetNets without compromising the quality of service (QoS) of each MT. The proposed handover strategy aims to capture the effect of the stochastic behavior of handover parameters and the expected energy consumption due to handover execution when making a handover decision. To identify the validity of the proposed handover strategy, we formulate a handover problem as a constrained Markov decision process (CMDP), by which the effects of the stochastic behaviors of handover parameters and consequential handover energy consumption can be accurately reflected when making a handover decision. In the CMDP, the aim is to minimize the energy consumption to service an MT over the lifetime of its connection, and the constraint is to guarantee the QoS requirements of the MT given in terms of the transmission delay and call-dropping probability. We find an optimal policy for the CMDP using a combination of the Lagrangian method and value iteration. Simulation results verify the validity of the proposed handover strategy.

]]>
<![CDATA[Decoding Single Molecule Time Traces with Dynamic Disorder]]> https://www.researchpad.co/article/5989dad1ab0ee8fa60bb6205

Single molecule time trajectories of biomolecules provide glimpses into complex folding landscapes that are difficult to visualize using conventional ensemble measurements. Recent experiments and theoretical analyses have highlighted dynamic disorder in certain classes of biomolecules, whose dynamic pattern of conformational transitions is affected by slower transition dynamics of internal state hidden in a low dimensional projection. A systematic means to analyze such data is, however, currently not well developed. Here we report a new algorithm—Variational Bayes-double chain Markov model (VB-DCMM)—to analyze single molecule time trajectories that display dynamic disorder. The proposed analysis employing VB-DCMM allows us to detect the presence of dynamic disorder, if any, in each trajectory, identify the number of internal states, and estimate transition rates between the internal states as well as the rates of conformational transition within each internal state. Applying VB-DCMM algorithm to single molecule FRET data of H-DNA in 100 mM-Na+ solution, followed by data clustering, we show that at least 6 kinetic paths linking 4 distinct internal states are required to correctly interpret the duplex-triplex transitions of H-DNA.

]]>
<![CDATA[In Vivo Volume and Hemoglobin Dynamics of Human Red Blood Cells]]> https://www.researchpad.co/article/5989dab8ab0ee8fa60badb50

Human red blood cells (RBCs) lose ∼30% of their volume and ∼20% of their hemoglobin (Hb) content during their ∼100-day lifespan in the bloodstream. These observations are well-documented, but the mechanisms for these volume and hemoglobin loss events are not clear. RBCs shed hemoglobin-containing vesicles during their life in the circulation, and this process is thought to dominate the changes in the RBC physical characteristics occurring during maturation. We combine theory with single-cell measurements to investigate the impact of vesiculation on the reduction in volume, Hb mass, and membrane. We show that vesicle shedding alone is sufficient to explain membrane losses but not volume or Hb losses. We use dry mass measurements of human RBCs to validate the models and to propose that additional unknown mechanisms control volume and Hb reduction and are responsible for ∼90% of the observed reduction. RBC population characteristics are used in the clinic to monitor and diagnose a wide range of conditions including malnutrition, inflammation, and cancer. Quantitative characterization of cellular maturation processes may help in the early detection of clinical conditions where maturation patterns are altered.

]]>
<![CDATA[Predicting explorative motor learning using decision-making and motor noise]]> https://www.researchpad.co/article/5989db5aab0ee8fa60bdf2bc

A fundamental problem faced by humans is learning to select motor actions based on noisy sensory information and incomplete knowledge of the world. Recently, a number of authors have asked whether this type of motor learning problem might be very similar to a range of higher-level decision-making problems. If so, participant behaviour on a high-level decision-making task could be predictive of their performance during a motor learning task. To investigate this question, we studied performance during an explorative motor learning task and a decision-making task which had a similar underlying structure with the exception that it was not subject to motor (execution) noise. We also collected an independent measurement of each participant’s level of motor noise. Our analysis showed that explorative motor learning and decision-making could be modelled as the (approximately) optimal solution to a Partially Observable Markov Decision Process bounded by noisy neural information processing. The model was able to predict participant performance in motor learning by using parameters estimated from the decision-making task and the separate motor noise measurement. This suggests that explorative motor learning can be formalised as a sequential decision-making process that is adjusted for motor noise, and raises interesting questions regarding the neural origin of explorative motor learning.

]]>
<![CDATA[Quantitative Analysis of Axonal Branch Dynamics in the Developing Nervous System]]> https://www.researchpad.co/article/5989da8cab0ee8fa60b9e4f7

Branching is an important mechanism by which axons navigate to their targets during neural development. For instance, in the developing zebrafish retinotectal system, selective branching plays a critical role during both initial pathfinding and subsequent arborisation once the target zone has been reached. Here we show how quantitative methods can help extract new information from time-lapse imaging about the nature of the underlying branch dynamics. First, we introduce Dynamic Time Warping to this domain as a method for automatically matching branches between frames, replacing the effort required for manual matching. Second, we model branch dynamics as a birth-death process, i.e. a special case of a continuous-time Markov process. This reveals that the birth rate for branches from zebrafish retinotectal axons, as they navigate across the tectum, increased over time. We observed no significant change in the death rate for branches over this time period. However, blocking neuronal activity with TTX slightly increased the death rate, without a detectable change in the birth rate. Third, we show how the extraction of these rates allows computational simulations of branch dynamics whose statistics closely match the data. Together these results reveal new aspects of the biology of retinotectal pathfinding, and introduce computational techniques which are applicable to the study of axon branching more generally.

]]>
<![CDATA[A Spectrum Access Based on Quality of Service (QoS) in Cognitive Radio Networks]]> https://www.researchpad.co/article/5989d9e3ab0ee8fa60b6a316

The quality of service (QoS) is important issue for cognitive radio networks. In the cognitive radio system, the licensed users, also called primary users (PUs), are authorized to utilize the wireless spectrum, while unlicensed users, also called secondary users (SUs), are not authorized to use the wireless spectrum. SUs access the wireless spectrum opportunistically when the spectrum is idle. While SUs use an idle channel, the instance that PUs come back makes SUs terminate their communications and leave the current channel. Therefore, quality of service (QoS) is difficult to be ensured for SUs. In this paper, we first propose an analysis model to obtain QoS for cognitive radio networks such as blocking probability, completed traffic and termination probability of SUs. When the primary users use the channels frequently, QoS of SUs is difficult to be ensured, especially the termination probability. Then, we propose a channel reservation scheme to improve QoS of SUs. The scheme makes the terminated SUs move to the reserved channels and keep on communications. Simulation results show that our scheme can improve QoS of SUs especially the termination probability with a little cost of blocking probability in dynamic environment.

]]>
<![CDATA[Time-Varying Transition Probability Matrix Estimation and Its Application to Brand Share Analysis]]> https://www.researchpad.co/article/5989da8eab0ee8fa60b9f0af

In a product market or stock market, different products or stocks compete for the same consumers or purchasers. We propose a method to estimate the time-varying transition matrix of the product share using a multivariate time series of the product share. The method is based on the assumption that each of the observed time series of shares is a stationary distribution of the underlying Markov processes characterized by transition probability matrices. We estimate transition probability matrices for every observation under natural assumptions. We demonstrate, on a real-world dataset of the share of automobiles, that the proposed method can find intrinsic transition of shares. The resulting transition matrices reveal interesting phenomena, for example, the change in flows between TOYOTA group and GM group for the fiscal year where TOYOTA group’s sales beat GM’s sales, which is a reasonable scenario.

]]>
<![CDATA[Prediction-based association control scheme in dense femtocell networks]]> https://www.researchpad.co/article/5989db50ab0ee8fa60bdbfba

The deployment of large number of femtocell base stations allows us to extend the coverage and efficiently utilize resources in a low cost manner. However, the small cell size of femtocell networks can result in frequent handovers to the mobile user, and consequently throughput degradation. Thus, in this paper, we propose predictive association control schemes to improve the system’s effective throughput. Our design focuses on reducing handover frequency without impacting on throughput. The proposed schemes determine handover decisions that contribute most to the network throughput and are proper for distributed implementations. The simulation results show significant gains compared with existing methods in terms of handover frequency and network throughput perspective.

]]>
<![CDATA[Complexity, Predictability and Time Homogeneity of Syntax in the Songs of Cassin’s Vireo (Vireo cassinii)]]> https://www.researchpad.co/article/5989da9eab0ee8fa60ba4e65

Many species of animals deliver vocalizations in sequences presumed to be governed by internal rules, though the nature and complexity of these syntactical rules have been investigated in relatively few species. Here I present an investigation into the song syntax of fourteen male Cassin’s Vireos (Vireo cassinii), a species whose song sequences are highly temporally structured. I compare their song sequences to three candidate models of varying levels of complexity–zero-order, first-order and second-order Markov models–and employ novel methods to interpolate between these three models. A variety of analyses, including sequence simulations, Fisher’s exact tests, and model likelihood analyses, showed that the songs of this species are too complex to be described by a zero-order or first-order Markov model. The model that best fit the data was intermediate in complexity between a first- and second-order model, though I also present evidence that some transition probabilities are conditioned on up to three preceding phrases. In addition, sequences were shown to be predictable with more than 54% accuracy overall, and predictability was positively correlated with the rate of song delivery. An assessment of the time homogeneity of syntax showed that transition probabilities between phrase types are largely stable over time, but that there was some evidence for modest changes in syntax within and between breeding seasons, a finding that I interpret to represent changes in breeding stage and social context rather than irreversible, secular shifts in syntax over time. These findings constitute a valuable addition to our understanding of bird song syntax in free-living birds, and will contribute to future attempts to understand the evolutionary importance of bird song syntax in avian communication.

]]>
<![CDATA[Performance Evaluation Model for Application Layer Firewalls]]> https://www.researchpad.co/article/5989dabeab0ee8fa60bafc2b

Application layer firewalls protect the trusted area network against information security risks. However, firewall performance may affect user experience. Therefore, performance analysis plays a significant role in the evaluation of application layer firewalls. This paper presents an analytic model of the application layer firewall, based on a system analysis to evaluate the capability of the firewall. In order to enable users to improve the performance of the application layer firewall with limited resources, resource allocation was evaluated to obtain the optimal resource allocation scheme in terms of throughput, delay, and packet loss rate. The proposed model employs the Erlangian queuing model to analyze the performance parameters of the system with regard to the three layers (network, transport, and application layers). Then, the analysis results of all the layers are combined to obtain the overall system performance indicators. A discrete event simulation method was used to evaluate the proposed model. Finally, limited service desk resources were allocated to obtain the values of the performance indicators under different resource allocation scenarios in order to determine the optimal allocation scheme. Under limited resource allocation, this scheme enables users to maximize the performance of the application layer firewall.

]]>