StocHy - automated verification and synthesis of stochastic processes: poster abstract

Stochastic hybrid systems (SHS) are a rich mathematical modelling framework capable of describing complex systems, where uncertainty and hybrid (that is, both continuous and discrete) components are relevant. We introduce a new software tool - StocHy-aimed at simplifying both the modelling of SHS and their analysis. StocHy can (i) perform verification tasks, e.g., compute the probability of staying within a certain region of the state space from a given set of initial conditions; (ii) automatically synthesise strategies maximising this probability, and (iii) simulate the SHS evolution over time. We highlight the performance of StocHy, via a set of experiment that are run on a standard laptop, with an Intel Core i7-8550U CPU at 1.80GHz × 8 and with 8 GB of RAM. StocHy is available at gitlab.com/natchi92/StocHy.


IMPLEMENTATION
StocHy is implemented in c++ and employs manipulations based on vector calculus, the symbolic construction of probabilistic kernels, and multi-threading. shs are described by parsing well-known and -used state-space models from which StocHy generates a standard shs model automatically and formats it to be analysed. StocHy is modular, and has separate simulation, verification and synthesis engines, which are implemented as independent libraries.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.
• Q = {q 1 , q 2 , . . . , q m }, m ∈ N, represents a finite set of modes (locations); • n ∈ N is the dimension of the continuous space R n of each mode; the hybrid state space is then D= ∪ q ∈ Q {q} × R n ; • U is a continuous set of actions, e.g. R v ; • T q : Q × D × U → [0, 1] is a discrete stochastic kernel on Q given D × U, which assigns to each s = (q, x) ∈ D and u ∈ U, a probability distribution over Q : T q (·|s, u); chastic kernel on R n given D × U, which assigns to each s ∈ D and u ∈ U a probability measure on the Borel space (R n , B(R n )) : T x (·|s, u). In this model the discrete component takes values in a finite set Q of modes (a.k.a. locations), each endowed with a continuous domain (the Euclidean space R n ). The semantics of transitions at any point over a discrete time domain, are as follows: given a point s ∈ D, the discrete state is chosen from T q , and depending on the selected mode q ∈ Q the continuous state is updated according to the probabilistic law T x . Non-determinism in the form of actions can affect both discrete and continuous transitions.

FORMAL VERIFICATION
StocHy performs formal verification of shs via either of two abstraction techniques: (i) for discrete-time, continuous-space models with additive disturbances, and possibly with multiple discrete modes,we employ formal abstractions as general Markov chains (mc) or Markov decision processes (mdp); StocHy improves techniques the state-of-the-art faust 2 tool [4] by simplifying the input model description and by reducing the computational time needed to generate the abstractions; and (ii) for models with a finite number of actions, we employ interval Markov decision processes (imdp) and the model checking framework in [3]; StocHy incorporates a novel abstraction algorithm allowing for efficient computation [2].

Comparison of verification methods
We consider a simple shs, consisting of one discrete mode Q = {q 0 } with two continuous variables x ∈ R 2 which evolve according to Here, N (·; η, ν ) denotes a Gaussian density function with mean η and covariance matrix ν 2 ; A q = [ 0.935 0 0 0.916 ] and G q = [ 1.782 0 0 0.511 ]. We are interested in computing the probability of remaining within 18 24 ] over K = 3 time steps. This can be encoded into a bounded ltl property, φ 1 := □ ≤K =3 X saf e where □ is the "always" temporal operator considered over a finite horizon K. We instantiate this shs and feed this into StocHy. We verify the model against φ 1 using both verification algorithm's within StocHy and perform a comparison with the tool faust 2 [4] with respect to the total time taken and the corresponding abstraction error for a fixed number of states. We depict the results of this comparison within Fig. 1. It can be seen that StocHy provides a significant improvement over the state of the art. □

FORMAL SYNTHESIS
StocHy carries out control (strategy, policy) synthesis via formal abstractions, employing : (i) stochastic dynamic programming; StocHy exploits the use of symbolic kernels; and (ii) robust synthesis using imdp; StocHy automates the synthesis algorithm with the abstraction procedure, and exploits the use of sparse matrices.

Strategy synthesis example
We consider a stochastic process with two modes Q = {q 0 , q 1 } and with two continuous variables x ∈ R 2 evolve using (2), with,  Fig. 2a. We would like to synthesise a strategy such that given any initial condition we avoid the "purple" region until we reach the "green" region. This requirement can be expressed as an ltl formula φ 2 := (¬purple) U дreen where U is the "until" operator and the atomic propositions {purple, дreen} denote regions within the set X = [−1.5, 1.5] 2 . We synthesise a strategy, π ⋆ , using imdp algorithm within StocHy. This generates an abstraction with a total of 2410 states, a maximum probability of 1, a maximum abstraction error of 0.21 and takes 1639.3 [s]. The lower probabilities of satisfying φ 2 for each mode are shown in Fig.2b and Fig.2c. Fig.2a shows the simulation of a trajectory under π ⋆ with a starting point of (−0.5, −1) in q 0 . □

SIMULATION
StocHy allows simulation of complex stochastic processes by means of Monte Carlo techniques; StocHy automatically generates statistics from the simulations in the form of histograms, visualising the evolution of both the continuous random variables and the discrete modes. We consider a shs consisting of Q = {q 0 , q 1 , q 2 , q 3 } with the Lower probabilities of satisfying φ 2 for q 0 (b) and q 1 (c).
where A q , B q , G q are appropriately sized matrices, N q,i represents the bilinear influence of the i−th input component u i . The actual values of the matrices A q , B q , G q , N q are provided within the tool distribution. We depict the mc for the discrete modes and the input control signal u within Fig.3a. We simulate the evolution of this dynamical model over a fixed time horizon K = 32 steps, with an initial