Srdjan Stanković

Milos S. Stankovic, Marko Beko, Srdjan S. Stankovic

In this paper we propose several novel distributed gradient-based temporal difference algorithms for multi-agent off-policy learning of linear approximation of the value function in Markov decision processes with strict information structure constraints, limiting inter-agent communications to small neighborhoods. The algorithms are composed of: 1) local parameter updates based on single-agent off-policy gradient temporal difference learning algorithms, including eligibility traces with state dependent parameters, and 2) linear stochastic time varying consensus schemes, represented by directed graphs. The proposed algorithms differ by their form, definition of eligibility traces, selection of time scales and the way of incorporating consensus iterations. The main contribution of the paper is a convergence analysis based on the general properties of the underlying Feller-Markov processes and the stochastic time varying consensus model. We prove, under general assumptions, that the parameter estimates generated by all the proposed algorithms weakly converge to the corresponding ordinary differential equations (ODE) with precisely defined invariant sets. It is demonstrated how the adopted methodology can be applied to temporal-difference algorithms under weaker information structure constraints. The variance reduction effect of the proposed algorithms is demonstrated by formulating and analyzing an asymptotic stochastic differential equation. Specific guidelines for communication network design are provided. The algorithms' superior properties are illustrated by characteristic simulation results.

Andjela Draganic, Milan Maric, Irena Orovic et al.

Although the protection of ownership and the prevention of unauthorized manipulation of digital images becomes an important concern, there is also a big issue of image source origin authentication. This paper proposes a procedure for the identification of the image source and content by using the Public Key Cryptography Signature (PKCS). The procedure is based on the PKCS watermarking of the images captured with numerous automatic observing cameras in the Trap View cloud system. Watermark is created based on 32-bit PKCS serial number and embedded into the captured image. Watermark detection on the receiver side extracts the serial number and indicates the camera which captured the image by comparing the original and the extracted serial numbers. The watermarking procedure is designed to provide robustness to image optimization based on the Compressive Sensing approach. Also, the procedure is tested under various attacks and shows successful identification of ownership.