Hoang Duong Tuan

Haixiang Sun, Ye Shi, Jingya Wang et al.

The idea of embedding optimization problems into deep neural networks as optimization layers to encode constraints and inductive priors has taken hold in recent years. Most existing methods focus on implicitly differentiating Karush-Kuhn-Tucker (KKT) conditions in a way that requires expensive computations on the Jacobian matrix, which can be slow and memory-intensive. In this paper, we developed a new framework, named Alternating Differentiation (Alt-Diff), that differentiates optimization problems (here, specifically in the form of convex optimization problems with polyhedral constraints) in a fast and recursive way. Alt-Diff decouples the differentiation procedure into a primal update and a dual update in an alternating way. Accordingly, Alt-Diff substantially decreases the dimensions of the Jacobian matrix especially for optimization with large-scale constraints and thus increases the computational speed of implicit differentiation. We show that the gradients obtained by Alt-Diff are consistent with those obtained by differentiating KKT conditions. In addition, we propose to truncate Alt-Diff to further accelerate the computational speed. Under some standard assumptions, we show that the truncation error of gradients is upper bounded by the same order of variables' estimation error. Therefore, Alt-Diff can be truncated to further increase computational speed without sacrificing much accuracy. A series of comprehensive experiments validate the superiority of Alt-Diff.

Wanxing Chang, Ye Shi, Hoang Duong Tuan et al.

Universal Domain Adaptation (UniDA) aims to transfer knowledge from a source domain to a target domain without any constraints on label sets. Since both domains may hold private classes, identifying target common samples for domain alignment is an essential issue in UniDA. Most existing methods require manually specified or hand-tuned threshold values to detect common samples thus they are hard to extend to more realistic UniDA because of the diverse ratios of common classes. Moreover, they cannot recognize different categories among target-private samples as these private samples are treated as a whole. In this paper, we propose to use Optimal Transport (OT) to handle these issues under a unified framework, namely UniOT. First, an OT-based partial alignment with adaptive filling is designed to detect common classes without any predefined threshold values for realistic UniDA. It can automatically discover the intrinsic difference between common and private classes based on the statistical information of the assignment matrix obtained from OT. Second, we propose an OT-based target representation learning that encourages both global discrimination and local consistency of samples to avoid the over-reliance on the source. Notably, UniOT is the first method with the capability to automatically discover and recognize private categories in the target domain for UniDA. Accordingly, we introduce a new metric H^3-score to evaluate the performance in terms of both accuracy of common samples and clustering performance of private ones. Extensive experiments clearly demonstrate the advantages of UniOT over a wide range of state-of-the-art methods in UniDA.

Ho N. Phien, Hoang D. Tuan, Johann A. Bengua et al.

This paper proposes a novel formulation of the tensor completion problem to impute missing entries of data represented by tensors. The formulation is introduced in terms of tensor train (TT) rank which can effectively capture global information of tensors thanks to its construction by a well-balanced matricization scheme. Two algorithms are proposed to solve the corresponding tensor completion problem. The first one called simple low-rank tensor completion via tensor train (SiLRTC-TT) is intimately related to minimizing the TT nuclear norm. The second one is based on a multilinear matrix factorization model to approximate the TT rank of the tensor and called tensor completion by parallel matrix factorization via tensor train (TMac-TT). These algorithms are applied to complete both synthetic and real world data tensors. Simulation results of synthetic data show that the proposed algorithms are efficient in estimating missing entries for tensors with either low Tucker rank or TT rank while Tucker-based algorithms are only comparable in the case of low Tucker rank tensors. When applied to recover color images represented by ninth-order tensors augmented from third-order ones, the proposed algorithms outperforms the Tucker-based algorithms.

Y. Shi, H. D. Tuan, P. Apkarian et al.

Optimal power flow (OPF) over power transmission networks poses challenging large-scale nonlinear optimization problems, which involve a large number of quadratic equality and indefinite quadratic inequality constraints. These computationally intractable constraints are often expressed by linear constraints plus matrix additional rank-one constraints on the outer products of the voltage vectors. The existing convex relaxation technique, which drops the difficult rank-one constraints for tractable computation, cannot yield even a feasible point. We address these computationally difficult problems by an iterative procedure, which generates a sequence of improved points that converge to a rank-one solution. Each iteration calls a semi-definite program. Intensive simulations for the OPF problems over networks with a few thousands of buses are provided to demonstrate the efficiency of our approach. The suboptimal values of the OPF problems found by our computational procedure turn out to be the global optimal value with computational tolerance less than 0.01%.

Y. Shi, H. D. Tuan, A. A. Nasir et al.

In this paper, phasor measurement unit (PMU) placement for power grid state estimation under different degrees of observability is studied. Observability degree is the depth of the buses' reachability by the placed PMUs and thus constitutes an important characteristic for PMU placement. However, the sole observability as addressed in many works still does not guarantee a good estimate for the grid state. Some existing works also considered the PMU placement for minimizing the mean squared error or maximizing the mutual information between the measurement output and grid state. However, they ignore the observability requirements for computational tractability and thus potentially lead to artificial results such as acceptance of the estimate for an unobserved state component as its unconditional mean. In this work, the PMU placement optimization problem is considered by minimizing the mean squared error or maximizing the mutual information between the measurement output and grid state, under grid observability constraints. The provided solution is free from the mentioned fundamental drawbacks in the existing PMU placement designs. The problems are posed as binary nonlinear optimization problems, for which this paper develops efficient algorithms for computational solutions. The performance of the proposed algorithms is analyzed in detail through numerical examples on large-scale IEEE power networks.

Y. Shi, H. D. Tuan, A. V. Savkin

The problem of joint coordination of plug-in electric vehicles (PEVs) charging and grid power control is to minimize both PEVs charging cost and energy generation cost while meeting both residential and PEVs' power demands and suppressing the potential impact of PEVs integration. A bang-bang PEV charging strategy is adopted to exploit its simple online implementation, which requires computation of a mixed integer nonlinear programming problem (MINP) in binary variables of the PEV charging strategy and continuous variables of the grid voltages. A new solver for this MINP is proposed. Its efficiency is shown by numerical simulations.

Y. Shi, H. D. Tuan, T. Q. Duong et al.

The success of the transportation electricification in this century particularly requires the penentration of the internet of plug-in electric vehicles (PEVs) into the smart power grid. Beside the function of serving the traditional residential power demand, next-generation power grids also aim to support the internet of PEVs at the same time. The distinct difference between the traditional power demand and PEVs' power demand is that while the statistics of the former is rich enough for treating it as inelastic/known before hand, the latter is unknown until random PEVs' arrivals. Massive penentration of PEVs certainly causes the grid unpredictable fluctuation. The present paper considers the joint PEVs charging coordination and grid power generation to minimizing both of the negative impact of PEVs' integration and the cost of power generation while meeting the grid operating constraints and all parties' demand. The bang-bang PEVs charging strategy is adopted to exploit its simple implementation. By using a recently developed model predictive control (MPC) model for this problem, the online compuation is based on a predictive mixed integer nonlinear programming (MINP). A new solution computation for this optimization problem is developed. Its capacity of achieving the globally optimal solution is shown by numerical comparison between its performance and that by an off-line optimal solution.

Y. Shi, H. D. Tuan

Proportional-integral-derivative (PID) structured controller is the most popular class of industrial control but still could not be appropriately exploited in fuzzy systems. To gain the practicability and tractability of fuzzy systems, this paper develops a parameterized bilinear matrix inequality characterization for the ${\cal H}_{\infty}$ fuzzy PID control design, which is then relaxed into a bilinear matrix inequality optimization problem of nonconvex optimization. Several computational procedures are then developed for its solution. The merit of the developed algorithms is shown through the benchmark examples.

Y. Shi, H. D. Tuan, A. V. Savkin et al.

Next-generation power grids will likely enable concurrent service for residences and plug-in electric vehicles (PEVs). While the residence power demand profile is known and thus can be considered inelastic, the PEVs' power demand is only known after random PEVs' arrivals. PEV charging scheduling aims at minimizing the potential impact of the massive integration of PEVs into power grids to save service costs to customers while power control aims at minimizing the cost of power generation subject to operating constraints and meeting demand. The present paper develops a model predictive control (MPC)- based approach to address the joint PEV charging scheduling and power control to minimize both PEV charging cost and energy generation cost in meeting both residence and PEV power demands. Unlike in related works, no assumptions are made about the probability distribution of PEVs' arrivals, the known PEVs' future demand, or the unlimited charging capacity of PEVs. The proposed approach is shown to achieve a globally optimal solution. Numerical results for IEEE benchmark power grids serving Tesla Model S PEVs show the merit of this approach.

Johann A. Bengua, Ho N. Phien, Hoang D. Tuan et al.

This paper introduces matrix product state (MPS) decomposition as a new and systematic method to compress multidimensional data represented by higher-order tensors. It solves two major bottlenecks in tensor compression: computation and compression quality. Regardless of tensor order, MPS compresses tensors to matrices of moderate dimension which can be used for classification. Mainly based on a successive sequence of singular value decompositions (SVD), MPS is quite simple to implement and arrives at the global optimal matrix, bypassing local alternating optimization, which is not only computationally expensive but cannot yield the global solution. Benchmark results show that MPS can achieve better classification performance with favorable computation cost compared to other tensor compression methods.

Johann A. Bengua, Hoang D. Tuan, Ho N. Phien et al.

This paper proposes a novel framework called concatenated image completion via tensor augmentation and completion (ICTAC), which recovers missing entries of color images with high accuracy. Typical images are second- or third-order tensors (2D/3D) depending if they are grayscale or color, hence tensor completion algorithms are ideal for their recovery. The proposed framework performs image completion by concatenating copies of a single image that has missing entries into a third-order tensor, applying a dimensionality augmentation technique to the tensor, utilizing a tensor completion algorithm for recovering its missing entries, and finally extracting the recovered image from the tensor. The solution relies on two key components that have been recently proposed to take advantage of the tensor train (TT) rank: A tensor augmentation tool called ket augmentation (KA) that represents a low-order tensor by a higher-order tensor, and the algorithm tensor completion by parallel matrix factorization via tensor train (TMac-TT), which has been demonstrated to outperform state-of-the-art tensor completion algorithms. Simulation results for color image recovery show the clear advantage of our framework against current state-of-the-art tensor completion algorithms.

Johann A. Bengua, Ho N. Phien, Hoang D. Tuan et al.

This paper introduces matrix product state (MPS) decomposition as a computational tool for extracting features of multidimensional data represented by higher-order tensors. Regardless of tensor order, MPS extracts its relevant features to the so-called core tensor of maximum order three which can be used for classification. Mainly based on a successive sequence of singular value decompositions (SVD), MPS is quite simple to implement without any recursive procedure needed for optimizing local tensors. Thus, it leads to substantial computational savings compared to other tensor feature extraction methods such as higher-order orthogonal iteration (HOOI) underlying the Tucker decomposition (TD). Benchmark results show that MPS can reduce significantly the feature space of data while achieving better classification performance compared to HOOI.