Gang Wu

Yang Zhao, Gangwei Xu, Gang Wu

Current state-of-the-art flow methods are mostly based on dense all-pairs cost volumes. However, as image resolution increases, the computational and spatial complexity of constructing these cost volumes grows at a quartic rate, making these methods impractical for high-resolution images. In this paper, we propose a novel Hybrid Cost Volume for memory-efficient optical flow, named HCV. To construct HCV, we first propose a Top-k strategy to separate the 4D cost volume into two global 3D cost volumes. These volumes significantly reduce memory usage while retaining a substantial amount of matching information. We further introduce a local 4D cost volume with a local search space to supplement the local information for HCV. Based on HCV, we design a memory-efficient optical flow network, named HCVFlow. Compared to the recurrent flow methods based the all-pairs cost volumes, our HCVFlow significantly reduces memory consumption while ensuring high accuracy. We validate the effectiveness and efficiency of our method on the Sintel and KITTI datasets and real-world 4K (2160*3840) resolution images. Extensive experiments show that our HCVFlow has very low memory usage and outperforms other memory-efficient methods in terms of accuracy. The code is publicly available at https://github.com/gangweiX/HCVFlow.

Gang Wu

We are interested in computing a simple eigenpair $(λ,{\bf x})$ of a large non-Hermitian matrix $A$, by a general harmonic Rayleigh-Ritz projection method. Given a search subspace $\mathcal{K}$ and a target point $τ$, we focus on the convergence of the harmonic Ritz vector $\widetilde{\bf x}$ and harmonic Ritz value $\widetildeλ$. In [{Z. Jia}, {\em The convergence of harmonic Ritz values, harmonic Ritz vectors, and refined harmonic Ritz vectors}, Math. Comput., 74 (2004), pp. 1441--1456.], Jia showed that for the convergence of harmonic Ritz vector and harmonic Ritz value, it is essential to assume certain Rayleigh quotient matrix being {\it uniformly nonsingular} as $\angle({\bf x},\mathcal{K})\rightarrow 0$. However, this assumption can not be guaranteed theoretically for a general matrix $A$, and the Rayleigh quotient matrix can be singular or near singular even if $τ$ is not close to $λ$. In this paper, we abolish this constraint and derive new bounds for the convergence of harmonic Rayleigh-Ritz projection methods. We show that as the distance between ${\bf x}$ and $\mathcal{K}$ tends to zero and $τ$ is satisfied with the so-called {\it uniform separation condition}, the harmonic Ritz value converges, and the harmonic Ritz vector converges as $\frac{1}{λ-τ}$ is well separated from other Ritz values of $(A-τI)^{-1}$ in the orthogonal complement of $(A-τI)\widetilde{\bf x}$ with respect to $(A-τI)\mathcal{K}$.

Aravind Reddy, Ryan A. Rossi, Zhao Song et al.

In this paper, we introduce the online and streaming MAP inference and learning problems for Non-symmetric Determinantal Point Processes (NDPPs) where data points arrive in an arbitrary order and the algorithms are constrained to use a single-pass over the data as well as sub-linear memory. The online setting has an additional requirement of maintaining a valid solution at any point in time. For solving these new problems, we propose algorithms with theoretical guarantees, evaluate them on several real-world datasets, and show that they give comparable performance to state-of-the-art offline algorithms that store the entire data in memory and take multiple passes over it.

Youngsuk Park, Ryan A. Rossi, Zheng Wen et al.

Linear quadratic regulator (LQR) is one of the most popular frameworks to tackle continuous Markov decision process tasks. With its fundamental theory and tractable optimal policy, LQR has been revisited and analyzed in recent years, in terms of reinforcement learning scenarios such as the model-free or model-based setting. In this paper, we introduce the \textit{Structured Policy Iteration} (S-PI) for LQR, a method capable of deriving a structured linear policy. Such a structured policy with (block) sparsity or low-rank can have significant advantages over the standard LQR policy: more interpretable, memory-efficient, and well-suited for the distributed setting. In order to derive such a policy, we first cast a regularized LQR problem when the model is known. Then, our Structured Policy Iteration (S-PI) algorithm, which takes a policy evaluation step and a policy improvement step in an iterative manner, can solve this regularized LQR efficiently. We further extend the S-PI algorithm to the model-free setting where a smoothing procedure is adopted to estimate the gradient. In both the known-model and model-free setting, we prove convergence analysis under the proper choice of parameters. Finally, the experiments demonstrate the advantages of S-PI in terms of balancing the LQR performance and level of structure by varying the weight parameter.

Aritra Ghosh, Saayan Mitra, Somdeb Sarkhel et al.

In programmatic advertising, ad slots are usually sold using second-price (SP) auctions in real-time. The highest bidding advertiser wins but pays only the second-highest bid (known as the winning price). In SP, for a single item, the dominant strategy of each bidder is to bid the true value from the bidder's perspective. However, in a practical setting, with budget constraints, bidding the true value is a sub-optimal strategy. Hence, to devise an optimal bidding strategy, it is of utmost importance to learn the winning price distribution accurately. Moreover, a demand-side platform (DSP), which bids on behalf of advertisers, observes the winning price if it wins the auction. For losing auctions, DSPs can only treat its bidding price as the lower bound for the unknown winning price. In literature, typically censored regression is used to model such partially observed data. A common assumption in censored regression is that the winning price is drawn from a fixed variance (homoscedastic) uni-modal distribution (most often Gaussian). However, in reality, these assumptions are often violated. We relax these assumptions and propose a heteroscedastic fully parametric censored regression approach, as well as a mixture density censored network. Our approach not only generalizes censored regression but also provides flexibility to model arbitrarily distributed real-world data. Experimental evaluation on the publicly available dataset for winning price estimation demonstrates the effectiveness of our method. Furthermore, we evaluate our algorithm on one of the largest demand-side platforms and significant improvement has been achieved in comparison with the baseline solutions.

Ryan A. Rossi, Anup Rao, Sungchul Kim et al.

In this paper, we introduce the notion of motif closure and describe higher-order ranking and link prediction methods based on the notion of closing higher-order network motifs. The methods are fast and efficient for real-time ranking and link prediction-based applications such as web search, online advertising, and recommendation. In such applications, real-time performance is critical. The proposed methods do not require any explicit training data, nor do they derive an embedding from the graph data, or perform any explicit learning. Existing methods with the above desired properties are all based on closing triangles (common neighbors, Jaccard similarity, and the ilk). In this work, we investigate higher-order network motifs and develop techniques based on the notion of closing higher-order motifs that move beyond closing simple triangles. All methods described in this work are fast with a runtime that is sublinear in the number of nodes. The experimental results indicate the importance of closing higher-order motifs for ranking and link prediction applications. Finally, the proposed notion of higher-order motif closure can serve as a basis for studying and developing better ranking and link prediction methods.

Yunjie Wang, Gang Wu

The eigenproblem of low-rank updated matrices are of crucial importance in many applications. Recently, an upper bound on the number of distinct eigenvalues of a perturbed matrix was established. The result can be applied to estimate the number of Krylov iterations required for solving a perturbed linear system. In this paper, we revisit this problem and establish some refined bounds. Some {\it a prior} upper bounds that only rely on the information of the matrix in question and the low-rank update are provided. Examples show the superiority of our theoretical results over the existing ones. The number of distinct singular values of a matrix after perturbation is also investigated.

Gang Wu, Lu Zhang

We propose a new method for computing the $φ$-functions of large sparse matrices with low rank or fast decaying singular values. The key is to reduce the computation of $φ_{\ell}$-functions of a large matrix to $φ_{\ell+1}$-functions of some $r$-by-$r$ matrices, where $r$ is the numerical rank of the large matrix in question. Some error analysis on the new method is given. Furthermore, we propose two novel strategies for estimating 2-norm condition numbers of the $φ$-functions. Numerical experiments illustrate the numerical behavior of the new algorithms and show the effectiveness of our theoretical results.