Yan Wang

Zhisong Zhang, Yan Wang, Xinting Huang et al. · tencent-ai

Large language models have shown remarkable performance across a wide range of language tasks, owing to their exceptional capabilities in context modeling. The most commonly used method of context modeling is full self-attention, as seen in standard decoder-only Transformers. Although powerful, this method can be inefficient for long sequences and may overlook inherent input structures. To address these problems, an alternative approach is parallel context encoding, which splits the context into sub-pieces and encodes them parallelly. Because parallel patterns are not encountered during training, naively applying parallel encoding leads to performance degradation. However, the underlying reasons and potential mitigations are unclear. In this work, we provide a detailed analysis of this issue and identify that unusually high attention entropy can be a key factor. Furthermore, we adopt two straightforward methods to reduce attention entropy by incorporating attention sinks and selective mechanisms. Experiments on various tasks reveal that these methods effectively lower irregular attention entropy and narrow performance gaps. We hope this study can illuminate ways to enhance context modeling mechanisms.

Wang Lituan, Zhang Lei, Wang Yan et al.

Weakly-supervised medical image segmentation is gaining traction as it requires only rough annotations rather than accurate pixel-to-pixel labels, thereby reducing the workload for specialists. Although some progress has been made, there is still a considerable performance gap between the label-efficient methods and fully-supervised one, which can be attributed to the uncertainty nature of these weak labels. To address this issue, we propose a novel weak annotation method coupled with its learning framework EAUWSeg to eliminate the annotation uncertainty. Specifically, we first propose the Bounded Polygon Annotation (BPAnno) by simply labeling two polygons for a lesion. Then, the tailored learning mechanism that explicitly treat bounded polygons as two separated annotations is proposed to learn invariant feature by providing adversarial supervision signal for model training. Subsequently, a confidence-auxiliary consistency learner incorporates with a classification-guided confidence generator is designed to provide reliable supervision signal for pixels in uncertain region by leveraging the feature presentation consistency across pixels within the same category as well as class-specific information encapsulated in bounded polygons annotation. Experimental results demonstrate that EAUWSeg outperforms existing weakly-supervised segmentation methods. Furthermore, compared to fully-supervised counterparts, the proposed method not only delivers superior performance but also costs much less annotation workload. This underscores the superiority and effectiveness of our approach.

Hamed Javidi, Dan Simon, Ling Zhu et al.

The ultimate goal of ridesharing systems is to matchtravelers who do not have a vehicle with those travelers whowant to share their vehicle. A good match can be found amongthose who have similar itineraries and time schedules. In thisway each rider can be served without any delay and also eachdriver can earn as much as possible without having too muchdeviation from their original route. We propose an algorithmthat leverages biogeography-based optimization to solve a multi-objective optimization problem for online ridesharing. It isnecessary to solve the ridesharing problem as a multi-objectiveproblem since there are some important objectives that must beconsidered simultaneously. We test our algorithm by evaluatingperformance on the Beijing ridesharing dataset. The simulationresults indicate that BBO provides competitive performancerelative to state-of-the-art ridesharing optimization algorithms.