Muhammad Rezaul Karim

Muhammad Rezaul Karim, David Messinger, Ye Yang et al.

Crowdsourced software development (CSD) offers a series of specified tasks to a large crowd of trustworthy software workers. Topcoder is a leading platform to manage the whole process of CSD. While increasingly accepted as a realistic option for software development, preliminary analysis on Topcoder's software crowd worker behaviors reveals an alarming task-quitting rate of 82.9%. In addition, a substantial number of tasks do not receive any successful submission. In this paper, we report about a methodology to improve the efficiency of CSD. We apply massive data analytics and machine leaning to (i) perform comparative analysis on alternative technique analysis to predict likelihood of winners and quitters for each task, (ii) significantly reduce the amount of non-succeeding development effort in registered but inappropriate tasks, (iii) identify and rank the most qualified registered workers for each task, and (iv) provide reliable prediction of tasks risky to get any successful submission. Our results and analysis show that Random Forest (RF) based predictive technique performs best among the alternative techniques studied. Applying RF, the tasks recommended to workers can reduce the amount of non-succeeding development effort to a great extent. On average, over a period of 30 days, the savings are 3.5 and 4.6 person-days per registered tasks for experienced resp. unexperienced workers. For the task-related recommendations of workers, we can accurately recommend at least 1 actual winner in the top ranked workers, particularly 94.07% of the time among the top-2 recommended workers for each task. Finally, we can predict, with more than 80% F-measure, the tasks likely not getting any submission, thus triggering timely corrective actions from CSD platforms or task requesters.

Muhammad Rezaul Karim

A Constraint Satisfaction Problem (CSP) is a framework used for modeling and solving constrained problems. Tree-search algorithms like backtracking try to construct a solution to a CSP by selecting the variables of the problem one after another. The order in which these algorithm select the variables potentially have significant impact on the search performance. Various heuristics have been proposed for choosing good variable ordering. Many powerful variable ordering heuristics weigh the constraints first and then utilize the weights for selecting good order of the variables. Constraint weighting are basically employed to identify global bottlenecks in a CSP. In this paper, we propose a new approach for learning weights for the constraints using competitive coevolutionary Genetic Algorithm (GA). Weights learned by the coevolutionary GA later help to make better choices for the first few variables in a search. In the competitive coevolutionary GA, constraints and candidate solutions for a CSP evolve together through an inverse fitness interaction process. We have conducted experiments on several random, quasi-random and patterned instances to measure the efficiency of the proposed approach. The results and analysis show that the proposed approach is good at learning weights to distinguish the hard constraints for quasi-random instances and forced satisfiable random instances generated with the Model RB. For other type of instances, RNDI still seems to be the best approach as our experiments show.