MTO

Multitask Optimization

[Wikipedia]

Introduction

Multitask Optimization is a paradigm in the optimization literature that focuses on solving multiple self-contained tasks at the same time. Inspired by the well-established concepts of transfer learning and multi-task learning in predictive analytics, the key motivation behind multitask optimization is that if optimization tasks are related to each other (in terms of their optimal solutions, or the general characteristics of their function landscapes), then the search progress on one can be transferred to substantially speedup the search on the other. Notably, the success of the paradigm is not necessarily limited to one-way knowledge transfers from simpler to more complex tasks. In fact, in an attempt to intentionally solve a harder task, several simpler ones may often be unintentionally solved.

News

Contact us


If you have any question, please contact us.


Email: liangf@cqu.edu.cn     stone_zhou@cqu.edu.cn.

Multitask Optimization

[Wikipedia]

Download

  • B. Da, Y. S. Ong, L. Feng, A.K. Qin, A. Gupta, Z. Zhu, C. K. Ting, K. Tang, and X. Yao, "Evolutionary Multitasking for Single-objective Continuous Optimization: Benchmark Problems, Performance Metric, and Baseline Results", Technical Report, 2016.

    Single-Objective MFO benchmark problems and baseline codes are available.Click here!

  • Y. Yuan, Y. S. Ong., L. Feng, A.K. Qin, A. Gupta., B. Da, Q. Zhang, K. C. Tan, Y. Jin, and H. Ishibuchi, "Evolutionary Multitasking for Multiobjective Continuous Optimization: Benchmark Problems, Performance Metrics and Baseline Results", Technical Report, 2016.

    Multi-Objective MFO benchmark problems and baseline codes are available.Click here!

New MTO Benchmarks for CEC 2019 Competition on Evolutionary Multi-task Optimization :


  • New Single-Objective Complex 2-task benchmark problems are available.Click here!

  • New Multi-Objective Complex 2-task benchmark problems are available.Click here!

  • New Single-Objective Manytask(50-Tasks) benchmark problems are available.Click here!

  • New Multi-Objective Manytask(50-Tasks) benchmark problems are available.Click here!


    • Multitask Optimization

    [Wikipedia]

    Methods

    In the existing literature, two common approaches for multitask optimization span Bayesian optimization and evolutionary computation.

    • Multitask Bayesian optimization is a recent model-based approach that leverages the concept of knowledge transfer to speedup the automatic hyperparameter optimization process of machine learning algorithms. The method builds a multitask Gaussian process model on the data originating from different searches progressing in tandem. The captured inter-task dependencies are thereafter utilized to better inform the subsequent sampling of candidate solutions in the respective search spaces.

    • Evolutionary multitasking has been explored as a means of exploiting the implicit parallelism of population-based search algorithms to simultaneously progress multiple distinct optimization tasks. By mapping all tasks to a unified search space, the evolving population of candidate solutions can harness the hidden relationships between them through continuous genetic transfer - which is induced when solutions associated with different tasks crossover with each other. More recently, modes of knowledge transfer that are different from direct solution crossover have been explored.

    Source Codes


    • Matlab code for "Multifactorial Evolutionary Algorithm with Online Transfer Parameter Estimation: MFEA-II" can be downloaded here.

    • Matlab alpha version of MFEA can be downloaded here.

    • Matlab beta version of MFEA for Continuous Optimization can be downloaded here.

    • Matlab alpha version for multi-objective continuous optimization via multitasking MO-MFEA available here.

    • Matlab code of MFPSO and MFDE can be downloaded here (MFPSO and MFDE).

    • Matlab code for "Curbing Negative Influences Online for Seamless Transfer Evolutionary Optimization" can be downloaded here.

    • Matlab code for the autoencoding component of "Evolutionary Multitasking via Explicit Autoencoding" can be downloaded here.

    Multitask Optimization

    [Wikipedia]

    Publications


    • Y. Chen, J. Zhong, L. Feng & J. Zhang. “An Adaptive Archive-Based Evolutionary Framework for Many-Task Optimization”. IEEE Transactions on Emerging Topics in Computational Intelligence, 2019.

    • J. Yin, A. Zhu, Z. Zhu, Y. Yu & X. Ma, “Multifactorial Evolutionary Algorithm Enhanced with Cross-task Search Direction”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • L. Zhou, L. Feng, K. Liu, C. Chen, S. Deng, T. Xiang & S. Jiang. “Towards Effective Mutation for Knowledge Transfer in Multifactorial Differential Evolution”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • X. Zheng, Y. Lei, A. K. Qin, D. Zhou, J. Shi & M. Gong. “Differential Evolutionary Multi-task Optimization”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • A. Gupta & Y. S. Ong. “Back to the roots: Multi-x evolutionary computation”. Cognitive Computation, 2019.

    • A. Rauniyar, R. Nath & P. K. Muhuri. “Multi-factorial evolutionary algorithm based novel solution approach for multi-objective pollution-routing problem”. Computers & Industrial Engineering, 2019.

    • Z. Liang, J. Zhang, L. Feng & Z. Zhu. “A hybrid of genetic transform and hyper-rectangle search strategies for evolutionary multi-tasking”. Expert Systems with Applications, 2019.

    • H. T. T. Binh, N. Q. Tuan & D. C. T. Long. “A multi-objective multi-factorial evolutionary algorithm with reference-point-based approach”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • Q. Shang, L. Zhang, L. Feng, Y. Hou, J. Zhong, A. Gupta & H. L. Liu. “A Preliminary Study of Adaptive Task Selection in Explicit Evolutionary Many-Tasking”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • C. Jin, P. W. Tsai & A. K. Qin. “ A Study on Knowledge Reuse Strategies in Multitasking Differential Evolution”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • Z. Tang, M. Gong, F. Jiang, H. Li & Y. Wu. “Multipopulation Optimization for Multitask Optimization”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • A. Gupta & Y. S. Ong. “Multitask Knowledge Transfer Across Problems”. Memetic Computation, 2019.

    • H. Song, A. K. Qin, P. W. Tsai & J. J. Liang. “Multitasking Multi-Swarm Optimization”. IEEE Congress on Evolutionary Computation (CEC), 2019.

    • R. T. Liaw & C. K. Ting, “Evolutionary manytasking optimization based on symbiosis in biocoenosis”. AAAI Conference on Artificial Intelligence, 2019.

    • C. Wang, H. Ma, G. Chen & S. Hartmann. “Evolutionary Multitasking for Semantic Web Service Composition”. arXiv preprint arXiv:1902.06370., 2019.

    • Y. Lian, Z. Huang, Y. Zhou & Z. Chen. “Improve Theoretical Upper Bound of Jumpk Function by Evolutionary Multitasking”. High Performance Computing and Cluster Technologies Conference, 2019.

    • X. Zheng, A. K. Qin, M. Gong & D. Zhou. “Self-regulated Evolutionary Multi-task Optimization”. IEEE Transactions on Evolutionary Computation, 2019.

    • K. K. Bali, Y. Ong, A. Gupta & P. S. Tan. “Multifactorial Evolutionary Algorithm with Online Transfer Parameter Estimation: MFEA-II”. IEEE Transactions on Evolutionary Computation, 2019.

    • M. Gong, Z. Tang, H. Li & J. Zhang. “Evolutionary Multitasking with Dynamic Resource Allocating Strategy”. IEEE Transactions on Evolutionary Computation, 2019.

    • Amit Rauniyar, Rahul Nath & Pranab K. Muhuri. “Multi-factorial evolutionary algorithm based novel solution approach for multi-objective pollution-routing problem”. Computers & Industrial Engineering , 2019.

    • H. Li, Y. Ong, M. Gong & Z. Wang. “Evolutionary Multitasking Sparse Reconstruction: Framework and Case Study”. IEEE Transactions on Evolutionary Computation , 2018.

    • H. ThiThanh Binh, P. Dinh Thanh, T. Ba Trung & L. Phuong Thao. “Effective Multifactorial Evolutionary Algorithm for Solving the Cluster Shortest Path Tree Problem”. IEEE Congress on Evolutionary Computation (CEC), 2018.

    • D. Liu, S. Huang & J. Zhong. “Surrogate-Assisted Multi-Tasking Memetic Algorithm”. IEEE Congress on Evolutionary Computation (CEC), 2018.

    • Z. Tang & M. Gong. “Adaptive multifactorial particle swarm optimisation”. CAAI Transactions on Intelligence Technology, 2018.

    • Y. Chen, J. Zhong & M. Tan. "A Fast Memetic Multi-Objective Differential Evolution for Multi-Tasking Optimization”. IEEE Congress on Evolutionary Computation (CEC), 2018.

    • Yang, C., Ding, J., Jin, Y., Wang, C., & Chai, T. “Multitasking Multiobjective Evolutionary Operational Indices Optimization of Beneficiation Processes”. IEEE Transactions on Automation Science and Engineering, 2018.

    • Tang, J., Chen, Y., Deng, Z., Xiang, Y. and Joy, C.P. “A Group-based Approach to Improve Multifactorial Evolutionary Algorithm”. IJCAI 2018.

    • Hashimoto, Ryuichi, et al. "Analysis of evolutionary multi-tasking as an island model." Genetic and Evolutionary Computation Conference (GECCO), 2018.

    • Tuan, N. Q., Hoang, T. D., & Binh, H. T. T. “A Guided Differential Evolutionary Multi-Tasking with Powell Search Method for Solving Multi-Objective Continuous Optimization”S. IEEE Congress on Evolutionary Computation (CEC), 2018.

    • Li, G., Zhang, Q., & Gao, W. “Multipopulation evolution framework for multifactorial optimization”. Genetic and Evolutionary Computation Conference (GECCO), 2018.

    • Zhong, J., Feng, L., Cai, W. and Ong, Y.S., “Multifactorial Genetic Programming for Symbolic Regression Problems”. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018.

    • Thanh, P.D., Dung, D.A., Tien, T.N. and Binh, H.T.T., “An Effective Representation Scheme in Multifactorial Evolutionary Algorithm for Solving Cluster Shortest-Path Tree Problem”. IEEE Congress on Evolutionary Computation (CEC), 2018.

    • Gupta, A., Ong, Y.S. and Feng, L., “Insights on transfer optimization: Because experience is the best teacher”. IEEE Transactions on Emerging Topics in Computational Intelligence, 2018.

    • Feng L, Zhou L, Zhong J, et al. “Evolutionary Multitasking via Explicit Autoencoding”. IEEE Transactions on Cybernetics, 2018.

    • Bao, L., Qi, Y., Shen, M., Bu, X., Yu, J., & Li, Q., et al. "An Evolutionary Multitasking Algorithm for Cloud Computing Service Composition". World Congress on Services. 2018, Springer.

    • Yang, C., Ding, J., Tan, K. C., & Jin, Y. "Two-stage assortative mating for multi-objective multifactorial evolutionary optimization". IEEE Annual Conference on Decision and Control (CDC) , 2018.

    • Zhou, L., Feng, L., Zhong, J., Zhu, Z., Da, B., & Wu, Z. "A study of similarity measure between tasks for multifactorial evolutionary algorithm". Genetic and Evolutionary Computation Conference (GECCO) , 2018

    • Ding, J., Yang, C., Jin, Y., & Chai, T. "Generalized multi-tasking for evolutionary optimization of expensive problems". IEEE Transactions on Evolutionary Computation ,2017, PP(99), 1-1.

    • A. Gupta, Y. S. Ong, L. Feng and K. C. Tan, "Multiobjective Multifactorial Optimization in Evolutionary Multitasking ", IEEE Transactions on Cybernetics, 2017, 1652-1665.

    • B. Da, Y. S. Ong, L. Feng, A.K. Qin, A. Gupta, etc., "Evolutionary Multitasking for Single-objective Continuous Optimization: Benchmark Problems, Performance Metric, and Baseline Results", arXiv:1706.03470, 2017.

    • Y. Yuan, Y. S. Ong, L. Feng, A.K. Qin, A. Gupta, B. Da, etc., "Evolutionary Multitasking for Multiobjective Continuous Optimization: Benchmark Problems, Performance Metrics and Baseline Results", arXiv:1706.02766, 2017.

    • Chandra, R., Ong, Y. S., and Goh, C. K., "Co-evolutionary multi-task learning with predictive recurrence for multi-step chaotic time series prediction", Neurocomputing, 2017, 21-34.

    • Q. Chen, X Ma, Y. Sun and Z. Zhu, "Adaptive Memetic Algorithm Based Evolutionary Multi-tasking Single-Objective Optimization", SEAL, 2017, 462-472.

    • Chandra, R., "Dynamic Cyclone Wind-Intensity Prediction Using Co-Evolutionary Multi-task Learning", International Conference on Neural Information Processing (ICONIP), 2017, 618-627.

    • Chandra, R., "Co-evolutionary Multi-task Learning for Modular Pattern Classification", International Conference on Neural Information Processing (ICONIP), 2017, 692-701.

    • J. Mo, Z. Fan, W. Li, Y. Fang, Y. You and X. Cai, " Multi-Factorial Evolutionary Algorithm Based on M2M Decomposition", SEAL, 2017, 134-144.

    • Chandra, R., Gupta, A., Ong, Y. S., and Goh, C. K., "Evolutionary Multi-task Learning for Modular Knowledge Representation in Neural Networks", Neural Processing Letters, 2017, 1-17.

    • Chandra, R., Ong, Y. S., and Goh, C. K., "Co-Evolutionary Multi-Task Learning for Dynamic Time Series Prediction", arXiv preprint arXiv:1703.01887, 2017.

    • Scott, E. O., and De Jong, K. A., "Multitask Evolution with Cartesian Genetic Programming", arXiv preprint arXiv:1702.02217, 2017.

    • Yuan, Y., Ong, Y. S., Gupta, A., and Xu, H., "Objective Reduction in Many-Objective Optimization: Evolutionary Multiobjective Approaches and Comprehensive Analysis", IEEE Transactions on Evolutionary Computation, In Press, 2017.

    • Tang, Z., Gong, M., and Zhang, M., "Evolutionary Multi-task Learning for Modular Extremal Learning Machine", IEEE Congress on Evolutionary Computation (CEC) 2017, pp. 474-479.

    • Liaw, R. T., and Ting, C. K., "Evolutionary Many-tasking Based on Biocoenosis through Symbiosis: A Framework and Benchmark Problems", IEEE Congress on Evolutionary Computation (CEC) 2017, pp. 2266-2273.

    • Feng, L., Zhou, W., Zhou, L., Jiang, S.W., Zhong, J.H., Da, B.S., Zhu, Z.X. and Wang, Y., "An Empirical Study of Multifactorial PSO and Multifactorial DE", IEEE Congress on Evolutionary Computation (CEC) 2017, pp. 921-928.

    • Wen, Y. W., and Ting, C. K., "Parting Ways and Reallocating Resources in Evolutionary Multitasking", IEEE Congress on Evolutionary Computation (CEC) 2017, pp. 2404-2411.

    • Chandra, R., Ong, Y. S., and Goh, C. K., "Co-evolutionary Multi-task Learning with Predictive Recurrence for Multi-Step Chaotic Time Series Prediction". Neurocomputing, 2017, 21-34.

    • M. Cheng, Y. S. Ong, A. Gupta and Z. W. Ni, "Coevolutionary Multitasking for Concurrent Global Optimization: With Case Studies in Complex Engineering Design", Engineering Applications of Artificial Intelligence, In Press, 2017.

    • K. Bali, A. Gupta, L. Feng, Y. S. Ong, and P. S. Tan, "Linearized Domain Adaptation in Evolutionary Multitasking", IEEE Congress on Evolutionary Computation, Spain, June 5-8, 2017.

    • A. Gupta, B. Da, Y. Yuan and Y. S. Ong, "On the Emerging Notion of Evolutionary Multitasking: A Computational Analog of Cognitive Multitasking", Recent Advances in Evolutionary Multi-objective Optimization, pps. 139-157, 2017.

    • L. Zhou, L. Feng, J. Zhong, Y. S. Ong, Z. Zhu, E. Sha, "Evolutionary Multitasking in Combinatorial Search Spaces: A Case Study in Capacitated Vehicle Routing Problem", IEEE Symposium Series on Computational Intelligence (SSCI) 2016, Athens, Greece.

    • Sagarna, R., and Ong, Y. S., "Concurrently Searching Branches in Software Tests Generation through Multitask Evolution", IEEE Symposium Series on Computational Intelligence (SSCI) 2016, pp. 1-8.

    • Y. Yuan, Y. S. Ong, A. Gupta, P. S. Tan, H. Xu, "Evolutionary Multitasking in Permutation-Based Combinatorial Optimization Problems: Realization with TSP, QAP, LOP, and JSP", IEEE Region Ten Conference (TENCON) 2016.

    • A. Gupta, and Y. S. Ong, "Genetic Transfer or Population Diversification? Deciphering the Secret Ingredients of Evolutionary Multitask Optimization". IEEE Symposium Series on Computational Intelligence (SSCI) 2016, Athens, Greece.

    • Ong, Y. S., "Towards Evolutionary Multitasking: A New Paradigm in Evolutionary Computation", Computational Intelligence, Cyber Security and Computational Models, pp. 25-26, Springer, Singapore, 2016.

    • R. Chandra, A. Gupta, Y. S. Ong, and C. K. Goh, "Evolutionary multi-task learning for modular training of feedforward neural networks", International Conference on Neural Information Processing (International Conference on Neural Information Processing (ICONIP)) 2016.

    • A. Gupta, Y. S. Ong, B. Da, L. Feng, and S. D. Handoko, "Measuring Complementarity between Function Landscapes in Evolutionary Multitasking", IEEE WCCI - Congress on Evolutionary Computation, 2016.

    • Wen, Y. W., and Ting, C. K., "Learning Ensemble of Decision Trees through Multifactorial Genetic Programming", IEEE WCCI - Congress on Evolutionary Computation, 2016.

    • B. Da, A. Gupta, Y. S. Ong, L. Feng, and C. Wang, "Evolutionary Multitasking Across single and Multi-Objective Formulations for Improved Problem Solving", IEEE WCCI - Congress on Evolutionary Computation, 2016.

    • Y. S. Ong and A. Gupta, "Evolutionary Multitasking: A Computer Science View of Cognitive Multitasking", Cognitive Computation, 10.1007/s12559-016-9395-7, pp 1-18, 2016.

    • Da, B., Gupta, A., Ong, Y.S. and Feng, L., "The Boon of Gene-Culture Interaction for Effective Evolutionary Multitasking". In Australasian Conference on Artificial Life and Computational Intelligence, pp. 54-65. Springer, Cham, 2016.

    • A. Gupta, J. Mańdziuk, Y. S. Ong, 'Evolutionary multitasking in bi-level optimization', Complex and Intelligent Systems, Volume 1, Issue 1-4, pp 83-95, 10.1007/s40747-016-0011-y, 2015.

    • A. Gupta, Y. S. Ong, L. Feng, 'Multifactorial Evolution: Towards Evolutionary Multitasking', IEEE Transactions on Evolutionary Computation, 2015.

    Multitask Optimization

    [Wikipedia]

    Applications


    Algorithms for multitask optimization span a wide array of real-world applications. Recent studies highlight the potential for speedups in the optimization of engineering design parameters by conducting related designs jointly in a multitask manner. In machine learning, the transfer of optimized features across related datasets can enhance the efficiency of the training process as well as improve the generalization capability of learned models. In addition to the above, the concept of multitasking has led to advances in automatic hyperparameter optimization of machine learning models, and ensemble learning.

    Applications have also been reported in cloud computing, with future developments geared toward a cloud-based on-demand optimization service that can cater to multiple customers simultaneously.