Recently, our School made another important advance in fundamental artificial intelligence research. The paper entitled No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks has been accepted for publication in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), a leading journal in the field of artificial intelligence. This marks the second paper from our School to be published in TPAMI.
The basic computational units of deep learning models are commonly referred to as “neurons.” For a long time, improvements in artificial neural networks have mainly relied on network architecture design, such as increasing network depth or width, or introducing more complex connection patterns. At the neuron level, however, most models still adopt a relatively uniform and fixed form of linear computation, namely linear neurons. When dealing with complex and diverse data, this “one-size-fits-all” neuron design limits the expressive capacity of models in capturing task-specific features.
Inspired by the diversity of neurons in the biological brain, this work proposes the new concept of “task-based neurons,” in which neurons are adaptively designed according to the characteristics of specific tasks. The study shifts the focus of artificial intelligence model design from “how to design network architectures” to “how to design optimal neurons,” offering a new perspective for fundamental research on AI models.
The paper was co-first-authored by Meng Wang, a Ph.D. student at the School of Mathematics, Harbin Institute of Technology, and Assistant Professor Fenglei Fan from the City University of Hong Kong. Professor Jianwei Ma and Professor Tieyong Zeng served as co-corresponding authors.
To realize the design of task-based neurons, the research team proposed a vectorized symbolic regression method. This method can automatically discover mathematical expressions that match task-specific data characteristics and parameterize them into trainable neuron computation rules. As a result, neuron computation is no longer restricted to predefined linear forms, but can instead develop task-adaptive expressive mechanisms. Based on this idea, the team further constructed task-based neural networks, which enhance the representation and modeling capacity of neural networks while maintaining a relatively small number of parameters.
The research team conducted systematic evaluations on synthetic data, public datasets, and real-world application scenarios. Experimental results show that task-based neurons can effectively identify the dominant patterns in data. Across multiple tasks, neural networks built with task-based neurons outperformed traditional neuron-based networks while using fewer parameters, demonstrating strong expressive power and parameter efficiency.
In real-world application tests, the research team applied the proposed method to tabular data analysis, scientific data modeling, and image classification. In real scientific data tasks such as high-energy particle collision prediction and asteroid prediction, task-based neural networks achieved better results than a variety of representative machine learning models, demonstrating their potential for complex scientific data analysis.
Beyond tabular data, the team also extended task-based neurons to image classification tasks. For several challenging categories in the ImageNet dataset, the researchers constructed more difficult test scenarios and incorporated task-based neurons into image classification networks. Experimental results show that, with a relatively small number of model parameters, task-based neural networks achieved higher recognition accuracy on multiple challenging category subsets. As the number of classification categories increased and task complexity grew, task-based neuron networks continued to maintain superior classification performance, highlighting their adaptability to complex visual tasks.
This achievement opens up a new pathway for deep learning model design, distinct from conventional approaches such as deepening networks, widening networks, or conducting architecture search. Looking ahead, the research team will continue to improve the efficiency of task-based neuron design, further explore the joint optimization of network architectures and neuron computation mechanisms, and investigate applications in computer vision, scientific computing, geophysical data processing, and high-dimensional intelligent modeling.
