Few-Shot Meta-Learning in Heterogeneous Contexts

Few-shot learning has become an effective technique for helping parametric models improve performance on various tasks by leveraging knowledge from similar datasets. However, existing few-shot learning approaches are limited in their application to specific data types and tasks. This thesis addresses the challenges of few-shot learning in different domains: unstructured data, time series forecasting, and human motion prediction.
The first part of the thesis introduces Chameleon, a novel model that enables meta-learning weight initialization across tasks with different schemas in unstructured data. Chameleon learns to align predictor schemas to a common representation, allowing for successful parameter initializations across diverse tasks. Experimental results on various datasets demonstrate the effectiveness of Chameleon in achieving few-shot classification on unstructured data.
In the second part, the thesis presents HIDRA, a meta-learning approach designed to tackle the problem of few-shot learning in time series forecasting with varying numbers of target variables. HIDRA employs a master neuron to initialize output neurons for new tasks, providing a flexible and robust solution for training and evaluating across datasets with different target variable counts. Extensive experiments validate the superiority of HIDRA over standard approaches, especially in low-capacity models and complex tasks with a high number of classes.
In the third part of the thesis, we present TimeHetNet. We formalize the problem of few-shot time series forecasting with heterogeneous channels for the first time. Extending recent work on heterogeneous attributes in vector data, we develop a model composed of permutation-invariant deep set blocks incorporating a temporal embedding. We assemble the first meta-dataset of 40 multivariate time-series datasets and show through experiments that our model provides a good generalization, outperforming baselines from more straightforward scenarios that either fail to learn across tasks or miss temporal information.
The final part of the thesis focuses on few-shot motion prediction, a challenging task of predicting future sensor readings of motion body sensors. To address this, the thesis proposes a novel extension of the TimeHetNet model incorporating graph neural networks and deep set blocks to explicitly include spatial graph information while generalizing motion tasks with heterogeneous sensors. Experimental evaluations on various motion tasks showcase significant performance improvements compared to state-of-the-art models, demonstrating the model's effectiveness in capturing spatial and temporal information.
Overall, this thesis contributes to the advancement of few-shot learning techniques in diverse domains, providing valuable insights and innovative solutions to enhance the performance of parametric models in scenarios with limited labeled data.