Document Type

Conference Paper

Publication Date




Publication Title

Proceedings of the 30th ACM International Conference on Multimedia (MM '22)



Conference Name

ACM Multimedia 2022, The 30th ACM International Conference on Multimedia, October 10-14, 2022, Lisboa, Portugal


This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. A plausible idea to deal with such data streams is to establish a relationship between the old and new feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional multimedia data with complex feature interplay, which suffers a tradeoff between onlineness, which biases shallow learners, and expressiveness, which requires deep models. Motivated by this, we propose a novel OLD3S paradigm, where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building an intermediate feature mapping relationship. A key trait of OLD3S is to treat the model capacity as a learnable semantics, aiming to yield optimal model depth and parameters jointly in accordance with the complexity and non-linearity of the input data streams in an online fashion. Both theoretical analysis and empirical studies substantiate the viability and effectiveness of our proposed approach. The code is available online at


© 2022 Copyright held by the owner/authors.

This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.

Original Publication Citation

Lian, H., Atwood, J. S., Hou, B. J., Wu, J., & He, Y. (2022). Online deep learning from doubly-streaming data. In Proceedings of the 30th ACM International Conference on Multimedia (MM '22), Oct. 10-14, 2022, Lisboa, Portugal (pp. 3185-3194). Association for Computing Machinery.


0000-0002-6869-784X (Atwood), 0000-0003-0173-4463 (Wu), 0000-0002-5357-6623 (He)