Google Scholar / Semantic Scholar / Github / LinkedIn / Instagram / Galary
Yao Fu 符尧. yao.fu@ed.ac.uk
I am a Ph.D. student at the University of Edinburgh (2020-) with professor Mirella Lapata. I finished my M.S. at Columbia University (2018-2020) with professor John Cunningham and my B.S. at Peking University (2013-2018) with professor Yansong Feng. Before Ph.D., I spent great time visiting professor Alexander Rush at Cornell University (2019-2020).
I study probabilistic models for human language.
I build generative models guided by Bayesian principles, equipped with large language models, utilizing efficient inference, and grounded to real-world language scenarios.
My research is equally distributed into Probabilistic Machine Learning and Natural Language Processing. Usually, NLP people think I’m doing ML while ML people think I’m doing NLP. My publications cover the following topics:
- NLP: Language Generation; Linguistic Structure Prediction
- ML: Generative Models with Discrete Structures; Efficient Inference Algorithms
I’m maintaining the following reading lists as research roadmaps:
- Deep Generative Models for Natural Language Processing. (github)
- Compositional Generalization in Natural Language Processing. (github)
Preprints
- [Arvix 2022] Latent Topology Induction for Understanding Contextualized Representations. [paper]
- Yao Fu and Mirella Lapata
- Discover a latent network within the representation space of large language models.
Publications
- [ICML 2022] Scaling Structured Inference with Randomization. [paper]
- Yao Fu, John P. Cunningham and Mirella Lapata
- A family of randomized dynamic programming algorithms for scaling up classical structured prediction algorithms of different inferences (partition, marginal, entropy, reparameterization) of structures (chains, trees, and general sum-product).
- [TACL 2022] Data-to-text Generation with Variational Sequential Planning.[paper][code]
- Ratish Puduppully, Yao Fu, Mirella Lapata
- A latent planning model for generating very long document.
- [NAACL 2021] Noisy Labeled NER with Confidence Estimation. [paper][code]
- Kun Liu*, Yao Fu*, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang and Sheng Gao. *Equal contribution.
- A confidence estimation method for estimating label noise in NER annotations and a training method based on partial marginalization according to estimated noise.
- [ICLR 2021] Probing BERT in Hyperbolic Spaces. [paper][code]
- Boli Chen*, Yao Fu*, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, Liping Jing. *Equal contribution.
- A Poincare probe for recovering hierarchical structures from contextualized representations. Applied to probing syntax and sentiment in BERT.
- [ICLR 2021] Prototypical Representation Learning for Relation Extraction. [paper][code]
- Ning Ding, Xiaobin Wang, Yao Fu, Guangwei Xu, Rui Wang, Pengjun Xie, Ying Shen, Fei Huang, Hai-Tao Zheng, Rui Zhang
- A representation learning method for embedding relation prototypes on hyperspheres. Applied to supervised, semi-supervised, and few-shot relational learning.
- [AAAI 2021] Nested Named Entity Recognition with Partially Observed TreeCRFs. [paper][code]
- Yao Fu*, Chuanqi Tan*, Mosha Chen, Songfang Huang, Fei Huang. *Equal contribution.
- A Masked Inside algorithm for efficient partial marginalization of TreeCRFs. Applied to Nested NER.
- [NeurIPS 2020] Latent Template Induction with Gumbel-CRFs. [paper][code]
- Yao Fu, Chuanqi Tan, Mosha Chen, Bin Bi, Yansong Feng and Alexander Rush.
- A Gumbel-FFBS algorithm for reparameterizing and relaxing CRFs. Applied to controllable text generation with latent templates.
- [NeurIPS 2019] Paraphrase Generation with Latent Bag of Words. [paper][code]
- Yao Fu, Yansong Feng and John Cunningham.
- A differentiable planning and realization model with latent bag of words by Gumbel-topK reparameterization. Applied to paraphrase generation.
- [INLG 2019] Rethinking Text Attribute Transfer: A Lexical Analysis. [paper][code]
- Yao Fu, Hao Zhou, Jiaze Chen and Lei Li.
- A series of text mining algorithms for discovering words with strong influence on classification. Applied to analysing text attribute transfer models.
- [NAACL 2018] Natural Answer Generation with Heterogeneous Memory. [paper]
- Yao Fu and Yansong Feng.
- An attention mechanism fusing information from different source of knowledge. Applied to answer sentence generation.
Teaching
- Peking University. Empirical Methods for Natural Language Processing. 2022 Spring.
- Guest lecture on Text Generation. Tought by Yansong Feng.
- University of Edinburgh. Natural Language Understanding. 2022 Spring.
- Teaching Assistant. Tought by Alexandra Birch, Frank Keller, and Laura Perez.
- University of Edinburgh. Probabilistic Modeling and Reasoning. 2022 Spring.
- Teaching Assistant. Tought by Michael Gutmann.
- Peking University. Empirical Methods for Natural Language Processing. 2021 Spring.
- Guest lecture on Text Generation. Tought by Yansong Feng.
- Alibaba DAMO Academy. Advanced Probabilistic Machine Learning Seminar. 2020 Spring.
- Instructor.
- Columbia University. COMS 4995 Applied Machine Learning, 2019 Spring.
- Course Assistant. Tought by Andreas Muller.
Resources and Tutorials
- Why S4 is Good at Long Sequence: Remembering a Sequence with Online Function Approximation. Feb 2022.
- Deep Structured Prediction: Inference, Reparameterization and Applications. Jun 2021 [pdf]
- How to write Variational Inference and Generative Models for NLP: a recipe. Mar 2021 [pdf]
Internships
- Jun 22 - Aug 22. Allen Institute for Artificial Intelligence. Research Intern. Seattle.
- with Dr. Tushar Khot
- Jan 20 - Oct 20. Alibaba Damo Academy. Research Intern. Beijing and Hangzhou
- with Dr. Chuanqi Tan and Mosha Chen
- May 19 - Aug 19. Tencent AI Lab. Research Intern. Seattle
- Dec 17 - Aug 18. Bytedance AI Lab. Research Intern. Beijing