The Bayesian’s Creed
Deep belief and core value are independent of temporal environment variation
| Google Scholar | Semantic Scholar | Github | Galary |
My name is Yao Fu 符尧. My email address is yao.fu@ed.ac.uk
I am a Ph.D. student at the University of Edinburgh (2020-) with Prof. Mirella Lapata. I finished my M.S. at Columbia University (2018-2020) with Prof. John Cunningham doing deep generative models and my B.S. at Peking University (2013-2018) with Prof. Yansong Feng doing text generation. Before Ph.D., I spent great time visiting Prof. Alexander Rush at Cornell University (2019-2020) doing structured latent variable models.
Although my research focus is structured prediction and text generation, I believe that to persue the most fundamental principles of human language, one must utilize a spectrum of interdisciplinary techniques: ML, NLP, Linguistics, Statistics, Optimization, Geometry, .etc. Many of my research lay in the intersection of these areas. But in general, I derive probabilistic models guided by Bayesian principles, equipped with modern neural architectures, utilizing efficient inference, and grounded to linguistics and real-world scenarios.
In terms of specific topics, I am interested in
- NLP: Semantics; Text Generation; Linguistic Structure Prediction
- ML: Generalization; Discrete Latent Structures; Deep Generative Models
Many topics that I’m interested in are covered by the following reading list:
- Deep Generative Models for Natural Language Processing. (github)
- Compositional Generalization in Natural Language Processing. (github)
Publications
- [NAACL 2021] Noisy Labeled NER with Confidence Estimation. [paper][code]
- Kun Liu*, Yao Fu*, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang and Sheng Gao. *Equal contribution.
- A confidence estimation method for estimating label noise in NER annotations and a training method based on partial marginalization according to estimated noise.
- [ICLR 2021] Probing BERT in Hyperbolic Spaces. [paper][code]
- Boli Chen*, Yao Fu*, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, Liping Jing. *Equal contribution.
- A Poincare probe for recovering hierarchical structures from contextualized representations. Applied to probing syntax and sentiment in BERT.
- [ICLR 2021] Prototypical Representation Learning for Relation Extraction. [paper][code]
- Ning Ding, Xiaobin Wang, Yao Fu, Guangwei Xu, Rui Wang, Pengjun Xie, Ying Shen, Fei Huang, Hai-Tao Zheng, Rui Zhang
- A representation learning method for embedding relation prototypes on hyperspheres. Applied to supervised, semi-supervised, and few-shot relational learning.
- [AAAI 2021] Nested Named Entity Recognition with Partially Observed TreeCRFs. [paper][code]
- Yao Fu*, Chuanqi Tan*, Mosha Chen, Songfang Huang, Fei Huang. *Equal contribution.
- A Masked Inside algorithm for efficient partial marginalization of TreeCRFs. Applied to Nested NER.
- [NeurIPS 2020] Latent Template Induction with Gumbel-CRFs. [paper][code]
- Yao Fu, Chuanqi Tan, Mosha Chen, Bin Bi, Yansong Feng and Alexander Rush.
- A Gumbel-FFBS algorithm for reparameterizing and relaxing CRFs. Applied to controllable text generation with latent templates.
- [NeurIPS 2019] Paraphrase Generation with Latent Bag of Words. [paper][code]
- Yao Fu, Yansong Feng and John Cunningham.
- A differentiable planning and realization model with latent bag of words by Gumbel-topK reparameterization. Applied to paraphrase generation.
- [INLG 2019] Rethinking Text Attribute Transfer: A Lexical Analysis. [paper][code]
- Yao Fu, Hao Zhou, Jiaze Chen and Lei Li.
- A series of text mining algorithms for discovering words with strong influence on classification. Applied to analysing text attribute transfer models.
- [NAACL 2018] Natural Answer Generation with Heterogeneous Memory. [paper]
- Yao Fu and Yansong Feng.
- An attention mechanism fusing information from different source of knowledge. Applied to answer sentence generation.
Teaching
- PKU EECS. Empirical Methods for Natural Language Processing. 2021 Spring. Guest lecture on Text Generation. Tought by Prof. Yansong Feng.
- Alibaba Advanced Probabilistic Machine Learning Seminar. 2020 Spring. Instructor.
- Columbia COMS 4995 Applied Machine Learning, 19 Spring, Course Assistant. Tought by Prof. Andreas Muller.
Resources and Tutorials
- Deep Structured Prediction: Inference, Reparameterization and Applications. Talk at Bytedance. Jun 2021 [pdf]
- How to write Variational Inference and Generative Models for NLP: a recipe. Talk at Edinburgh. Mar 2021 [pdf]
Internships
- Jan 20 - Oct 20. Alibaba Damo Academy. Natural Language Processing Research Intern. Beijing and Hangzhou
- May 19 - Aug 19. Tencent AI Lab. Natural Language Processing Research Intern. Seattle
- Dec 17 - Aug 18. Bytedance AI Lab. Natural Language Processing Research Intern. Beijing