I’m currently a second year PhD student at UCLA, advised by Prof. Kai-Wei Chang.

My research interest lies in the intersection of Computer Vision (CV) and Natural Language Processing (NLP), aiming to equip computers with the ability to understand and relate data across different modalities. Specifically, I am interested in the following topcis:

• Representation Learning for Decision Making: Vision-and-Language Navigation (VLN), Robotic Manipulation, Generative Models for planning
• Learning reasoning via Interaction: Learning VL representations via embodied interactions, Learning relation-aware VL representations from video
• Compositionality skills for multimodal generation and reasoning: Open-World Image/Video Captioning, Language-Conditioned Image Manipulation

I’m looking for Research Intern opportunities for Summer 2024.

Publications

2024

Re-ReST: Reflection-Reinforced Self-Training for Language Agents

Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, Nanyun Peng EMNLP 2024

LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning

Silin Meng, Yiwei Wang,Cheng-Fu Yang, Nanyun Peng and Kai-Wei Chang EMNLP 2024

Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty

Cheng-Fu Yang, Haoyang Xu, Te-Lin Wu, Xiaofeng Gao, Kai-Wei Chang, Feng Gao NeurIPS OWA Workshop

2023

LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following

Cheng-Fu Yang, Yen-Chun Chen, Jianwei Yang, Xiyang Dai, Lu Yuan, Yu-Chiang Frank Wang, Kai-Wei Chang Accepted to EMNLP 2023

2022

Paraphrasing is all you need for Novel Object Captioning

Cheng-Fu Yang, Yao-Hung Hubert Tsai, Wan-Cyuan Fan, Yu-Chiang Frank Wang, Louis-Philippe Morency, Ruslan Salakhutdinov Accepted to NeurIPS 2022

Target-free Text-guided Image Manipulation

Wan-Cyuan Fan, Cheng-Fu Yang, Qiao-An Yang, Yu-Chiang Frank Wang Accepted to AAAI 2023

Scene Graph Expansion for Semantics-Guided Image Completion

Qiao-An Yang, Cheng-Fu Yang, Wan-Cyuan Fan, Cheng-Yo Tan, Meng-Lin Wu, Yu-Chiang Frank Wang Accepted to CVPR 2022

Cross-Modal Mutual Learning for Audio-Visual Speech Recognition and Manipulation

Chih-Chun Yang, Cheng-Fu Yang, Wan-Cyuan Fan and Yu-Chiang Frank Wang Accepted to AAAI 2022

2021

LayoutTransformer: Scene Layout Generation with Conceptual and Spatial Diversity

Cheng-Fu Yang*, Wan-Cyuan Fan*, Fu-En Yang and Yu-Chiang Frank Wang. Accepted to CVPR 2021