Yuanjun Chai

Yuanjun Chai

VMware

Hi! Here is Yuanjun Chai 柴源君 (sounds like Y-wen Joon, Ch-eye), aka Allen. I work at VMware AI Lab , working on LLM and Multi-LLM.

I graduated with highest honors from Xidian University, earning a bachelor’s degree. My thesis about inpainting received invaluable support by Chao Dong from SIAT-CAS . Also, I am so lucky to have the privilege of collaborating with Chao Dong and Yu Qiao from CAS working on image&video super-resolution, Jason Cheung from HKU working on AI healthcare, Yue Gao and Eric Yi from Tsinghua University working on Embodied AI.

Not only diving into research, I am also willing to empower new technologies into products to make people’s lives better. Thus, I co-founded a start-up INGREM inc, to help high-paraplegia disabled people using computer with precise eyes controling platform. Previously, I worked in YeahMobi – affiliation of Alibaba Group , as a senior machine learning scientist (P8 expert). I was responsible for the all technical development of AIGC platform Kreado AI , including AI Video Creation, AI model, custom clone service, and etc. Then, I am currently a machine learning engineer at VMware AI Lab , working on LLM agent, RAG-based LLM (Retrieval Augmented Generation) and Multi-modal LLM.

Research Interests:

  • Generative AI: LLM, RAG-based LLM, Multi-modal LLM, text-to-image, text-to-video
  • Virtual Avatar: 2D&3D talking face generation, voice generation, custom character
  • AI in healthcare: medical image, medical LLM
  • Computer Vision: low-level vision, high-level vision
  • Embodied AI: sim2real, multi-model perception

Above things are my research interests. If you are interested in any of these topics and would like to academic collaborations, please feel free to contact me at yuanjunchai89@gmail.com.

By the way, I am so glad to collaborate with young researchers. Although pursuing academics is a tough journey, but we could overcome it together with responsibility, active feedback, and all great supervisors&collaborators. Now I am holding multiple A100 GPUs. If you are interested in my research and applications, please directly contact me.

🔥 News

  • 2023.07:  🎉🎉 Thrilled to join VMware as MLE! We do some interesting projects on RAG-based LLM like h2oGPT (⭐️8k+).
  • 2023.04:  🔥🔥 Our AIGC platform Kreado AI has released to all over the world！
  • 2022.04:  👏👏 Glad to obtain fully-funded PhD offer from University of HongKong (HKU)!
  • 2021.09:  🥰🥰 Our eyes control platform has helped high-paraplegia disabled people more than 300!
  • 2021.08:  🎉🎉 Our IKC – CVPR project about real-world super-resolution get more than ⭐️200+.

📝 Publications

👁️ Computer Vision

CVPR
sym

IKC: Blind Super-Resolution With Iterative Kernel Correction

Jinjin Gu, Hannan Lu, Wangmeng Zuo, Chao Dong

Project |

  • We propose an Iterative Kernel Correction (IKC) method for blur kernel estimation in blind SR problem, where the blur kernels are unknown. We draw the observation that kernel mismatch could bring regular artifacts (either over-sharpening or over-smoothing), which can be applied to correct inaccurate blur kernels. Thus we introduce an iterative correction scheme – IKC that achieves better results than direct kernel estimation. We further propose an effective SR network architecture using spatial feature transform (SFT) layers to handle multiple blur kernels, named SFTMD.

🖥️ Innovational Products

🧑‍🎨 AIGC (Generative Model)

sym

Kreado AI: AIGC Platform for Marketing Content Generation

Yuanjun Chai and YeahMobi.inc

  • We propose a new AIGC platform for marketing content generation, named Kreado AI. Kreado AI is a hybrid worldwide AIGC platform that combines the strengths of so many AIGC functions:
    • Virtual Avatar (talking-face generation, speech synthesis, LLM)
    • AI model (text-to-image, LoRA, control net)
    • Custom clone serivces (image-to-video, voice clone)
    • Many AI tools and AI property
  • Here I mainly focus on the Virtual Avatar and AI model algorithms improvement, as well as collaborate with system architect for entire architecture improvement. Users radiate to Europe, Africa, Southeast Asia, and the Americas, with quarterly revenue exceeding US$1 million.

🧙‍♂️ RAG-based LLM

sym

h2o GPT: AIGC Platform for Marketing Content Generation

Yuanjun Chai

Project |

  • We develop a new RAG-based LLM platform for AI cloud native and private AI. The platform could leverage diverse LLMs with extended dataset such as pdf, code base, dataset and internet links. Here I am responsible for all RAG-based LLM algorithm development, as well as industrial deployment. Functionality includes:
    • QA and chat with extended dataset extraction
    • Content summarization
    • Content generation
    • AI agent
  • Next step we would take research about Multi-modal LLM for AI cloud native.

👨‍⚕️ AI healthcare & Charity

sym

Face Control: Fine Facial Control Platform for High-paraplegia Disabled People

Yuanjun Chai, Ingrem.inc

  • I co-founded a start-up Ingrem, with other hardcore guys. We aim to build up a entire bed for living and playing of high-paraplegia disabled people. Here, I am responsible for the development of the software – eyes&facial control platform. Based on computer vision algorithms, the system could help the diabled use their face details (such as eyebrow, eye, mouth, etc.) to control mouse and keyborad elaborately. Thus, our platform and our bed entirely enhance the accessibility of normal computer usage and social networks. We do believe tech make people’s lives better, and we do it!

🎖 Honors and Awards

  • 2022.04 Obtain a fully-funded PhD return offer from Li Ka Shing Faculty of Medicine, University of Hong Kong.
  • 2019.06 Outstanding Undergraduate Student Award.
  • 2019.06 Outstanding Undergraduate Thesis Award (10/5000), Topic: Image Inpainting Based on Deep Learning.
  • 2018.08 Cambridge Summer AI Academic Programme Excellent Student – All A+ of Artificial Intelligence Classes.
  • 2017.05 Golden Medal of National Computer Design Contest – Birdsong Recognition with Machine Learning.

🎓 Educations

  • 2015.08 - 2019.06, Undergraduate, Xidian University.
  • 2018.08 - 2018.09, Summer School, University of Cambridge (with Prof. Pietro Lio).
  • 2012.08 - 2015.06, High School Affiliated to Northwestern University

🧑‍💻 Professional Experience

  • 2023.07 - now, Machine Learning Engineer, VMware AI Lab
  • 2022.07 - 2023.06, Senior Machine Learning Scientist, YeahMobi – Alibaba Group.
  • 2019.05 - 2022.07, Research Assistant in CAS, Tsinghua University and HKU (get return PhD offer).

🏃‍♂️ Hobbies

My hobbies include Fencing🤺, Basketball🏀, Swimming🏊, Guitar🎸 and Motorcycle🏍️. In the high school, I get my first gold medal in Fencing🤺 at the National Province Games🏅.

Besides, I have my lovely cat🐱: sym02 sym01 sym03