National Taiwan University research found that AI has a phenomenon of flattery and is easily affected by users' false expectations, external suggestions and emotions.

When the user makes a mistake, will the AI ​​correct you or follow your words? Research by the Natural Language Processing Laboratory of National Taiwan University found that AI has a "sycophancy phenomenon", which means that the model tends to cater to the user's expectations or default position rather than giving objectively correct answers. The study also found that in addition to what the user said, where the external suggestions came from, and the emotional expression in the video may all influence the AI's answer. The team analyzed the flattery mechanism from three levels: content, source and modality, and proposed corresponding mitigation methods.

AI's "too agreeable" nature may not be easy to detect in ordinary chats, but it may cause risks when it enters fields such as medical care, finance, and law that require professional fact-checking. For example, National Taiwan University provides an example. If a patient asks, "Should it be okay if I cut the insulin dose in half?", if the AI ​​provides support to cater to the user, it may directly threaten life safety.

Please read on...

The study first analyzed "content" and found that the preference alignment technology commonly used in training models will actually amplify flattery behavior. The team proposes to add self-built SAA training data or use a self-augmented (Self-Augmented) data generation method so that when users make wrong suggestions, the model can still insist on giving correct answers based on facts, successfully reducing the potential harm rate of the model.

"External advice" can also sway AI. The team extended the research to multiple rounds of clinical consultations. When the patient told the AI ​​agent doctor, "I asked Gemini and it said it might be asthma...", the experiment found that the AI ​​was easily influenced by external suggestions, thereby changing the originally correct diagnosis; but if the external suggestions were placed at the end of the conversation, the impact was significantly lower. Therefore, the team developed a "second hypothesis re-evaluation mechanism" that can stably slow down flattery behavior under different role settings without retraining the model.

Research has also pushed the question from text to video. The team built a ViSyc video data set and combined voice replication and lip synchronization technology to change only the emotional expression when the sentences are exactly the same. The results show that emotional stimuli such as anger, disgust, and happiness will cause the multimodal model's answers to deviate from neutrality, forming "video-induced emotional flattery."

The team stated that in the future, they will continue to study cultural biases and mechanism explainability in real-world video interactions and flattery behaviors, hoping to further understand why AI is affected by the user's stance, external suggestions, and emotions, and move towards a safe, objective, and trustworthy artificial intelligence system.