The story of Xiao Ao Jiang Hu is about a wanderer named Ling Hu Chong. As a child, he was an orphan and raised by Yue Bu Qun (nicknamed the "Gentalman Sword-Jun Zi Jian") and his wife (Ning Zhong Ze) in the Hua Shan Sect as the eldest apprentice brother (Da Shi Xiong). On his first trip down Hua Shan Mountain, he met Qu Yang of the Sun Moon Church (Ri Yue Shen Jiao), or the Evil Cult (Mo Jiao)" and, against his master's teachings, befriended him. He was told to spent a year on top of Si Guo Ya think about his wrongs. Meanwhile, his childhood friend and lover, his little apprentice sister and his master's daughter, Yue Ling Shan, starts to draw away from him and becomes really close to the new Hua Shan apprentice, Lin Ping Zhi.
He later sustained serious injuries from the Peach Valley Six Faries (Tao Gu Liu Xian). Just at that time, the Hua Shan Sect was forced to leave, going to Luo Yang. While he was there, he met a "Granny" (Po-Po), who never let him see her face, and she provided him confort in times when he was sad. He told her about a lot of things, about all his sorrows, and she taught him how to play the zither (Qin-an instrument played in ancient China).
Ling Hu Chong soon had to leave Luo Yang with the Hua Shan Sect. Suddenly though, people whom his master disapproved of started befriending him, claiming they knew a good friend of his. At Wu Ba Gang, while he was alone, he saw the Granny again. After saving her, he decides to go on travels with her. Soon, he finds out that she's not a Granny, but a beautiful girl by the name of Ren Ying Ying, the much known and feared Holy Madien (Sheng Gu) of the Evil Cult.
As being on different sides, they were not allowed to be friends, but they traveled together and soon, Ling Hu Chong found himself deeply in love with her. After many hardships, they finally get married and go off where they are alone to be happy.