Acoustic emission is an important physical phenomenon when materials are deformed and fractured due to inner distortion or outer loading. It is also a direct parameter to show the mechanical property of rocks. Detecting or monitoring acoustic emission is significant in rock material failure theory and engineering. Because electromagnetism ultrasonic detection implements have some disadvantages of poor interference immunity, low sensitivity and inferior reliability, acoustic emission detection based on fiber Bragg grating (FBG) sensor is put forward to monitoring rocks. The strain and ultrasonic detection principles of FBG sensors are analyzed, at the same time, experimental system based on FBG ultrasonic sensor detection is also set up to simulate ultrasonic detection. The testing results of 500kHz acoustic shock signal and response are shown. A numerical simulation code RFPA is used to study rock acoustic emission. Mud and sand materials from river are used to process rock specimen. Rock specimen failure experiments are fulfilled via numerical control press device, which generate acoustic emission during the process of rock deformation and failure. Rock acoustic emission experimental signals' energy and counts are detected by FBG ultrasonic sensor detection system, which elementally proves FBG sensor can detect acoustic emission. Simulation and experiments of rock specimen give some useful results. All these show that FBG ultrasonic sensor can monitor rock distortion and failure effectively and is a novel technology and method in rock engineering.