Rotator cuff tear is one of the most common causes of shoulder disability in middle-aged and elderly people. Although many patients with rotator cuff tear have pain and dysfunction, there is a great variability in the clinical manifestations. The management of rotator cuff tear includes non-operative and operative methods. The purpose of this study was to evaluate the therapeutic effect of rehabilitation and to compare the rehabilitation effects between patients receiving operation or not. Twenty-two patients were enrolled in this study (13 operative and 9 non-operative). Each subject received rehabilitation therapy (including heat packing, TENS, massage, and therapeutic exercise) twice a week, and the therapeutic duration was 8 weeks. We evaluated the therapeutic effect every two weeks, including shoulder range of motion (ROM), muscle power and pain analogue scale. Statistical evaluation was performed using the Statistical Package for Social Science (SPSS). The results showed that rehabilitation therapy could improve ROM and relief of pain in both groups, but not muscle power. There was no significant rehabilitative effect difference between the operative and non-operative groups. No significant relationship was found between the severity of rotator cuff tear and the therapeutic effects. For middle-aged and elderly patients with rotator cuff tear, rehabilitation has positive therapeutic effects in both operative and non-operative patients with any size of rotator cuff tear. Therefore, rehabilitation therapy in the treatment of patients with rotator cuff tear cannot be overemphasized.