Introduction: Accelerometry-based activity counting for measuring arm use is prone to overestimation due to nonfunctional movements. In this paper, we used an inertial measurement unit (IMU)-based gross movement (GM) score to quantify arm use. Methods: In this two-part study, we first characterized the GM by comparing it to annotated video recordings of 5 hemiparetic patients and 10 control subjects performing a set of activities. In the second part, we tracked the arm use of 5 patients and 5 controls using two wrist-worn IMUs for 7 and 3 days, respectively. The IMU data was used to develop quantitative measures (total and relative arm use) and a visualization method for arm use. Results: From the characterization study, we found that GM detects functional activities with 50–60% accuracy and eliminates non-functional activities with >90% accuracy. Continuous monitoring of arm use showed that the arm use was biased towards the dominant limb and less paretic limb for controls and patients, respectively. Conclusions: The gross movement score has good specificity but low sensitivity in identifying functional activity. The at-home study showed that it is feasible to use two IMU-watches to monitor relative arm use and provided design considerations for improving the assessment method. Clinical trial registry number: CTRI/2018/09/015648