A hierarchical model of a polycrystalline aggregate of rigid viscoplastic grains is formulated, and a robust and efficient computational algorithm for its solution is proposed. The polycrystalline aggregate is modeled as a binary tree. The leaves of the binary tree represent grains, and higher tree nodes represent increasingly larger sub-aggregates of grains. The root of the tree represents the entire polycrystalline aggregate. Velocity and traction continuity are enforced across the interface between the children of each non-leaf node in the binary tree. The hierarchical model explicitly models intergranular interactions but is nevertheless comparable in computational effort to the mean field models of polycrystal plasticity. Simulations of tensile, compressive, torsional, and plane strain deformation of copper lead to predictions in good agreement with experiments, and highlight the interconnection between grain deformations and intergranular constraints. It is inferred from the results that a hybrid mean field/hierarchical model represents a computationally efficient methodology to simulate polycrystal deformation while accounting for intergranular interactions.