Given their nanoscale dimensions, colloidal semiconductor nanocrystals provide unique systems for investigating the dynamics controlling surface chemistry and fundamental issues regarding lattice reorganization upon changes in electron distribution. These systems are particularly amenable to ultrafast electron probes, offering an atomic level picture of the lattice reorganization involved following photoexcitation. Here, we study lead sulfide (PbS) quantum dots with ultrafast electron diffraction to characterize the atomic motions following high-intensity photoexcitation. Short-range nonthermal lattice distortions and increased atomic disorder were observed in PbS colloidal quantum dots ranging from 2.4 to 8.7 nm in size. These effects scaled inversely with size and were less pronounced in nanocrystals with a chloride-containing surface rather than only organic ligands, which is consistent with an effect arising at the surface. The anisotropic, nonthermal lattice disordering occurs preferentially along the (100) crystallographic directions, which could indicate an anisotropic distribution of localized charge between the differing lattice terminations of the {111} and {100} crystal facets. This is consistent with the larger anharmonicity for the lattice potential at lattice sites with reduced ligand coordination relative to the bulk, which has been shown to cause accelerated relaxation into dynamic and static surface trap sites. Through an exploration of quantum dot size and variation in surface termination, this work provides the missing structural details to advance our understanding and control of charge-carrier formation, trapping, and recombination processes in nanoscale semiconductor systems.