In conventional metals, there is plenty of space for dislocations—line defects whose motion results in permanent material deformation—to multiply, so that the metal strengths are controlled by dislocation interactions with grain boundaries1, 2 and other obstacles3, 4. For nanostructured materials, in contrast, dislocation multiplication is severely confined by the nanometre-scale geometries so that continued plasticity can be expected to be source-controlled. Nano-grained polycrystalline materials were found to be strong but brittle5, 6, 7, 8, 9, because both nucleation and motion of dislocations are effectively suppressed by the nanoscale crystallites. Here we report a dislocation-nucleation-controlled mechanism in nano-twinned metals10, 11 in which there are plenty of dislocation nucleation sites but dislocation motion is not confined. We show that dislocation nucleation governs the strength of such materials, resulting in their softening below a critical twin thickness. Large-scale molecular dynamics simulations and a kinetic theory of dislocation nucleation in nano-twinned metals show that there exists a transition in deformation mechanism, occurring at a critical twin-boundary spacing for which strength is maximized. At this point, the classical Hall–Petch type of strengthening due to dislocation pile-up and cutting through twin planes switches to a dislocation-nucleation-controlled softening mechanism with twin-boundary migration resulting from nucleation and motion of partial dislocations parallel to the twin planes. Most previous studies12, 13 did not consider a sufficient range of twin thickness and therefore missed this strength-softening regime. The simulations indicate that the critical twin-boundary spacing for the onset of softening in nano-twinned copper and the maximum strength depend on the grain size: the smaller the grain size, the smaller the critical twin-boundary spacing, and the higher the maximum strength of the material.