Heat flow between a large thermal 'bath' and a smaller system brings them progressively closer to thermal equilibrium while increasing their entropy1. Fluctuations involving a small fraction of a statistical ensemble of systems interacting with the bath result in deviations from this trend. In this respect, quantum and classical thermodynamics are in agreement1, 2, 3, 4, 5. Here we predict a different trend in a purely quantum mechanical setting: disturbances of thermal equilibrium between two-level systems (TLSs) and a bath6, caused by frequent, brief quantum non-demolition7, 8, 9, 10 measurements of the TLS energy states. By making the measurements increasingly frequent, we encounter first the anti-Zeno regime and then the Zeno regime (namely where the TLSs' relaxation respectively speeds up and slows down11, 12, 13, 14, 15). The corresponding entropy and temperature of both the system and the bath are then found to either decrease or increase depending only on the rate of observation, contrary to the standard thermodynamical rules that hold for memory-less (Markov) baths2, 5. From a practical viewpoint, these anomalies may offer the possibility of very fast control of heat and entropy in quantum systems, allowing cooling and state purification over an interval much shorter than the time needed for thermal equilibration or for a feedback control loop.