We explicitly demonstrate that the nonlocal ghost-free ultraviolet modification of general relativity (GR) known as the infinite derivative gravity (IDG) resolves nonscalar curvature singularities in exact solutions of the full theory. Exact pp-wave solutions of GR and IDG describing gravitational waves generated by null sources are analyzed. Curvature of GR and IDG solutions with the same energy-momentum tensor is compared in parallel-propagated frames along timelike and null geodesics. While the GR pp-wave solution contains a physically problematic nonscalar curvature singularity at the location of the source, the curvature of its IDG counterpart is finite.