Well, you could argue that the notice is "present" (in a very esoteric sense) in a binary distribution of the software because it was present in the source code used to build it. You could also argue that a compiled version of a program isn't a "copy or substantial portion" of the Software (compilation is effectively a form of translation, which is a derivative work under the Copyright Act in the US -- and not just a copy).
Personally I would still include it in both, but I always had the impression that MIT was looser than BSD-2-Clause about this. BSD-2-Clause explicitly states that binary distribution needs to include the notice in "the documentation and/or other materials provided with the distribution", and I have a feeling that the license authors might've had a reason to want to be explicit about it.
> The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
It's up to a court to decide what "copies or substantial portions of the Software" means.
Personally, I've always very much interpreted that to mean both binary and source.