This passcode is inherently limited to the service it bound to (IMAP or POP3), that's the whole point: don't expose your account password to something which only needs a finer-grained access.
You're directly contradicting your sibling comment. I guess I'll experiment with this in the coming days, although I'm a little worried tinkering too much will just completely lock me out of my account.
Edit: that's incorrect, see replies.