There isn't yet enough research on health hazards of toner to really give a good answer here... Black toner for example is mostly polyester plastic and carbon dust. Neither of those are particularly toxic, although they are in a much finer powder than you'd normally encounter them.

Basically I'd still steer clear of them, but exposing yourself to them isn't a certain death sentence like say organic mercury.

I would say akin to smoking a cigarette. One time won't hurt you in a measurable way but if your job is refilling toners or dealing with broken laser printers you need some protection.