The daemon has had many security issues. They also only recently grew support for ACLs (which are only available as plugins), so any user that can write to the Docker socket is essentially equivalent to root.