The Windows taskbar literally was just another window but without any decorations. There was even a bug in Windows 95 where you could close the taskbar.

As for the full screen bug you experienced, I don’t recall ever seeing that but Windows 9x had a plethora of weird bugs so it wouldn’t surprise me if you had issues.