Delete em. You should be iterating your unit tests.
Over time you will find some really useful unit tests for common mistakes, and others that are basically useless. Try and approach things from the perspective of what you would like to know works and not just tossing everything and the kitchen sink at the problem. When you find a bug add a unit test just in case it comes back, if you change major parts of the system, port the most useful unit tests and delete the rest.
I think of unit tests as saving you from the (I thought I just fixed this) problem.
PS: The major advantage of this approach is you keep adding more tests to the older parts of the system, which are also the most costly to change because other parts of the system assume it works. And assumptions you and your code is making can be really hard bugs to find and fix.
I think the best benefit of unit tests is really saving you from the "Will this break anything?" problem. Uncertainty is the real big killer for programming productivity; if you know what to code, you just type it out and you can get like 10 lines/minute. So have some way that you can run things and say "I know this broke something" or "I know this didn't break anything."