I rely on Python's built-in unittest module. While this is most seamless for Python code, with some customization you can make it manage anything (e.g. running external test programs and writing assertions on the results).