Software Testing on an Astronomical Scale

Track: Testing

Type: Talk

Room: Talk Room 2

Time: Oct 14 (Fri): 10:15

Duration: 0:45

In this talk, to give context to the audience, I will start by briefly introducing the MeerKAT radio telescope, and describing the functionality and purpose of its correlator, which makes up the majority of the scientific signal-processing power of the telescope.

I'll then describe how we have utilised Pytest and a few related libraries in the development of the next-generation GPU-based software correlator, and I'll contrast this with how testing and verification were done in previous generations of hardware-based correlators (where unit-testing is not impossible but it is considerably more difficult). By being able to do regression testing, we have already mitigated several instances of fixing one bug while breaking something else.

I'll show how we have used the same technology to facilitate automatic end-to-end testing of the correlator against its spec (something that system engineers would call qualification), including automatic report generation (greatly reducing the amount of typing that we have to do!). I'll describe some tricks and optimisations that we have used in order to speed up the process without compromising the validity of the tests.

Finally, I'll spend some time talking about the caveats of unit testing, such as potentially allowing for false confidence, and how they don't catch every problem before it gets to "production", but I'll also discuss some ways to address this, and why they shouldn't deter you from testing as thoroughly as you can anyway.

URLs