At the Australia Moot, Daymon Wiese, a Moodle HQ developer, gave a talk on how to automate testing in Moodle for Administrators.
Underlying the need for testing in Moodle is the idea that any change made by an administrator (everything from code changes to simply adding a plugin) ultimately has the potential to impact the functionality of the entire LMS. This impact grows exponentially the more changes you implement.
You can manage these changes through a change control process. This process should have at least three steps that include Testing, Staging and Production. Testing is often completed manually, which as well all know, if fraught with errors and variations. To get around this, you should/can implement machine testing.
Watch this important presentation in full below: