The longtime frustration of not being able to clear the messages in tests is marked as a delivered idea in Summer '18 here Add support for clearing PageMessages via Apex.

A failing of that forum is that there never seems to be a link to what was actually done, leaving you back in Google land trying to figure out what that was. In this case I haven't found any new methods in the documentation, and can confirm that the second assert here fails using API 45:

 System.assert(ApexPages.getMessages().size() > 0);
 ApexPages.getMessages().clear();
 System.assert(ApexPages.getMessages().size() == 0);

So how are the messages cleared?

It was easy enough to find the Release Note for this feature by looking at Development > Apex.

Clear Messages on Visualforce Pages While Testing

Use the new System.Test.clearApexPageMessages() method to clear the messages on a Visualforce page while executing Apex test methods.

Where: This change applies to Lightning Experience and Salesforce Classic in Enterprise, Performance, Unlimited, and Developer editions.

Why: You can use the System.Test.clearApexPageMessages() method to test the success or failure of each call to controller methods. Using this function along with the ApexPages.hasMessages() and ApexPages.getMessages() methods allows you to test Visualforce controllers more easily.

This method is also included in the documentation on the Test Class:

clearApexPageMessages()

Clear the messages on a Visualforce page while executing Apex test methods.

Signature

public static void clearApexPageMessages()

Return Value

Type: void

Usage

This method may only be used in tests.

Example

@isTest
static void clearMessagesTest() 
 Test.setCurrentPage(new PageReference('/'));
 ApexPages.addMessage(
 new ApexPages.Message(ApexPages.Severity.WARNING, 'Sample Warning')
 );
 System.assertEquals(1, ApexPages.getMessages().size());
 Test.clearApexPageMessages();
 System.assertEquals(0, ApexPages.getMessages().size());

OK, with a different set of keywords I hit this Clear Messages on Visualforce Pages While Testing release note that talks about the Test.clearApexPageMessages() method. That is explained in the Test Class documentation.

And happy days, both these assertions pass:

 System.assert(ApexPages.getMessages().size() > 0);
 Test.clearApexPageMessages();
 System.assert(ApexPages.getMessages().size() == 0);