Unfortunately, you can’t. You can only delete the whole conversation by unmatching the user, but watch out - this is a permanent and irreversible action!