Is there any way to restrict sending email through the SMTP service to only certain users?

My primary GWIA is behind a firewall and Spam appliance. I have setup a second GWIA that I would like to make visible to the internet for just IMAP users (for mobile devices and 3rd party clients like Thunderbird). I have set the default class of service to deny access to POP/IMAP and then setup a second class of service allowing these services to certain users. This works effectively at denying everyone from IMAP/POP except those explicitly allowed and they are required to login with a password.

However, this does not seem to work for SMTP. If I deny SMTP in the default class of service, I cannot override that in my other class of service. It effectively denies access to everyone.

If I allow it in the class of service, anyone can send an email through it. I even sent an email through it using an email return address in my client.

I also can not see any way to require authentication to send. Is this even possible?