Run the following command:
apt-get remove exim4 exim4-base exim4-config exim4-daemon-light
All your configs stay on the server so that you can easily get mailing running again by installing it:
apt-get install exim4
Source
exim – How to cleanly remove Exim4 mail server on Ubuntu – Stack Overflow. http://stackoverflow.com/questions/12061358/how-to-cleanly-remove-exim4-mail-server-on-ubuntu