Parsing POP email messages
Posted by Jack Altiere on March 29th, 2010I was recently tasked with coming up with a way to parse a pop email account and automatically save off the message and any attachments that came in the email. While it didn’t sound like too bad of a task, I know from my limited experience that parsing MIME content can be a pain in the rear. I thought it would be interesting to share what I came up with in a sample application.
First, I knew that I had no desire to reinvent the wheel and write my own MIME parser if I could find one that works. After some searching and some trial and error I ended up finding a .NET library that seems to work great from here. The full version of this library isn’t free, but the trial version is fully functional, it just rewrites the email subjects once in a while to tell you to buy the full version. (as a side note, I will probably end up buying this product, it worked exactly like I wanted it to)
Since I felt like showing this would go better if I had a sample app, I decided to write a little one page website that takes emails from an approved list of addresses, and posts any photo attachments in a little photo album. I decided I wanted a SQL Server backend, and that I would just use LINQ to SQL for data access. The plan is to just take the body of the email and make it the image caption and attach 1 image per email. It would also be nice to resize the image if it’s too big.
First, the backend. For the sake of simplicity I just created 2 tables…one to handle user validation and one to store the photos. After creating the LINQ to SQL classes for these two tables, my .dbml design view looked like this:
After that, I wanted to set up my mail parsing library. Like I mentioned above, I just wrapped a 3rd party utility, so my code for this isn’t too complex.
public class MailParser
{
private readonly string _popServer;
private readonly string _username;
private readonly string _password;
private readonly IMessageProcessor _messageProcessor;
public MailParser(IMessageProcessor messageProcessor)
{
_popServer = ConfigurationParser.GetAppSettingString("MailServer");
_username = ConfigurationParser.GetAppSettingString("MailUserName");
_password = ConfigurationParser.GetAppSettingString("MailPassword");
_messageProcessor = messageProcessor;
}
public void ProcessMessages()
{
var messages = new List<IMail>();
using (var pop3 = new Pop3())
{
pop3.Connect(_popServer);
pop3.Login(_username, _password);
foreach (var uid in pop3.GetAll())
{
// Add the email message to the list to be processed.
var message = new MailBuilder().CreateFromEml(pop3.GetMessageByUID(uid));
messages.Add(message);
// Delete the message from the server.
pop3.DeleteMessageByUID(uid);
}
pop3.Close(true);
// We have snagged all of the messages from the server, now process them.
foreach (var message in messages)
{
_messageProcessor.ProcessMessage(message);
}
}
}
}
public interface IMessageProcessor
{
void ProcessMessage(IMail message);
}
public class MessageProcessor : IMessageProcessor
{
private readonly PhotoDBDataContext _db = new PhotoDBDataContext();
public void ProcessMessage(IMail message)
{
var fromAddresses = message.From;
var body = message.HtmlData ?? message.TextData;
if (fromAddresses.Count == 0)
return;
// I only want to process emails from a list of valid email addresses.
var address = (fromAddresses[0].Address.IsNullOrEmpty()) ? string.Empty : fromAddresses[0].Address;
var user = ValidateUser(address);
if (user == null)
return;
// If there are no attachments ignore the message.
if (message.Attachments.Count == 0)
return;
// Process the attachments. (only supporting 1 attachment per email)
var counter = 0;
foreach (var att in message.Attachments)
{
if (counter > 0)
break;
// Make sure that the attachment is a valid image.
if (!att.ContentType.MimeType.ToString().ToLower().Equals("image"))
return;
var photo = new Photo
{
Caption = body.Text,
DateUploaded = DateTime.Now,
UserID = user.UserID,
ImageData = att.Data,
ContentType = att.ContentType.ToString(),
FileName = att.FileName
};
_db.Photos.InsertOnSubmit(photo);
_db.SubmitChanges();
counter ++;
}
}
private User ValidateUser(string email)
{
var query = from u in _db.Users
where u.EmailAddress.Equals(email)
select u;
return query.FirstOrDefault();
}
}
The code should be fairly straight forward. I created a message processing interface, because it is conceivable that I would want to reuse this and processes messages differently. You may notice the ConfigurationParser references, that is just a class I wrote to wrap the access of configuration variables allowing me to load them strongly typed. All I am doing is looping over the messages on the POP server, loading them into a list to process, and deleting the original message from the server. I’ll throw out the standard disclaimer here and say that this is an example application! I would definitely consider persisting messages into some sort of a queue to process before deleting them from the mail server if this were a production application.
Using LINQ to SQL it’s a snap to validate the user and make sure we’re only posting pictures from trusted email addresses. (I know, we would have to think about how to protect this against spoofing if this were a real app) Notice how I’m also using the MIME type to make sure that we only accept image attachments. Lastly, I’m only processing the first attachment to make sure to only accept one image per email, I did this because I’m using the message body as the caption of the photo.
Now that images are being scraped from the email account and stored in the database, I thought I’d close the loop on this sample application and put a quick web page up to display the album. I created a new ASP.NET MVC 2 web application, and then went through the same steps as above to create my LINQ to SQL classes. The first thing I wanted to do was create a strongly typed view. The way I do this is to right-click in the HomeController.cs file and select Add View. You will get a dialog like this one:
All I did was name the view, select my LINQ to SQL generated Photo class, and choose List as the View content. This basically says that the view will have access to a collection of photos. It’s trivial to pull the list of photos out of the database with LINQ:
public ActionResult Index()
{
var db = new PhotoDBDataContext();
var photos = (from p in db.Photos
select p).ToList();
ViewData.Model = photos;
return View();
}
I wanted to do is to display an image straight from an MVC controller. I’m not going to post all the code I used to accomplish this, but the short version: I modified a guide I found here. One requirement that I still haven’t addressed is that I never resized the images. Since I took the images straight from the POP account and put them in the database, I have no idea how big they really are. Rather than go way off track and figure out how to do this, I used a method I read about here to resize my images from a byte array if needed. Now I wanted to come up with a clever way to be able to thumbnail images on the fly, so I changed the default route in the global.asax file to look like this:
public static void RegisterRoutes(RouteCollection routes)
{
routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
routes.MapRoute(
"Default", // Route name
"{controller}/{action}/{photoID}/{thumbnail}", // URL with parameters
new { controller = "Home", action = "Index", photoID = UrlParameter.Optional, thumbnail = false } // Parameter defaults
);
}
What this allows me to do is use a URL like mysite.com/Home/Images/1 to show the full image and mysite.com/Home/Images/1/true if I want to show the thumbnail of the same image. My controller method for the image creation looks like this:
public ActionResult Images(int photoID, bool thumbnail)
{
var db = new PhotoDBDataContext();
var photo = (from p in db.Photos
where p.PhotoID.Equals(photoID)
select p).SingleOrDefault();
if (photo == null)
throw new Exception("Photo not loaded");
var bytes = (thumbnail)
? ResizeFromStream(100, new MemoryStream(photo.ImageData.ToArray()), photo.FileName)
: photo.ImageData.ToArray();
var image = bytes;
var contentType = photo.ContentType;
return this.Image(image, contentType);
}
The view ended up being trivial:
<%@ Page Title="" Language="C#" MasterPageFile="~/Views/Shared/Site.Master" Inherits="System.Web.Mvc.ViewPage<IEnumerable<BlogSamples.DisplayPhotos.Photo>>" %>
<asp:Content ID="Content1" ContentPlaceHolderID="TitleContent" runat="server">
Photo Album - Jaltiere.Com
</asp:Content>
<asp:Content ID="Content2" ContentPlaceHolderID="MainContent" runat="server">
<div id="container">
<ul>
<% foreach (var item in Model) { %>
<li>
<a href="Home/Images/<%= Html.Encode(item.PhotoID) %>" target="_blank" style="background-image: url('Home/Images/<%= Html.Encode(item.PhotoID) %>/true')">
<span><%= Html.Encode(item.Caption) %><br /><i>uploaded <%= Html.Encode(item.DateUploaded.ToShortDateString()) %></i></span></a>
</li>
<% } %>
</ul>
</div>
</asp:Content>
And with a little CSS magic…
body {
text-align:center;
font-family: tahoma, arial, sans-serif;
font-size: 8pt;
}
#container {
position:relative;
width:770px;
height:396px;
margin:20px auto 0 auto;
border:1px solid #aaa;
}
ul {
padding:0;
margin:0;
list-style-type:none;
}
li {
display: inline;
float: left;
width: 101px;
height: 101px;
margin: 4px;
}
li a {
display: block;
width: 101px;
height: 101px;
background-position: center;
background-repeat: no-repeat;
text-decoration: none;
}
li { height: 115px; }
li a span {
font-size: 9px;
position: relative;
top: 103px;
color: #666;
display: block;
text-align: center;
}
li a:hover span { color: red; }
You get a functional photo album:
This article kind of took off on me and turned out to be more MVC than POP mail parsing, but it was a fun little exercise. Things to take away:
- I recommend the mail parsing library I used, I found it to be intuitive and easy to use.
- LINQ to SQL makes data access easy. (coming from a guy with a big stored procedure background)
- MVC is much more fun to work with than Webforms in my opinion.
- I freely admit that I am definitely not a designer, so my web layout is not very good.
Recent Comments