E-gineering Golf Outing 2011
E-gineering's annual golf invitational turned out wonderful this year! It took place at Golf Club of Indiana. Not only were we blessed with incredible weather, but also with incredible fun and fellowship with our clients, partners, and employees.
We had an eighteen hole shotgun start scramble format tournament. Not only were there prizes for the tournament, but we also had several side games where participants could win extra goodies. If all of that failed, there were several more door prizes given away in a random drawing at the end! (Thank you to our prize sponsors, the Indiana Pacers, Worksmart Systems, and Complete Office Supply!) It was not all about prizes, however, as we had the most fun building new friendships, maintaining old ones, and generally enjoying a day together.
Registration.
Warm-up on the driving range.
Nick thanking everyone and getting everyone ready for tee-off.
Ready for tee-off.
Smashing the ball!
Sharing the great day.
One of the side games!
Looking for a lost ball.
One of the mascots.
Thanks again to all our clients, partners, and employees. We wouldn't be able to be who we are without all of you.
Posted at 10:51PM Sep 22, 2011 by Christian in General | Comments[0]
Running Multiple Versions of IE on Win 7 with Virtual PC
My coworker Chad sent out a link to this article on the SitePoint blog a long time ago (probably about a year and a half ago) and I've been meaning to try it since that day, but just haven't gotten around to it. Recently I started a new project for a client that is still supporting IE7 so I figured I better try to have an easy way to verify things in IE7. The article is really well written and is very easy to follow. I'm only writing about it is to say how cool it is (or sad, depending on how you look at it) to be able to get around the restriction of only being able to run one version of IE on each Windows instance and to point out the one hiccup I ran into and how to fix it.
I got through all the steps to get the XP Mode VM created and running, but when I went to adjust the Terminal Services "Limit Maximum Color Depth" policy on the guest XP OS (Windows Virtual PC uses Remote Desktop to display the VM desktop and 16 bit color is the default depth restriction) the policy wasn't there for me to edit. If it's not there, you have to add the system policy template back in to the Administrative Templates folder in the Group Policy editor (Start --> Run : gpedit.msc). I got this info from a forum post. Right-click the Administrative Templates folder, select Add/Remove Templates and add the system.adm template. The Terminal Services "Limit Maximum Color Depth" policy will now be there to edit as described in the article. I'm not sure if this has to be done any time a fresh XP VM is created, or if I just had some Windows Update fluke when updating the guest XP OS that somehow eliminated the policy template. Anyway, that's how to remedy the situation to fix the color depth issue if the policy is missing.
After following the rest of the steps, I have a shortcut to launch IE7 running on XP almost as if it were a native application running directly in my main Windows 7 OS. If the VM is running, it launches just as fast as if it were a native app. If the VM has hibernated, it takes a few seconds, but is not a problem (launches about as fast as FF4 does). The only things that give it away as a non-native application is that it is in an XP-style window and since it is a remote desktop, the window does not support all the Aero theme transparency and peek preview type stuff. I'll probably need to create another for IE8 eventually since IE9 is out...
Posted at 12:48AM May 27, 2011 by Christian in General | Comments[0]
JMeter JUG
I presented an introduction to JMeter at our local Java User's Group here in Indianapolis, known as IndyJUG. For those who attended the meeting, I promised to make the presentation slides and JMeter example test plan from the demo available here on my blog. Here are the presentation slides in (OpenOffice) Open Document Presentation format. Here are the slides in PowerPoint format. If none of those work for you, then here they are in PDF format. Also, here is the JMeter test plan we put together in the demo.
I cited a few sources as well:
For those who didn't attend and who are located in the greater Indianapolis area, let me take this opportunity to give a shout-out the Indianapolis Java Users Group, IndyJUG. To find out more about the organization, meetings, etc. please visit their site at http://www.indyjug.net/index.shtml.
Posted at 10:26PM Feb 23, 2011 by Christian in Java | Comments[0]
AutoItX3 and QTP Part 2
First, let's take a look at how I solved my problem of automating keystrokes from the number pad. This will be genericized so that you can run it inside QTP or even just save it in a .vbs file and execute it. It launches Notepad, waits for the notepad window to exist, makes sure it is activated, and then sends the keystrokes to generate the copyright symbol ©.
Set oAutoIt = CreateObject("AutoItX3.Control")
oAutoIt.Run("notepad.exe")
oAutoIt.WinWait "Untitled - Notepad"
oAutoIt.WinActivate "Untitled - Notepad"
oAutoIt.Send "{ALTDOWN}{NUMPAD0}{NUMPAD1}{NUMPAD6}{NUMPAD9}{ALTUP}"
Set oAutoIt = Nothing
Through messing around with AutoIt while trying to solve the problem of simulating typing on the number pad, I found that it is a great compliment to QTP's functionality. The mouse simulations are great, the window management methods are wonderful (wait for a particular window's existence, etc.), the registry management methods are outstanding, it has methods for setting and retrieving data from the system's clipboard, and the list goes on and on! So, I really wanted to use AutoIt for more than just that one test case. That just presents the problem of knowing whether or not the system running the tests in the future would have the AutoItX3 COM object available.
To work around that fact, I figured out I could uploaded the AutoItX3.dll file to the Quality Center project's "Subject" folder and create a quick script that can be run at the beginning of a test set in a stand-alone script or as part of any test script that requires AutoIt's functionality. The code downloads and sets up the AutoIt COM object automatically on the machine running the tests. This script could most likely be refined, but I just haven't really had the time or need to at this point. (Click here to view full-screen)
Dim regClr
Set regClr = DotNetFactory.CreateInstance("Microsoft.Win32.Registry", "mscorlib").ClassesRoot
If IsEmpty(regClr.OpenSubKey("AutoItX3.Control")) Then
Dim sbjNode, attachmentFilter, attachmentList, attachment
Dim fso, autoItX3File
'This is the name and destination path for the AutoItX3.dll file
Dim autoItX3DllName, autoItX3DllPath
autoItX3DllName = "AutoItX3.dll"
autoItX3DllPath = "C:WINDOWSsystem32" & autoItX3DllName
'Isolate the file in QC and download it
Set sbjNode = QCUtil.TDConnection.TreeManager.TreeRoot("Subject")
Set attachmentFilter = sbjNode.Attachments.Filter
attachmentFilter.Filter("CR_REFERENCE") = "'ALL_LISTS_" & sbjNode.NodeID & "_" & autoItX3DllName & "'"
Set attachmentList = attachmentFilter.NewList
Set attachment = attachmentList.Item(1)
attachment.Load True, ""
'Move it to the destination path
Set fso = CreateObject("Scripting.FileSystemObject")
Set autoItX3Dll = fso.GetFile(attachment.FileName)
autoItX3Dll.Copy(autoItX3DllPath)
'Register it
Set WshShell = CreateObject("WScript.Shell")
WshShell.Exec "RegSvr32 /s " & autoItX3DllPath
'Release objects
Set sbjNode = Nothing
Set attachmentFilter = Nothing
Set attachmentList = Nothing
Set attachment = Nothing
Set autoItX3Dll = Nothing
Set fso = Nothing
Set WshShell = Nothing
End If
Set regClr = Nothing
Of course, this code could be used to load any COM object. This could be really handy if you develop your own custom dll's for your tests. Other enhancements could be to make paths more dynamic, but I'm confident that this would work on any computer in my current client's organization.
Posted at 04:25PM May 28, 2010 by Christian in General | Comments[0]
AutoItX3 and QTP Part 1
As usual, after I got into writing this, it got too long to be a single post. Part 1 covers the background for why I began to use AutoIt in QTP and how you can go about setting up a system to use it within QTP without installing all the tools that come with a full version of the AutoIt software. Part 2 will follow shortly and will cover some sample code that demonstrates how I used AutoIt in my QTP script to solve my problem and will cover some code that allows you to automate downloading and registering the AutoItX3.dll from a Quality Center project. This allows you to be confident that any computer that runs any script requiring the AutoItX3.dll will automatically have it set up for them.
First the background, I recently had a requirement that I wanted to automate that was apparently something that is very difficult to do with automation. The requirement was that the inputs for a web application would be able to accept special characters like ¢ or ©. These special characters are available via Windows Character Map, or there are also ALT+<Number> keyboard shortcuts for the most common ones. For example, © is generated by ALT+0169. The rub here is that the shortcuts only work with the number pad keys (the block of keys to the right of a standard keyboard). They do not work with the number keys above the letters. Therefore, you cannot use the shortcuts on most laptop keyboards and as I found out, most keystroke automations do not specifically support sending the number pad keys.
I figured it couldn't be THAT hard to automate the entering of a special character via its shortcut. I figured there must be a way using WshShell.SendKeys or the .NET System.Windows.Forms.SendKeys object. Boy was I wrong. I tried several methods using the above approaches, none of which worked since, as I already pointed out, those methods do not have separate commands for simulating the number pad. Having given up on finding the answer without help, I posed the question asking how to do it on my favorite QTP forum/site, AdvancedQTP.
One of my favorite people, Tarun Lalwani, gave me a solution privately and asked that I not share it since he was planning on writing an article about it on his site (which I think may have taken over as my favorite QTP forum/site). He hasn't posted an article about it yet so I can't link to it, sorry. I know that he has been busy developing yet another awesome contribution to the community called PowerDebug, so I'm pretty sure the article in question is not a priority for him at this point.
His solution was quite complex and he had to go deeper into the Windows API than I probably would have for the answer. It employs the usage of the Extern object and the user32.dll. Of course, I probably could have just automated the character map application to enter special characters, but that seemed like a ham-handed approach to me. Or, I suppose I could have just set the text with some text generated ahead of time, but I was convinced there must be a way to automate the generation of the characters.
Anyway, I digress. Another solution I researched was an automation scripting tool called AutoIt. I found out that I could easily simulate the keystrokes I needed with it, but I always hesitate to use anything other than pure QTP since I don't know who is going to run my tests in the future and I can only count on their configuration including QTP. Of course, the main reason I didn't want to use it was that I didn't want to have to figure out a new scripting language and research or devise a method of calling it from QTP (laziness).
However, I found out that they have a COM automatable version that could be registered with COM and then created as an object and used directly in QTP scripts (or any VBScript for that matter), no fuss, no muss. Here are the steps to get the AutoItX3 COM object working on your system without installing any of the other software associated with a full install of the AutoIt environment (this is on a WinXP system, I haven't done it on Vista or Win7 yet):
- Go to http://www.autoitscript.com/autoit3/downloads.shtml and click the "Download ZIP" button. (Or just click this link.)
- Extract the contents of the zip file (this is a self-extracting zip, just double click it).
- Open the "install" folder, then open the "AutoItX" folder.
- Copy the AutoItX3.dll file and paste the copy of it in the C:\WINDOWS\system32 folder.
- Copy the AutoItX.chm file and paste the copy somewhere handy for future reference. This is the compiled help file that contains the documentation for all the available methods.
- Open a command prompt and type "regsvr32 AutoItX3.dll".
- You should get confirmation that the registration succeeded.
- You may now create an instance of the AutoItX3.Control object and use it in QTP or VBScript to your heart's content!
- Refer to the help file in step 5 for all the details about what you can do with it. AutoIt is a very, very powerful automation tool!
Stick around for part 2, with code!
Posted at 08:10PM May 27, 2010 by Christian in General | Comments[0]
Bringing Windows to the Front in QTP Scripts Part 2
Well, due to recent discoveries, I have to recant a lot of what I said in Part 1. Embarrassingly, that functionality is a snap to do with QTP's Window object, and I finally figured out how to successfully call ShowDialog on a .NET Form and pass the QTP window to it making the dialog modal to the QTP window. This is what I was trying to accomplish and the whole reason I began this quest in the first place. In my defense, I have only used QTP on web applications, so I wasn't familiar with the QTP Window object. All effort was not wasted though, at least I learned about the QTP's Extern object!
I didn't realize how easy it was to find windows by their title with Descriptive Programming and the Window object. To easily find the QTP window you can just code: Window("RegExpWndTitle:=QuickTest Professional"). QTP only allows one instance running, so I would think this is pretty safe, but you could add more identification properties if you are worried that this could come back identifying more than one window. From there you can get runtime object properties, including the Handle for the QTP window.
Since it is easy to pick out windows this way, Part 1's functionality can be much more easily duplicated by finding the window that you want to bring to the front by using the identification properties of the Window object and simply calling the Window().Activate function. This is obviously much easier than defining and calling external methods as presented in Part 1, most embarrassing.
Anyway, on the subject of making the .NET Form modal to the QTP window, one of the strategies I was unsuccessfully trying to implement was to call the FromHandle method on the .NET Control class to get an object that implements the IWin32Window interface to pass to the ShowDialog method on the Form object. The reason I couldn't get this to work was that I kept attempting to hand the "raw" Handle directly into the method. What made this the most confusing was that .NET objects throw a "Method not found" exception when an argument is of the wrong type, so I was just figuring that the method was not available due to the fact that I instantiated the Control object indirectly with the QTP's DotNetFactory object.
I found a solution here. This forum post clued me into the piece of the puzzle I was missing. The FromHandle and similar methods are looking for an argument of the IntPtr type. The IntPtr object is an assembly that holds the handle value. The post also clued me into the NativeWindow class and its AssignHandle method. When calling the ShowDialog method, the only thing defined by the IWin32Window interface is that it has a Handle property, so all we need is an object that implements the interface to which we can assign the Handle property.
To make a modal dialog from a .NET Form with the QTP window as the parent:
Dim myForm, hWnd, intPtrHWnd, qtpWnd
Set myForm = DotNetFactory.CreateInstance("System.Windows.Forms.Form", "System.Windows.Forms")
hWnd = Window("RegExpWndTitle:=QuickTest Professional").GetROProperty("hWnd")
Set intPtrHWnd = DotNetFactory.CreateInstance("System.IntPtr", "Mscorlib", hWnd)
Set qtpWnd = DotNetFactory.CreateInstance("System.Windows.Forms.NativeWindow", "System.Windows.Forms")
qtpWnd.AssignHandle intPtrHWnd
myForm.ShowDialog qtpWnd
myForm.Dispose
Set myForm = Nothing
Set intPtrHWnd = Nothing
Set qtpWnd = Nothing
The flexibility here is amazing. You can add any .NET form components to the form to build custom dialogs, inputs for passwords, etc. There are even SaveFileDialogs, FolderBrowserDialogs, andColorDialogs (to name a few), that can be shown modally to the QTP window in the same maner.
As promised, though, I was going to show the Shell.Application BrowseForFolder method, but of course, I will get the QTP window handle using the Window object this time.
Dim shell, hWnd, flder
Set shell = CreateObject("Shell.Application")
hWnd = Window("RegExpWndTitle:=QuickTest Professional").GetROProperty("hWnd")
Set flder = shell.BrowseForFolder(hWnd, "My Folder Browser", 0)
'Third arg is options arg, fourth arg is optional for root directory (Desktop is default, check docs for more info)
If flder Is Nothing Then
'Handle error
Else
MsgBox flder.Self.Path
End If
Set flder = Nothing
Set shell = Nothing
Posted at 11:52AM Feb 26, 2010 by Christian in General | Comments[1]
Bringing Windows to the Front in QTP Scripts Part 1
This became too long to put in one post, so I broke it into two parts. Part one will cover the QTP Extern Object and the User32.dll function SetForegroundWindow. Part two will cover the BrowseForFolder function on the Shell.Application object.
Over the past few months, I have done a couple of custom testing functions that require user input. For example, in one script I was exporting an Excel file with some metrics pertaining to average log in and page load times. I wanted to prompt the user (usually me) to enter a location to save the Excel file. I found the BrowseForFolder function on a Windows Shell.Application object, but the folder browser dialog never popped up in front. I looked high and low for a way to move my desired window to the front. The only thing I could find was WScript.AppActivate, but I never had any success with that strategy. I ended up giving up for a while, since I was usually the only one running either of these scripts, so I knew what to do when the script paused, anyway.
My next attempt came a few days ago as I was trying to figure out a good, best practices method for abstracting test data and environment specific values such as URLs and DB connections. I am attempting to set up a system where if the test is running in a test set, then the test set itself will dictate the environment specific values, but when you are developing and debugging in QTP, a local config file would dictate the environment specific values. I also wanted it smart enough that if someone who had no knowledge of this system could run the tests and it would prompt them for input regarding which environment they wished to be the default and to automatically create the local config file.
Enter the DotNetFactory. I must preface the rest of this by saying that I am obsessed with using .NET objects in my scripts now, so a bulk of my postings about QuickTest Pro will most likely be on that subject. I figured that I would present the user with a custom form with some instructions, a drop-down, and a couple buttons to allow them to input the default environment facilitating the creation of the local config file for a user who doesn't already have one. This seemed all the more feasible given the fact that the .NET Form Class has a ShowDialog method. After coding up my dialog, testing it directly in a script, and thinking I got it, I moved the code to a function library attached to the scripts for use throughout the project. Alas, when the code is called from a function library, the dialog never pops to the front.
I spent a bunch more time trying AppActivate and other fruitless strategies, but all the research into these techniques, however futile, ultimately educated me on Windows Handles, the QTP Extern object, and other useful info. The solution I found to bring my custom .NET Form to the front was to access the User32.dll function SetForegroundWindow. The only way I could figure out how to utilize this function was to use the QTP Extern object. I will offer a warning at this point that in my research, I got the impression that this technique may not always work, although it's worked for me so far, and that it may not work for Vista or Windows 7 computers. I'm not exactly sure why, but I guess Microsoft was trying to make it more difficult for applications to push their way to the front since a lot of developers were using this technique. Apparently, this is why the flashing application handle on the task bar has become the norm.
The Extern object is used for declaring a function from an external .dll file for use in a script or function library. One of the things that confuses me about the Extern object is that in the function declaration, you specify the return type for the function and in this case it is a boolean indicating if the function was successful or not, but there is no mic constant to specify a boolean return type. If anyone knows what that is all about drop me a line. Since there is no constant for a boolean return type and I'm not sure if I would really want, or need, to do anything about it if the function returned false anyway, I used void for the return type. Here is how you declare SetForegroundWindow in a script or function library:
Extern.Declare micVoid, "SetForegroundWindow", "System.User32.dll", "SetForegroundWindow", micHwnd
The first argument is a built-in QTP constant for return type. Refer to the QTP help files for more information. The second is the function name. The third is the .dll file name. The fourth is the function "alias" which can specify different entry points for different character encoding. Click here for more information. The fifth and any following arguments are the type(s) for the arguments that will be passed to the SetForegroundWindow function. In this case there is one argument required and it is the Windows Handle for the application you want to bring to the front. I discovered that this is simply a number and you can pass it as a long integer.
I want to point out that I truly don't have a full grasp on the ins and outs of the underlying functionality going on here. Most of my knowledge comes from reading documentation, other blogs and articles, web searches, and a ton of trial and error. Anyway, here's the code to show a .NET Form and bring it to the front:
Dim myForm
Set myForm = DotNetFactory.CreateInstance("System.Windows.Forms.Form", "System.Windows.Forms")
Extern.Declare micVoid, "SetForegroundWindow", "user32.dll", "SetForegroundWindow", micHwnd
Extern.SetForegroundWindow CLng(myForm.Handle)
myForm.ShowDialog
myForm.Dispose
Set myForm = Nothing
Stick around for part two where we'll look at making the BrowseForFolder dialog on the Shell.Application object snap to the front and explore Windows Handles in a little more detail.
Posted at 05:51PM Jan 19, 2010 by Christian in General | Comments[1]
QuickTest Pro Introduction
For almost a year now, I have been developing automated tests for a client who uses HP's (formerly Mercury's) QuickTest Professional and Quality Center. In my search for more knowledge and expertise as the months have passed, I have found that there is an amazingly huge community revolving around these products out there. The capabilities for automating application testing are amazing. However, I have also found that, like other software products, the documentation does not begin to hold the unreal amount of functionality included in these software tools, nor can the documentation possibly be able to cover all the real-world scenarios, best practices, or hacks.
The only way to really dig into the cutting edge of what can be accomplished is to go find those who are honing this edge and get yourself involved in the community. In fact, it even seems as if there is a lack of print books out there that cover these tools. The most complete one, which was written by an expert user in the community, was only recently released, http://www.amazon.com/QuickTest-Professional-Unplugged/dp/B002ACTWSM. I have not read the book yet hint, hint to the E-g management ;-), but luckily I have been able to pick the author, Tarun Lalwani's brain on more than one occasion on a couple community forums. Here is my list of resources that I use on an almost daily basis:
- Adavanced QTP: http://www.advancedqtp.com (This site is what I see as THE headquarters of QuickTest Pro and Quality Center online. Forum requires registration.)
- HP IT Resource Center (Quality Center): http://forums13.itrc.hp.com/service/forums/categoryHome.do?categoryId=895 (Requires registration)
- KnowledgeInbox: http://knowledgeinbox.com (Tarun Lalwani's site. Requires registration)
- Relevant Codes: http://relevantcodes.com (An Advanced QTP forum moderator, Anshoo Arora's blog)
- Learn QTP: http://www.learnqtp.com (Ankur's blog and forums. Forum requires registration)
There have been many challenges during my introduction to these products and how they can best benefit me, as a developer, and the client. As I have pointed out, there are tons of community resources and forums out there that provide plenty of knowledge, but I would also like to write about a few tricks I have learned, as well. Some of this might be repeat information, but I hope to present it in an easily digestible for anyone looking for some decent hints or advice about these tools. Stay tuned!
Posted at 09:33PM Jan 11, 2010 by Christian in General | Comments[0]
JavaMail and Gmail Simplified Part 5: Epilogue
I know that this is way too long as it is, but in going through my folders I found a folder where I was keeping example code found out in cyberland and I wanted to share part of the code that seems to make it into a lot of example Gmail code out there. This shows where people are populating configuration properties for which there are no reasons. If the protocol is specified properly, all of the things listed here are unnecessary.
Properties props = new Properties();
props.put("mail.transport.protocol", "smtp");
props.put("mail.host", "smtp.gmail.com");
props.put("mail.smtp.auth", "true");
props.put("mail.smtp.port", "465");
props.put("mail.smtp.socketFactory.port", "465");
props.put("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
props.put("mail.smtp.socketFactory.fallback", "false");
props.put("mail.smtp.quitwait", "false");
Session session = Session.getDefaultInstance(props, null);
Posted at 06:44PM Jul 01, 2009 by Christian in Java | Comments[0]
JavaMail and Gmail Simplified Part 4: The Code, Sending and Receiving
OK, back to the task at hand. If we are successful at connecting, we can do some real work with our Gmail account. Sending and receiving messages with attachments, multipart messages, HTML messages, etc. is beyond the scope of these posts since this is just a basic intro that will cover only the basic concepts of JavaMail and connecting to a hosted email account. We will just focus on plain text.
First, let's look at sending email. As stated, the Transport object handles this. Keep in mind that we are only scratching the surface of what the JavaMail API can do. Once you are familiar with the basics, you can do just about anything fairly easily.
/**
* Sends a plain text email
* @param to
* @param subject
* @param text
* @throws SimpleGmailClientException
*/
public void sendMessage(String to, String subject, String text)
throws SimpleGmailClientException {
Message message = new MimeMessage(session);
try {
message.setFrom(new InternetAddress("test@gmail.com"));
message.setRecipient(Message.RecipientType.TO, new InternetAddress(to));
message.setSubject(subject);
message.setText(text);
} catch (AddressException ae) {
throw new SimpleGmailClientException("Invalid internet address", ae);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error creating message", me);
}
try {
transport.sendMessage(message, message.getAllRecipients());
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error sending message", me);
}
}
As you can see, the code is pretty basic and straight forward. The method takes Strings for the "to" address, the subject, and the message text. One of the more complicated things is handling the exceptions that JavaMail will throw. In this case, I am wrapping all the exceptions in my own exception to add a friendlier message about where an error occurred. You can, of course, decide to throw the exceptions out to make the code a little cleaner and apply your own exception handling scheme if you wish. When learning from a newbie's perspective, though, I think that using the code the way it is will help point errors back to a recognizable point.
Next, let's look at reading email. In this case, I am simply going to take a String for a message subject to search for and read. When the message with the subject is found, a simple String representation of it is returned. Otherwise the method will return null.
/**
* Reads an email message with a given subject.
* Returns null if the subject is not found.
* @param subject
* @return A simple String representation of an email message,
* null if not found
* @throws SimpleGmailClientException
*/
public String readMessage(String subject) throws SimpleGmailClientException {
String messageString = null;
try {
for (Message message : Arrays.asList(inbox.getMessages())) {
if (message.getSubject().equals(subject)) {
messageString = "To: " + Arrays.asList(message.getAllRecipients()) + "n" +
"From: " + Arrays.asList(message.getFrom()) + "n" +
"Sent: " + message.getSentDate() + "n" +
"Subject: " + message.getSubject() + "n" +
"Text: " + message.getContent();
break;
}
}
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error reading Gmail Inbox", me);
} catch (IOException e) {/* Not using streams, only plain text */}
return messageString;
}
One other aside here, this is where a lot of confusion comes from as well. The user interface for most email clients place "deleted" messages in a "Deleted Messages" or "Trash" folder, but flagging and expunging in JavaMail does not. Messages that are expunged from folders with this technique are deleted forever! To emulate the way the UI works, you would have to explicitly move the message to the "Deleted Messages" or "Trash" folder with that folder's appendMessages() method.
All that's left is to create a main() method to use the client. To demonstrate the send and receive capabilities, we will login, send a message to ourselves, give it some time to arrive, read it, delete it, and logout. All you need to know is your Gmail username and password. (Click here to view full screen)
package com.e_gineering.simplegmailclient;
public class Main {
/**
* @param args
*/
public static void main(String[] args) {
SimpleGmailClient client = new SimpleGmailClient();
String subject = "This is a Test";
try {
String username = "{INSERT YOUR USERNAME HERE}";
String password = "{INSERT YOUR PASSWORD HERE}";
client.login(username, password);
client.sendMessage(username + "@gmail.com", subject, "This is some test text");
// Give the message some time to arrive...
try {Thread.sleep(10000l);} catch (InterruptedException e) {}
String message = client.readMessage(subject);
System.out.println(message);
client.deleteMessage(subject);
client.logout();
} catch (SimpleGmailClientException sgce) {
sgce.printStackTrace();
}
}
}
Oh, and here's the SimpleGmailClientException: (Click here to view full screen)
package com.e_gineering.simplegmailclient;
/**
* Wrapper Exception for exceptions thrown inside SimpleGmailClient
* @author christian.desserich
*/
@SuppressWarnings("serial")
public class SimpleGmailClientException extends Exception {
/**
* @param message
* @param cause
*/
public SimpleGmailClientException(String message, Throwable cause) {
super(message, cause);
}
}
I hope that this helps people get started using JavaMail and gets people to stop putting SMTP socket factory, SMTP port, and various other extraneous properties in their code. It is not necessary! I also hope this will put to bed all the questions about how to interface with Gmail via Java. There is, of course, so much more you can do with the JavaMail API. There are hooks to use it with a GUI front end, for example the Session factory method will take an Authenticator argument. The Message classes enable you to create complicated emails with attachments, multi-part messages, HTML messages, etc. There are folder listeners that enable you to monitor the status of the messages contained within. The list goes on and on
Sources:
http://java.sun.com/products/javamail/JavaMail-1.4.pdf
http://java.sun.com/products/javamail/javadocs/index.html
http://java.sun.com/products/javamail/FAQ.html
Posted at 10:43AM Jun 30, 2009 by Christian in Java | Comments[0]
JavaMail and Gmail Simplified Part 3: The Code, Logging In
Finally!! Let's look at some code. Here is a simple Gmail client object in its entirety (Click here to see SimpleGmailClient full-page):
package com.e_gineering.simplegmailclient;
import java.io.IOException;
import java.util.Arrays;
import java.util.Properties;
import javax.mail.Flags;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.NoSuchProviderException;
import javax.mail.Session;
import javax.mail.Store;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
/**
* A simple email client that will send and receive email with a hosted Gmail account
* @author christian.desserich
*/
public class SimpleGmailClient {
private Session session;
private Transport transport;
private Store store;
private Folder inbox;
private static final String INBOX = "INBOX";
/**
* Logs in to a Gmail account
* @param username
* @param password
* @throws SimpleGmailClientException
*/
public void login(String username, String password) throws SimpleGmailClientException {
session = Session.getInstance(new Properties());
try {
transport = session.getTransport("smtps");
transport.connect("smtp.gmail.com", username, password);
} catch (NoSuchProviderException nspe) {
throw new SimpleGmailClientException("Error getting Gmail Transport", nspe);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error connecting to Gmail Transport", me);
}
try {
store = session.getStore("imaps");
store.connect("imap.gmail.com", username, password);
} catch (NoSuchProviderException nspe) {
throw new SimpleGmailClientException("Error getting Gmail Store", nspe);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error connecting to Gmail Store", me);
}
try {
inbox = store.getFolder(INBOX);
inbox.open(Folder.READ_WRITE);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error opening Gmail Inbox", me);
}
}
/**
* Sends a plain text email
* @param to
* @param subject
* @param text
* @throws SimpleGmailClientException
*/
public void sendMessage(String to, String subject, String text)
throws SimpleGmailClientException {
Message message = new MimeMessage(session);
try {
message.setFrom(new InternetAddress("test@gmail.com"));
message.setRecipient(Message.RecipientType.TO, new InternetAddress(to));
message.setSubject(subject);
message.setText(text);
} catch (AddressException ae) {
throw new SimpleGmailClientException("Invalid internet address", ae);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error creating message", me);
}
try {
transport.sendMessage(message, message.getAllRecipients());
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error sending message", me);
}
}
/**
* Reads an email message with a given subject.
* Returns null if the subject is not found.
* @param subject
* @return A simple String representation of an email message,
* null if not found
* @throws SimpleGmailClientException
*/
public String readMessage(String subject) throws SimpleGmailClientException {
String messageString = null;
try {
for (Message message : Arrays.asList(inbox.getMessages())) {
if (message.getSubject().equals(subject)) {
messageString = "To: " + Arrays.asList(message.getAllRecipients()) + "n" +
"From: " + Arrays.asList(message.getFrom()) + "n" +
"Sent: " + message.getSentDate() + "n" +
"Subject: " + message.getSubject() + "n" +
"Text: " + message.getContent();
break;
}
}
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error reading Gmail Inbox", me);
} catch (IOException e) {/* Not using streams, only plain text */}
return messageString;
}
/**
* Deletes (permanently) an email message with the given subject
* @param subject
* @throws SimpleGmailClientException
*/
public void deleteMessage(String subject) throws SimpleGmailClientException {
try {
for (Message message : Arrays.asList(inbox.getMessages())) {
if (message.getSubject().equals(subject)) {
message.setFlag(Flags.Flag.DELETED, true);
break;
}
}
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error reading Gmail Inbox", me);
}
try {
inbox.expunge();
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error deleting messages from Gmail Inbox", me);
}
}
/**
* Logs out of Gmail
*/
public void logout() {
if (transport != null) {
try {transport.close();} catch (MessagingException me) {/* We could log this error */}
}
if (store != null) {
try {store.close();} catch (MessagingException me) {/* We could log this error */}
}
}
}
Fist let's dissect each piece. This is the setup and connection code:
/**
* Logs in to a Gmail account
* @param username
* @param password
* @throws SimpleGmailClientException
*/
public void login(String username, String password) throws SimpleGmailClientException {
session = Session.getInstance(new Properties());
try {
transport = session.getTransport("smtps");
transport.connect("smtp.gmail.com", username, password);
} catch (NoSuchProviderException nspe) {
throw new SimpleGmailClientException("Error getting Gmail Transport", nspe);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error connecting to Gmail Transport", me);
}
try {
store = session.getStore("imaps");
store.connect("imap.gmail.com", username, password);
} catch (NoSuchProviderException nspe) {
throw new SimpleGmailClientException("Error getting Gmail Store", nspe);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error connecting to Gmail Store", me);
}
try {
inbox = store.getFolder(INBOX);
inbox.open(Folder.READ_WRITE);
} catch (MessagingException me) {
throw new SimpleGmailClientException("Error opening Gmail Inbox", me);
}
}
Notice the creation of the empty Properties object to hand to the factory method. This is where you could potentially read in a properties file with the load() methods that will take a Reader or InputStream. As stated (many, many times), we will hand in our info as we go, instead. In this case, I am connecting to both the SMTP and IMAP servers so we can have both send and receive capabilities. If you only need one or the other of course you can split out the code as you please.
The other interesting thing going on here is the opening of the "INBOX" folder. The Store object is represented as a folder hierarchy. There is the "default folder" that is the top level of the hierarchy. Servers can have the "INBOX" folder as the default folder, apparently, but in Gmail's case, the conceptual inbox is contained in the default folder along with another folder hierarchy that I will explain below. If you are going to apply this code to a different host besides Gmail, it is supposed to be standard that all mail servers have a folder, called "INBOX" that you can access, but don't count on it unless you know its there.
So to open the inbox, we have to explicitly get it by passing its name to the getFolder() method. JavaMail will automatically search down the hierarchy to find the folder name you pass into the method, so there is no need to loop through the hierarchy to find and get a certain folder. If you have the name correct, JavaMail will find it. The other thing going on is that you have to explicitly open a folder with read-only or read-write access. If you attempt to open a folder with read-write permissions, but are forbidden to do so, an exception is thrown.
Here's a quick overview of Gmail's IMAP folder hierarchy. Inside the default folder ("") there is the "INBOX" folder which we are already familiar with and the "[Gmail]" folder which we are not.
Inside the "[Gmail]" folder, there are the Gmail specific folders like the sent mail folder. These folders are where you could do things like purge sent mail, move messages to other folders, delete spam, etc.
Posted at 12:28AM Jun 26, 2009 by Christian in Java | Comments[0]
JavaMail and Gmail Simplified Part 2: The Objects
First let's explore the main objects that we are concerned with for simple interfacing of a Java application with a Gmail account using the Javamail API. The Session object is the main object holds the connection configuration information and factory methods for itself and the other two main objects, Transport and Store. Almost all configuration information needed for connecting to, authenticating with, and sending and receiving email can be handed to the Session object's Session.getInstance(Properties props) factory method via the Java Properties object argument, but at least in this case, we are not going to hand anything into the Session, so we can see where the info is coming from as we step through the code (Yes, there is code coming eventually!). The Session is not really a session like an active connection, just the holder for the settings and the factory methods for the Transport and Store objects.
Transport and Store represent send and receive capabilities respectively. This didn't make a whole lot of sense to me at first, but I realized that the two capabilities speak to two different hosts with two different protocols. Also, I would think that the send capability is used most often by developed applications (sending mass emails, email alerts, confirmation emails, etc.) so there is typically no need to have both capabilities simultaneously.
To get a Transport, we call the factory method on our Session object, session.getTransport(). This is where things can get confusing since there are several ways to configure the connection. This factory method will take no arguments (which will get all its configuration from the configuration stored in the Session object), an Address object (which will apparently provide a place to load the Transport object with the information it needs to configure itself), a Provider object (which will configure the returned Store object), a URLName object (not really sure the use of this one), or a String designating the protocol.
To get a Store, we call the factory method on our Session object, session.getStore(). This is the same place things can get confusing for the same reason as the Transport factory method. This factory method will take no arguments (which will get its configuration from the configuration stored in the Session object), a Provider object (which will configure the returned Store object), a URLName object (not really sure the use of this one), or a String designating the protocol.
For simplicity's sake in my example code, I did not hand the Session object ANY configuration via its factory method with the Java Properties object argument so that we can see and know for sure where the configuration info comes from. For Gmail specifically, for both the Transport and the Store, I used the respective factory methods that take the String for the protocol. For Gmail, send is "smtps", receive in my case is "imaps." Gmail allows POP3 as well, but I liked IMAP better since there were more methods on the Message object that were supported by IMAP (for example getReceivedDate()). The "s" on the end of the otherwise familiar protocols is for SSL.
Now! After all this, we need to actually connect to the Transport and Store. Both objects have the same connect methods that take similar arguments. This is another point of confusion, since the method will take usernames and passwords along with the host name. This is where I passed this information in instead of passing it into the Session object via a Java Properties object. I think this is better for learning since we know where all the values are coming from at development time.
As the code matures, things could be progressively moved to a "higher" configuration level so that all configuration efforts could be focused in one place. Every value used directly in the code (protocols, host names, usernames, and passwords) could be read in from a properties file using a Properties object and handed into the Session factory method enabling us to use the no-argument Transport and Store factory methods on the Session and the no-argument connect() methods on the Transport and Store. However, I think for demonstration purposes, this makes it easier to understand. You can figure out your own best way to do things, there are several choices with the JavaMail API.
Posted at 02:58PM Jun 25, 2009 by Christian in Java | Comments[0]
JavaMail and Gmail Simplified Part 1: Introduction
This post in its entirety turned out to be quite a bit longer than I originally thought it would be, so I am going to break it into parts. Caution: This is going to be very basic and is written for beginners. I felt that there was a need out there for this type of introductory tutorial based on a hosted email account, especially Gmail. So, if you're looking for an in-depth guide, the JavaMail documentation is a better place to start. If you wish to continue, here's part 1, stay tuned for parts 2, 3, and 4
In developing a simple email client for a customer that needed to programmatically (no UI) send, receive, and read email from a Gmail account, I had to do a lot more research into the JavaMail API than I really wanted to. I, just like a lot of people, was simply looking for a code snippet that would point me in the right direction for logging into a Gmail account and performing the most basic of operations. In searching for examples and trying to apply certain aspects of them to my application, I found that there is a lot of code out there that is over-coded and a lot that simply doesn't work.
It seems that many people find the example code packaged with the JavaMail API confusing. I can't say that I agree with this perception, nor can I say I don't disagree because many of the simple examples people are looking for are either not covered in a simple, "JavaMail-for-Dummies" manner or not covered specifically. Also, some examples make extensive use of command-line arguments for configuration. The following of the command-line arguments through the code and, of course, the configuration itself are very likely to confuse newbies.
The biggest problem I see with the code I found on the Internet is the over-coding. People have a tendency to code in tons of configuration properties that are totally unnecessary. This over-coding comes from a lack of understanding about what information is needed for what situation and how to communicate that to the API. There seems to be several ways to pass this configuration information to the API. These are listed in rough order that they will override the settings in each "higher-level" technique. Disclaimer: What follows is too simplified an explanation. The reasons for using the different methods of setting up the API exist for very different reasons for each application and I certainly don't profess to know all the ins and outs of the entire API. Also, for our purposes, we don't need to know all this, but you can refer to the JavaMail Specification or the API JavaDocs for more info.
1. The API can be initialized with "registry" files that can be placed in a location that the API searches for.
2. A Session object is created with the appropriate settings passed as an argument to the static Session.getInstance(Properties props) factory method as a Java Properties object (This is the most common way).
3. Settings can be passed into factory methods on Session that create the JavaMail objects, Transport and Store, and settings can be passed into their connect() methods.
4. There are setters for a few of the settings.
One of the things that frustrates beginners is that some of the properties that can be passed into the static Session.getInstance() method via a Java Properties object are not documented. This seems to be where a lot of the over-coding comes from. I see a ton of code out there that specifies ports, SSL socket factory classes, and other unnecessary information. If you use the correct methods and specify the correct protocol, JavaMail does the rest. All that extraneous information is needless for interfacing with a simple hosted email account and must be the result of cutting and pasting from more advanced code. The whole purpose of Javamail is to make the job of interfacing with different email server protocols and designs easier!
Posted at 10:15PM Jun 24, 2009 by Christian in Java | Comments[0]
Converting Database Data Directly to XML
I had another interesting problem when an application framework we've been working with came out with a new version. Before, a value relating to the application user's "Position Name" (i.e. Manager) was a user-entered value. In the new release, this value was supposed to come from an XML configuration. Our client had already entered over 200 different positions and understandably didn't want to take the time to hand enter each one into this new configuration. I found out that DB2 has the XMLSERIALIZE() function and decided that this would be the easiest way to accomplish this without hand-typing each position name back into the system.
As mentioned above, in the original version of the framework the position name was typed in, but in the new version the configuration would be populated in a drop-down. Plus the configuration XML added two new elements that were obviously not present before. So the technique I used capitalized on some of the knowledge I gained working out the Vacuum of Columns. It turned out to be a simple select with two "dummied" columns.
select distinct XMLSERIALIZE(content XMLELEMENT(name "Position_Name", XMLELEMENT(name "Value", system_users.position_name), XMLELEMENT(name "Dollar_Limit", CAST(NULL AS VARCHAR(1))), XMLELEMENT(name "Level", CAST(NULL AS VARCHAR(1)))) as VARCHAR(120)) as "Result" from system_users
You can see that I used the cast(null as ___) technique from my earlier post to make sure the element tags existed even though they were empty, so that I didn't have to put those in by hand either. I executed this in SQireL and just cut and pasted it into the configuration and voila! All done with little fuss and less muss.
This particular example was on the verge of not being worth the time it took to figure this out, but I again learned something that may be extremely handy in the future, and it might save someone else some time, too. For more information, here is the article on IBM developerWorks :
http://www.ibm.com/developerworks/db2/library/techarticle/dm-0511melnyk/
Posted at 05:00PM Jan 26, 2009 by Christian in General | Comments[0]
The Vacuum of Columns
I know that I'm laying on the space analogies pretty thick, but it just seems to work so far... Anyway, I learned something interesting today and I thought that I would pass it on. I'd like to start off saying that I don't know if this is the best way to go about trying to combine fairly unrelated records from fairly unrelated tables. If anyone has better ways to do this type of thing PLEASE let me know. I think given my constraints, this is the best way, but I would love to refactor if I could reduce the complexity.
Nevertheless, here goes. I had a situation where I had to pull records from two fairly unrelated tables that really were related based on certain conditions. One table (table B) was actually the child of a child of the other table, but there was not always a "grandparent" record in the other table (table A). Basically, I needed the record from the "grandchild" table (table B) unless it had a "grandparent" record (in table A), in which case I needed to see that record. That wouldn't be a huge deal if the tables shared column names and primary keys, but they didn't and we didn't want to do a whole redesign at this point just to satisfy this "list" view of the combined records. What I needed was to be able to alias null values for the columns that the tables didn't have in common so I could do two different selects with a SQL UNION. It took me a while to find some solutions, but this blog is where I found my answers. Here's how I did it:
| Table_A | |||
|---|---|---|---|
| primary_key | col_1 | col_2 | col_3 |
| 1 | aaa | bbb | ccc |
| 2 | ddd | eee | fff |
| 3 | ggg | hhh | iii |
| 4 | jjj | kkk | lll |
| Table_B | |||
|---|---|---|---|
| composite_key_1 | composite_key_1 | col_1 | col_2 |
| 1 | 1 | mmm | nnn |
| 1 | 2 | ooo | ppp |
| 2 | 1 | qqq | rrr |
| 2 | 2 | sss | ttt |
You can see that there a some common columns, but obviously the primary keys need to be separated and I need one column that doesn't exist in the other table. When using the UNION operator, though, the number of columns and their names need to be the same. Plus, as I mentioned some of the records in table B might have a related record two generations away (really just linked through one other table) in table A, but in that case I only want to see the record from table A. The approach I wanted to use was to have the columns that did not exist in table A to be there in the results, but be null in value and vice-versa the other direction. I found out that you could call a function that cast "null" as the correct data type and then alias it to make this happen. Basically, to get null integers (for the primary keys) I used the CAST() function like this:
FROM Table_A
FROM Table_B
LEFT INNER JOIN Intermediate_Table ON Table_B.foreign_key = Intermediate_Table.foreign_key
LEFT INNER JOIN Table_A ON Intermediate_primary_key = Table_A.primary_key
WHERE primary_key IS NULL
FROM Table_A
UNION
SELECT CAST(NULL AS INTEGER) AS primary_key, Table_B.composite_key_1, Table_B.composite_key_2, Table_B.col_1, Table_B.col_2, CAST(NULL AS VARCHAR(25)) AS col_3
FROM Table_B LEFT INNER JOIN Intermediate_Table ON Table_B.foreign_key = Intermediate_Table.foreign_key
LEFT INNER JOIN Table_A ON Intermediate_primary_key = Table_A.primary_key
WHERE primary_key IS NULL
| Result Set | |||||
|---|---|---|---|---|---|
| primary_key | composite_key_1 | composite_key_2 | col_1 | col_2 | col_3 |
| 1 | null | null | aaa | bbb | ccc |
| 2 | null | null | ddd | eee | fff |
| 3 | null | null | ggg | hhh | iii |
| 4 | null | null | jjj | kkk | lll |
| null | 1 | 1 | mmm | nnn | null |
| null | 1 | 2 | ooo | ppp | null |
| null | 2 | 1 | qqq | rrr | null |
| null | 2 | 2 | sss | ttt | null |
You can see where the columns have been created where they didn't exist before and are filled with null values. In fact you could fill the result set with any values, but for this purpose null had the most (or least) meaning.
Posted at 09:17PM Aug 04, 2008 by Christian in General | Comments[0]