A year or so ago, I thought it would be great if there was a tool that would allow me to easily record notes and thoughts via command line. At the time, nothing really impressive existed (and I was too lazy to create something myself). Luckily, wandering the the net this morning, I came across jrnl1. Coupled with intelligent date parsing, this is the ideal tool for me to quickly record anything that needs noting with minimal effort. Export options, including markdown2 and JSON make it doubly appealing.
At a high level, MTOM1 is a SOAP optimisation feature for binary data. As SOAP uses XML as the standard message format, this presents an issue when binary data must be included in a SOAP message. To work around this limitation, binary content can be in-lined in a SOAP message as a Base64-encoded string (specifically, the
base64Binary primitive data type2). Encoding to Base64 text does not come without a price; a 33% size increase of the binary content3. Enter MTOM to the rescue (sort of).
MTOM allows you to work around the Base64 performance hit by replacing the in-line Base64 text with a reference to a separate MIME part containing the binary data. For example:
SOAP Message Content without MTOM
Note: the binary content in line -
<?xml version="1.0" ?> <S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope"install> <S:Header> . . . </S:Header> <S:Body xml:id="id"> <Request xmlns="http://somenamespace/for/Request"> Li4uIGJpbmFyeSBjb250ZW50IC4uLg== </Request> </S:Body> </S:Envelope>
SOAP Message content with MTOM
Note: the binary content now replaced with an XOP include reference to the
Content-Id of the additional MIME part —
--uuid:952d2ab0-043e-4233-bc73-40e3f2c01694 Content-Id: <rootpart*952d2ab0-043e-4233-bc73-40e3f2c01694> Content-Type: application/xop+xml;charset=utf-8;type="application/soap+xml" Content-Transfer-Encoding: binary <?xml version="1.0" ?> <S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope"install> <S:Header> . . . </S:Header> <S:Body xml:id="id"> <Request xmlns="http://somenamespace/for/Request"> <xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:64449861-d66c-4540-86bc-4486bfa796cc"></xop:Include> </Request> </S:Body> </S:Envelope> --uuid:952d2ab0-043e-4233-bc73-40e3f2c01694 Content-Id: <64449861-d66c-4540-86bc-4486bfa796cc> Content-Type: application/octet-stream Content-Transfer-Encoding: binary ... binary content ... --uuid:952d2ab0-043e-4233-bc73-40e3f2c01694
So this brings me to back to the title of this post and what problems arise when you start to alter your SOAP message. While this post is specifically targeted at JAX-WS4 and Java, I have seen similar issues on the .Net side of the fence when examining or altering the SOAP message and its MTOM multi-part structure. In my experience,
SOAPHandlers5 are typically used to access the state of your request or response before it heads out into the ether. At times, this might be to log the content or to alter it directly. Irrespective of your objective (and I’ve tried both independently), adding a
SOAPHandler to your port proxy’s handler chain6 breaks MTOM in that your outgoing SOAP request/response reverts to the following:
SOAP Message Content with MTOM and a SOAPHandler
Note: The Base64 string is back!
--uuid:952d2ab0-043e-4233-bc73-40e3f2c01694 Content-Id: <rootpart*952d2ab0-043e-4233-bc73-40e3f2c01694> Content-Type: application/xop+xml;charset=utf-8;type="application/soap+xml" Content-Transfer-Encoding: binary <?xml version="1.0" ?> <S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope"install> <S:Header> . . . </S:Header> <S:Body xml:id="id"> <Request xmlns="http://somenamespace/for/Request"> Li4uIGJpbmFyeSBjb250ZW50IC4uLg== </Request> </S:Body> </S:Envelope> --uuid:952d2ab0-043e-4233-bc73-40e3f2c01694
So you’ve effectively got MTOM working (it’s still got an MTOM MIME type encapsulating the SOAP envelope), but with none of the bloat-reducing benefits.
Of all the difficulties I’ve had, it was determining the root cause of this issue. Unfortunately, I still don’t have a solid answer. However, this old JIRA issue is the best reference for an explanation I could find:
When I use a logging handler that implements javax.xml.ws.handler.soap.SOAPHandler
, and use MTOM the message that is returned by the service is still MTOM but the file attachment is no longer returned as an xop reference; instead it is returned as inline base64 text. If I omit the handler by commenting out the @HandlerChain annotation in AddNumbersImpl.java and rebuild the service, the file attachment is returned as an xop reference.
To recreate the issue deploy the attached war file in Tomcat 6.0 and then run the csharp client from the csharp-client directory by invoking the run.bat file without arguments. Make sure you also capture the input and output messages on the Tomcat console by enabling the following JVM option:
When @HandlerChain is commented out you will see the attachment returned as an xop reference. When it is used, you will see the attachment returned as inline base64 text.
Joe Roberts - JAX-WS JIRA Ticket WSIT - 13207
The key information to note here is that the resolution of the ticket is Will Not Fix, so to my knowledge, this remains unfixed8.
So what to do? I found there were a number of suggestions out there including use of tubes9 and codecs10. These options proved difficult to implement; more a low-level hack than a simpler solution. Fortunately, my attention was drawn in more detail to the
SOAPMessageContext11. In my frantic efforts to solve this problem, I had overlooked the basic functionality at the
SOAPMessage12 level that provides the ability to add additional
AttachmentParts13 to the SOAP message. To cut a long story short and get to the workaround or solution, this allowed me to do the following:
- Extract the Base64-encoded text from the SOAP body.
- Create a new
AttachmentPartto hold the extracted content in byte form.
- Replace the original Base64-encoded text from the SOAP body with a reference element that points at the part in Step 2.
While this is at a relatively high level, that’s essentially all that is required. The following snippets give you a rough idea of the code required to carry out these steps:
/** * Note: no null checking or anything nice like that done as I only want to illustrate the * process. */ SOAPEnvelope envelope = context.getMessage.getMessage().getSOAPPart().getEnvelope(); SOAPBody body = envelope.getBody(); // Create a new reference for use with the attachment part. UUID ref = UUID.randomUUID(); // You then get the element content using something like the following. // This will be dependent on the type of body you're dealing with. NodeList nodeList = body.getElementsByTagNameNS("namespace", "localname"); Element element = element = (Element) nodeList.item(0); String elementContent = element.getFirstChild().getNodeValue(); InputStream is = IOUtils.toInputStream(elementContent); // Create a new part with a reference ID. AttachmentPart attachment = context.getMessage().createAttachmentPart(); attachment.setBase64Content(is, "content/type"); attachment.setContentId(ref.toString()); context.getMessage().addAttachmentPart(attachment); // Clear the old content. SOAPBodyElement bodyElement = (SOAPBodyElement) element; bodyElement.removeContents(); // Add the new include element, using the reference ID. bodyElement .addChildelement("Include", "xop", "http://www.w3.org/2004/08/xop/include") .setAttribute("href", "cid:" + ref.toString());
This all should take place within a new
SOAPHandler that you add to your handler chain. Depending on your use case, you can do this on request and/or response.
At best, this solution is a workaround but it will get you past a fairly significant hurdle, should you be constrained to using MTOM in your service calls. As always, if anyone has any comments or recommendations on how better to solve this problem, please leave them in the comments section.
http://jax-ws.java.net/nonav/2.1.2/docs/handlers.html - Do a search for Programmatic Case for an example on adding a handler to your chain. ↩
I’m constrained to Java 1.6 and by extension, JAX-WS 2.1. The problem does still seem prevalent in later versions based on my limited testing. ↩
I’ve been using Instapaper1 for a while now. While it’s an excellent service, more often than not, I’ve needed a way of getting through some of the more lengthy articles on my Kindle2. Fortunately, there’s a feature that allows you to download a Kindle/mobi format file. To give you and idea of what’s produced, see the image below:
- Login to Instapaper
- Check if there is are unread items
- Download a compiled mobi format book
- Save mobi book to Dropbox
- Archive the unread items
Having done the above, it’s then relatively easy to access the books via the web browser on the Kindle. The script isn’t particularly robust or useful outside of this use case but feel free to take a look at it to give you an idea on:
- logging in to pages programmatically
- page scrapping
If you are interested, feel free to check out the code and fork me on Github5.
I recently found the need to record a screen cast for a demo of an application I was working on. If you’re using a Mac, there is an extremely simple option available to you using the in-built QuickTime1 application.
To get started, you need to launch
You then need to select the option to create a new
Actions, do a search for
Screen Captureor something similar. You should see the following appear:
New Screen Captureaction into the right-hand workflow pane and you’re almost done:
If you click the
Runicon, you should be presented with the following after QuickTime has launched:
Previously, I needed to rely on third-party apps that cost money so it’s nice to have this feature so tightly integrated with the OS. Having created and saved the workflow, the only thing left to do is make it easy to launch. I’m a big fan of Alfred2, so I can use the features provided by the Alfred Powerpack to associate workflows with the launcher:
It shows up like this when you attempt to launch the new workflow from Alfred:
Alternatively, you can create a keyboard shortcut using these3 instructions.
- From the
- From the leftmost panel, pick the
- On the rightmost panel, you should now see a
Show Notification Centercheckbox. Ensure it’s checked and then add a shortcut. I picked
N. That one doesn’t seem to conflict with anything obvious so far.