Category Archives: web services

Endless polling ajax gotchas with recursive calls

There’s a pretty good piece on how to make your page continually update over here which works pretty well. With one problem. In my mind, a rather major problem. If you ever have an exception in the success, error or complete handlers inside the $.ajax() call, javascript error handling junks the rest of the function, and you never make the recursive call.

Old and Busted

(function loopsiloop() {
   setTimeout(function() {
       $.ajax({
           url: 'foo.htm',
           success: function(response) {
               // BANG! TypeError Cannot read property 'xxx' of null
               console.log(response.expected_property);
           },
           complete: function() {
               // Only works if success didn't throw an exception
               loopsiloop();
           }
       });
   }, 5000);
})();

In certain conditions, jquery’s success handler gets called with what could hardly be called a successful response. Maybe you got a 404 page instead of the json you were expecting, or whatever. Fair bet that your handler code wasn’t expecting that. Fair bet that it was actually transient (rebooting the server for instance) Fair bet that you actually want it to just try again in five seconds.

Well, sucks to be you, but the first time your success handler throws any sort of exception, javascript just ate your function.

New and Sexy

Use jQuery’s deferred objects

(function loopsiloop_sexy() {
        setTimeout(function() {
            $.ajax({
                 url: 'foo.htm',
                success: function(response) {
                    // BANG! (we didn't stop it throwing TypeErrors)
                    console.log(response.expected_property);
                }
            }).always(function() {
                // This will always be called, regardless of exceptions in the ajax call
                loopsiloop_sexy();
            });
        }, 5000);
    })();

Note that the .always call is outside the anon function that $.ajax is calling. So now, even when your success/error/complete methods explode, you haven’t broken the endless loop. And as soon as your server starts serving what you expected again, it all starts working again. Yay!

SoapUI 3.6.1 is broken for REST, use 3.6.0 or 3.6.2-SNAPSHOT

That’s about all you need to know :) SoapUI 3.6.1 has a bug that causes duplicate requests to be made to REST endpoints. It will mostly seem to work fine, but you’re getting more requests to your server than you expected, with not quite the data you expected :) I’ve moved to a snapshot of 3.6.2, which seems to be working fine so far.

common object models in webmethods and java webservices

What all the ranting and soap tedium of the previous post was leading to, was that we want to have some common object models shared across multiple java/.Net webservices, as well as webmethods business processes.

So, we made a few nice schemas, all in one namespace, and made a few test webservices and clients, and made sure that the objects could be happily passed from one to the other without needing to do any field by field level copying.

To repeat, the object model of customers and products, is multiple schema files, all in one namespace. This makes it easier to work on one part at a time, and means less people have to get their heads around 20 different namespaces.

Each service then has it’s own separate namespace, to allow later deployment of different versions side by side, but uses the same common object model.

Because this leads to having wsdl documents that reference multiple schemas, that may reference further schemas, you need to be very fucking sure that your webservices library is doing the right thing&tm; so that everyone actually gets the types they were expecting.

JAX-WS Commons doesn’t do a very good job of this. You can _make_ it do the right thing, but it takes a lot of work. And when you want to generate code from the wsdls, and include niceties like toString and equals and hashcode and the other tasty xjc plugins, you get a world of hurt with overlapping maven plugins and snapshot dependencies. Again, you can make it work, but it’s more difficult than it should be.

CXF seems to just work so far.

And that means that webmethods can actually import wsdls from CXF, and we can be confident that those are the same ones that we wrote, and that the objects will be portable.

WebMethods gives us a nice big warningwebmethods warning about duplicate types but it works. As best I can tell, as long as your “duplicate” types really are the same, you can safely ignore this warning.

Multiple schemas in WSDL, jaxws, cxf, soapui and webmethods.

This is the _short_ version.

We write a few wsdls here. We’ve been exposing them with plain stock JAXWS built into java 6, along with JAX-WS Commons and regular spring. This has mostly been working fine. We even have some maven archetypes to generate the templates around this and it’s all well and good.

As long as your schemas and types are all in a single .wsdl file

You see, jax-ws commons is basically unmaintained. As you can see in the quickstart, you should just need this…

<wss:binding url="/sub">
    <wss:service>
      <ws:service bean="#myService" />
    </wss:service>
  </wss:binding>
 
  <!-- this bean implements web service methods -->
  <bean id="myService" class="foo.MyService" >
      <property name="something" value="somevalue"/>
  </bean>

And, that actually works. Except…. Even if you started from a wsdl, and then created a service that implemented the wsdl, the wsdl jax-ws commons will expose at runtime is generated dynamically. So none of that documentation and and xml annotations you slaved over for hours/days/weeks are actually visible anywhere.

Ok, No big deal! It’s still all standards based, we just lost our documentation. Welllll. Only if you added more annotations specifying the namespaces. If you just have the @WebService(serviceInterface = “the.generated.from.wsdl.Interface”) jax-ws commons will make you a webservice whose runtime wsdl will have a namespace created from your implementation package name! (Instead of the originally defined wsdl namespace)

So you can add more annotations, oh, one of them is wsdl location! sweet! oh. no. It can’t reference into a jar.

So you look at the wss: namespace based config, oh look, we can specify the wsdl there too! Oh. no. It can’t reference into a jar. Hmm, the schema’s broken. primaryWsdl is an attribute, but can’t be used as such…

You check the documentation again, and give it a go… Ok, primaryWsdl is a child element? Ok! this seems to work!

Then you look at your runtime wsdl…

<xsd:import schemaLocation="Core_v1.xsd" namespace="http://core.vf.is/dom/v1"/>

Hmm, how’s it going to see that file at runtime? Oops. you’ve just made an invalid wsdl. Back to the docs….. what docs? There are none. but suffice to say, there’s a magic key that does what you want… ws:metadata It’s defined as taking a list of schemas. When you get this right, it knows how to magically create runtime links to them. Of course, there’s a bug for this, but no-one’s working on it.

Except, as we mentioned, jax-ws commons is effectively unmaintained. There’s a bug in the schema that only allows one metadata element, instead of a list. So you can go and use the full expanded plain old spring config. But good lord that’s ugly. And how on earth would you have survived this far?

    <wss:binding id="wscustomer" url="/services/customer1">
        <wss:service>
            <bean class="org.jvnet.jax_ws_commons.spring.SpringService">
                <property name="bean" ref="servicesCustomer1"/>
                <property name="impl" value="is.vf.test.ws.customers.v1.CustomersV1Impl"/>
                <property name="primaryWsdl" value="customers_v1.wsdl"/>
                <property name="serviceName">
                    <bean class="javax.xml.namespace.QName">
                        <constructor-arg index="0" value="http://core.vf.is/customers/jaxws/"/>
                        <constructor-arg index="1" value="customers_v1"/>
                        <constructor-arg index="2" value="tns"/>
                    </bean>
                </property>
                <property name="portName">
                    <bean class="javax.xml.namespace.QName">
                        <constructor-arg index="0" value="http://core.vf.is/customers/jaxws/"/>
                        <constructor-arg index="1" value="customers_v1SOAP"/>
                        <constructor-arg index="2" value="tns"/>
                    </bean>
                </property>
                <property name="metadata">
                    <list>
                        <value>Core_v1.xsd</value>
                        <value>Customers_v1.xsd</value>
                        <value>ProductCatalog_v1.xsd</value>
                    </list>
                </property>
            </bean>
        </wss:service>
    </wss:binding>

There’s patches that aren’t applied, bugs not fixed, it’s a zoo. But yeah, you can make it work with the very manual and repetitive raw spring bean config.

But where did webmethods come into this?

Well, basically, it just fails abominably when it gets one of these wsdls that references unreachable schemas. And if you were using the autogenerated ones, and weren’t super careful, you ended up with types in webmethods that weren’t actually the same type.

Did I mention that when you import a wsdl into webmethods, it creates a doc type for every type, even if the same type has already been imported via another wsdl? It gives you a warning for each duplicate type, but fortunately, at run time, it seems acknowledge that the two types are actually the same thing. Thank god.

Enough of this shit.

Want to do the same thing with CXF?

     <jaxws:endpoint id="wsCustomer" implementor="#servicesCustomer1" wsdlLocation="customers_v1.wsdl" address="/customer1"
                    xmlns:kz="http://core.vf.is/customers/cxf/" serviceName="kz:customers_v1"
                    endpointName="kz:customers_v1SOAP"/>

You still have to put in the serviceName and enpointName, even though they just copied from the wsdl itself but hey, it works, it’s way more intuitive, and it’s less typing.

(Don’t even get me started on the quality of the generated types from CXF vs JAXWS-Commons.)

bleh.

problems with webmethods developer/designer 8 (eclipse based)

So I’ve upgraded from v7 of webmethods developer (with the custom three column sort of view) over to v8 of webmethods designer (eclipse based)

Here are the stupid, annoying, weird things that I haven’t managed to fix (yet?)

  • Entire flows are marked in grey until you check them out for editing. I don’t want to edit it, but I want to be able benefit from having a colour screen. I also want to be able to separate disabled steps. Sure, the icons are still there, but it’s all grey.
    Which steps are active?

    Which steps are active?

  • Comments on a flow are now a single line property. They used to be a whole panel of free form text. Given how awkward it is to comment the “code” any other way, and given that this is the only place to put any overall comments on the whole flow, and given how crap the VCS integration is (at least in our setup?) this box is pretty important for staying sane. yes, you can use the “…” link to get into a box editor, and i’ll probably get used to this one, but it used to be a bit easier
  • No keyboard shortcut to expand the flow. And no config setting to always start expanded. Although right and left arrows will open and close, they only open/close a single level. That’s not good enough.
  • Pressing delete on the keyboard doesn’t work for flows/folders in the Package Navigator. It does work for flow steps. Even though right click “delete” shows Delete as being the keyboard shortcut
  • Enough for now. bleh.

WebMethods Designer 8 with eclipse 3.6

SoftwareAG’s webmethods designer 8, the eclipse based replacement for “Developer” comes packaged into a rather old version of eclipse, 3.4. if you’d rather use it with, say, 3.6, here’s what you need to do…

Install it as it likes, then open up c:\softwareAG\eclipse\updates. This is the goodness :)

Start up your eclipse 3.6, and under “help->install new software” choose “add” then “archive” and find first of all, the com.softwareag.common.zip and install it. The feature should install happily there.

But, to install the DeveloperPlugins.zip, you need to do a bit of hackery.

Inside the zip, open up features/com.softwareag.is.feature_8.1.0.0000-0193.jar, and then open up com.softwareag.is.feature_8.1.0.0000-0193.jar (your version may differ, but it should be much the same)
Then, open up feature.xml and you’ll see the following list of requirements:

   <requires>
      <import feature="org.eclipse.gef" version="3.4.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.wst" version="3.0.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.jst" version="3.0.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.platform" version="3.4.2" match="greaterOrEqual"/>
   </requires>

Unfortunately, org.eclipse.wst and org.eclipse.jst don’t exist anymore, they’ve been broken up into smaller pieces, and renamed.

Remove those two lines from the requires list, and replace them with this:

   <requires>
      <import feature="org.eclipse.gef" version="3.4.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.wst.ws_core.feature" version="3.2.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.jst.web_core.feature" version="3.2.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.jst.enterprise_core.feature" version="3.2.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.jst.server_core.feature" version="3.2.0" match="greaterOrEqual"/>
      <import feature="org.eclipse.platform" version="3.4.2" match="greaterOrEqual"/>
   </requires>

Save your way all the way back out, and then use this zip as a software source as before, and the new feature should install just fine :)

You may not need all three jst.XXXX_core features, but this was enough to get it working for me.

oh, You’ll know you need this, if you get the following errors when you try and install from the original .zip files.


Ack, try and get back to the initial state where this didn't work to catch the whole error message to make googling easier.

org.apache.xerces.parsers.XML11Configuration cannot be cast to org.apache.xerces.xni.parser.XMLParserConfiguration

Update 2010-09-29:
So, depsite being able to trigger it live, and knowing that it’s always a problem with an exception being thrown by the web service, and something in the soap fault parsing being broken, I still haven’t been able to actually fix this. Sometimes I get a parser cast error for a soapfault, sometimes I don’t. Classloader hell somewhere I presume. This one still bites me a lot, and I don’t feel any closer to permanently solving it than I did on day 1, if anything I feel further from resolution, because of how much I’ve tried.

Original post below….

Not much information out there about this one.   What little there is will indicate some sort of class path or library conflict.  That may be true, but there’s other ways of getting here.  I finally managed to trigger it myself today, and now I know what causes it.  And I’m disgusted.

DISGUSTED

Now, admittedly, I’m having a hard time coming up with a test case for this, but it very much appears that this is happening when certain characters that are illegal in XML are returned via a webservice call.  FYI, I’m using JAX-WS wsdl first services, with JAX-WS service clients.  The _clients_ throw the error.

After eventually finding one of the problem calls, I found that the invoke was actually failing on the service side, and because ERICSSON ARE MORONS, the error message included ^c in the output. While this got sent back to the client without the service failing, the client exploded, with the delightfully helpful xml parser casting error.

So, JAX-WS RI, despite being billed as non-intrusive, and something that let’s you expose your existing services as webservices, is HIGHLY INTRUSIVE, and requires that _you_ take pains to make sure you _NEVER_ send out any invalid xml.  Even in you string fields, where you should be able to use what you want.  Because, you know, the library’s just exposing my service.

Still, there’s a fine tradition of this moronic behaviour.  XFire does the same thing.  [1] They flat out refused to accept that this was something the webservices library should be handling.

[1] You’ll get something like: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 3)) at line blah…

Spring JAX-WS clients, with less config

More with less. I got rather annoyed at some of the default messages that came out of the JaxWsPortProxyFactoryBean. Things like, “service name blah does not exist in the wsdl, did you mean wop?” Well, SURE! I told you the wsdl, there’s only one service, I mean THAT ONE!

So, I made my own port proxy factory….

Ugly old way

portname, serviceName and namespaceUri are all completely mechanical drudgery, but you have to get them exactly right.

    <bean id="oldStyleClient" class="org.springframework.remoting.jaxws.JaxWsPortProxyFactoryBean">
        <property name="serviceInterface" value="full.class.path.IService"/>
        <property name="wsdlDocumentUrl" value="http://server:port/webapp/services/BlahService?wsdl"/>
        <property name="endpointAddress" value="http://server:port/webapp/services/BlahService"/>
        <property name="lookupServiceOnStartup" value="false"/>
        <property name="portName" value="BlahServiceSOAP"/>
        <property name="serviceName" value="BlahService"/>
        <property name="namespaceUri" value="http://some.namespace/from/the/wsdl/"/>
    </bean>

Sexy new way

Sexy ways are always better ways.

    <bean id="potsClient" class="is.vf.common.jaxws.SimpleJaxWsFactory">
        <property name="serviceInterface" value="full.class.path.IService"/>
        <property name="localWsdlName" value="/BlahService.wsdl"/>
        <property name="endpointAddress" value="http://server:port/webapp/services/BlahService"/>
    </bean>

Hooray! So how? Just xpath into the wsdl! I used dom4j, because I’ve already got that in my app, but you could use anything really. I briefly tried using the xml apis in the jdk, but gave up very quickly.

Full Source below

package blah;
 
import org.dom4j.Document;
import org.dom4j.DocumentException;
import org.dom4j.Node;
import org.dom4j.io.SAXReader;
import org.springframework.remoting.jaxws.JaxWsPortProxyFactoryBean;
import java.net.URL;
 
/**
 * Set up a jaxws client with less effort.
 * Originally, I couldn't work out how to get spring/jaxws to reference a local wsdl file that was actually inside a jar.
 * And then, because I was damn sick of having to go and look up hardcoded strings from the wsdl just to copy into the config,
 * I decided to just read the damn wsdl and suck up the necessary blobs myself.
 *
 * No guarantees that this will work on all wsdls, but most people use the wsdl: prefix right?
 * Now, the only _required_ parameters are, localWsdlName and serviceInterface! whee. 
 * @author karlp
 *         Date: 8.2.2010
 */
public class SimpleJaxWsFactory extends JaxWsPortProxyFactoryBean {
    private String localWsdlName;
 
    /**
     * use dom4j to determine what's in the wsdl.
     * (I tried using the java built in methods, but a) it's ugly, and b) it didn't work)
     * Then just go back to the parent to load up the rest of the properties, now that we've done  the tedious ones...
     */
    @Override
    public void afterPropertiesSet() {
        if (localWsdlName != null) {
            URL url = getServiceInterface().getResource(localWsdlName);
            setWsdlDocumentUrl(url);
            Document doc;
            try {
                doc = new SAXReader().read(url);
            } catch (DocumentException e) {
                throw new IllegalStateException("Shouldn't be getting exceptions reading your local wsdl!" + e.getMessage(), e);
            }
            this.setServiceName(doc.selectSingleNode("//wsdl:service/@name").getStringValue());
            this.setPortName(doc.selectSingleNode("//wsdl:service/wsdl:port/@name").getStringValue());
            this.setNamespaceUri(doc.selectSingleNode("/wsdl:definitions/@targetNamespace").getStringValue());
        }
        super.afterPropertiesSet();
    }
 
    /**
     * This is the name of the wsdl, as seen in the jar containing the {@link serviceInterface}
     * @param localWsdlName the path to the wsdl in the containing jar
     */
    public void setLocalWsdlName(String localWsdlName) {
        this.localWsdlName = localWsdlName;
    }
}

xfire, inheritance and annotations, via spring

Yuck, xfire sure was about 1000% better than axis, but compared to modern jax-ws and contract first, it really can be just painful sometimes. So, I had some subclasses I wanted to expose over web services, but the directions for inheritance in xfire, either with Aegis or JaxB, both rely on you having horribly verbose spring configs.

We just use @WebService annotations, but we still needed a way to list these extra types that should be listed in the generated wsdl. (It’s worth noting that you get no errors if you don’t do this, but only fields from the parent class will be sent to web service clients)

The “fix” was committed in XFIRE-594 But as usual, the documentation got shotgunned all over somewhere. What you need for inherited types, when you’re using annotations is:

@WebService(endpointInterface = "acme.package.ISomeInterface")
@ServiceProperties(properties = {
    @ServiceProperty(key="writeXsiType", value="true"),
    @ServiceProperty(key="overrideTypesList", list = {
        "acme.package.SomeChildType",
        "acme.package.SomeOtherChildType"}) 
})
public class SomeService implements ISomeInterface
{
    // blah
    SomeParentType someMethod();
}

Then, if you’re just using the simple config from Spring remoting You’re done!

JaxWs, Spring, and referencing the wsdl in a jar, not via http

Well, this was awkward. Spring provides a quite handy factory class for creating JaxWs clients with JaxWsPortProxyFactoryBean

The config is quite simple really, you just do:

<bean id="partnerProxy" class="org.springframework.remoting.jaxws.JaxWsPortProxyFactoryBean">
        <property name="serviceInterface" value="is.vf.conan.ws.ConanPartnerPortType"/>
        <property name="wsdlDocumentUrl" value="http://localhost:8088/mockConanPartnerBinding?WSDL"/>
        <property name="namespaceUri" value="http://is.vf.conan/1.0/"/>
        <property name="serviceName" value="ConanPartnerService"/>
        <property name="portName" value="ConanPartnerPort"/>
</bean>

The “namespaceUri”, “serviceName” and “portName” are all boiler plate, that you have to extract from the wsdl, and shouldn’t really have to specify, but whatever, it’s boiler plate.  “serviceInterface” has to already exist.  You  have to have generated the types and interfaces before hand. But, if you’re doing wsdl first development, and you’re reading this, you already have all that. (Update 2010-07: If you don’t want to type in all that boiler plate (and who does), see: Spring JaxWS clients with less config)

Here in the office, we have a client jar, which contains the generated types and interfaces, and the wsdls themselves.  This is easy to distribute, and lets you add in support for a new service by simply dropping in a new client jar. (The jar is made with maven, using the jaxws-maven-plugin and it’s wsimport goal)

All well and good. Except. You can’t start this application unless that wsdl is up and available. You can add

        <property name="lookupServiceOnStartup" value="false"/>

Which is certainly a good start, but what if my endpoint doesn’t actually provide the wsdl at runtime? What if the ?wsdl url suffix doesn’t work here.

Well, we have two options:

  1. Have a server running all the time to provide the wsdl, and use the “endpoint” property to specify the final destination. (yuck)
  2. Have the proxy factory reference the WSDL inside that client jar.

You will see in some docs that you can do things like this:

<property name"wsdlDocumentUrl" value="classpath:blah.wsdl"/>

This works, if the wsdl can be found. If you have the wsdl in the same class structure as the rest of your code, this is probably all you need. But if your wsdl is inside a jar in the classpath, well, I tried lots of singing and dancing, but couldn’t work out how to make a url that pointed reliably to the wsdl inside the jar. :( This made me very sad.

I really don’t like having to override things, but well…. Here’s an extended spring factory bean. You now must use the endpoint property, plus the new property “localWsdlName” as well as “lookupServiceOnStartup = false” I was going to be using endpoint properties anyway, to make it nice and easy to switch out the different test environments we use, so that was not a big problem.

“localWsdlName” refers to the path inside the jar. It _should_ be able to be a “classpath:blah” reference as well, but it’s all relative to the jar that provides the service interface.

public class VodafoneJaxWsFactoryBean extends JaxWsPortProxyFactoryBean {
    private String localWsdlName;
 
    @Override
    public void afterPropertiesSet() {
        super.afterPropertiesSet();
        // This will be a looonnng file:// url, reaching into the jar 
        // holding the service interface
        URL url = getServiceInterface().getResource(localWsdlName);
        setWsdlDocumentUrl(url);
    }
 
    public void setLocalWsdlName(String localWsdlName) {
        this.localWsdlName = localWsdlName;
    }
}

And the new config you need to use this…

    <bean id="partnerProxy" class="is.vf.conan.VodafoneJaxWsFactoryBean">
        <property name="serviceInterface" value="is.vf.conan.ws.ConanPartnerPortType"/>
        <property name="endpointAddress" value="http://localhost:8088/mockConanPartnerBinding"/>
        <property name="localWsdlName" value="/conanPartner.wsdl"/>
        <property name="lookupServiceOnStartup" value="false"/>
        <property name="namespaceUri" value="http://is.vf.conan/1.0/"/>
        <property name="serviceName" value="ConanPartnerService"/>
        <property name="portName" value="ConanPartnerPort"/>
    </bean>

There you have it! If you have everything in one big blob project, you don’t need this, just specify the localWsdlName as “classpath:blah” and you’re done.