Integration Tests on ICN

After my few posts on Unit Testing, I would like to introduce how to do Integration Testing on ICN. Integrations Testing means interaction with other systems. in our case, it means FileNet. However for the IT, we don’t want to rely on the ICN server, because it would require user interaction, and this is the purpose of the UI test (see this post for more information).

The purpose of the Integration Tests is to call every services we wrote in our plug-in, directly and check changes made to the FileNet repository or other repositories are what we expect them to be.

When you implement a new service in a ICN plug-in, the main method called by the ICN engine when you are invoking the service from the client browser looks like this:

public void execute(PluginServiceCallbacks callbacks, HttpServletRequest request, HttpServletResponse response) throws Exception;

That’s the method we want to call so we can run our service on our FileNet platform. In order to do that, we will have to mock the callbacks, request and response object since obviously we are not in a server environment. We will connect to the repository and inject it into the mocked callback so the service can use it as it was in ICN.

The test code should look like this:

public class MyServiceTests {

    @Rule
    public PowerMockRule rule = new PowerMockRule();

    private String repositoryId = "TARGETOS";
    private String repositoryType = "p8";
    private MyService service = new MyService();
    @Mock PluginLogger logger;
    @Mock private HttpServletRequest request;
    @Mock private HttpServletResponse response;
    private JSONObject config = new JSONObject();
    private JSONArray items = new JSONArray();
    MockServletOutputStream outputStream = new MockServletOutputStream();
    // Don't use PluginServiceCallbacks as type because it will load the class before the ByteCodeModifier has a change to modify it
    //in the @BeforeClass method and it will throw an exception
    private Object callbacks;

    @Before
    public void setUp() throws Exception {

        // Connect to your repository here to inject it in the mocked callback object

        Connection conn = Factory.Connection.getConnection("http://host:9080/wsi/FNCEWS40MTOM");
        String stanza = "FileNetP8WSI";

        Subject subject = UserContext.createSubject(conn, "P8Admin", "MyPassword", stanza);
        UserContext.get().pushSubject(subject);
        Domain domain = Factory.Domain.getInstance(conn, null);
        // Get an object store
        ObjectStore os = Factory.ObjectStore.fetchInstance(domain, platform.getObjectStore(), null);

        // Settings the config
        config.put(Constants.CONFIG_MAX_NB_DOCUMENTS, (long) 500);

        PluginServiceCallbacks callback = mock(PluginServiceCallbacks.class);
        when(callback.getLogger()).thenReturn(logger);
        when(callback.getP8ObjectStore(repositoryId)).thenReturn(os);
        when(callback.getP8Domain(Matchers.eq(repositoryId), Matchers.any(PropertyFilter.class))).thenReturn(domain);
        when(callback.loadConfiguration()).thenReturn(config.toString());
        when(callback.getP8Subject(repositoryId)).thenReturn(subject);
        this.callbacks = callback;

        // Prepare the request
        when(request.getParameter(Constants.PARAM_REPOSITORY)).thenReturn(repositoryId);
        when(request.getParameter(Constants.PARAM_SERVER_TYPE)).thenReturn(repositoryType);

        // Prepare the response
        when(response.getOutputStream()).thenReturn(outputStream);
    }

    @After
    public void after() throws Exception {
        UserContext.get().popSubject();
    }

    /**
     * Launch a test
     *
     * @throws Exception
     */
    @Test
    public void testServiceNormalUseCase() throws Exception {

        // Here you use the Java API to create folder and doc to set your test case up

        // Add the item you want to call the service on, or any iformation your service need as service parameters
        when(request.getParameter(Constants.PARAM_ITEM)).thenReturn(folderYouCreated.get_Id().toString());

        // Then call your service here, as ICN would be calling it. At this point t should havec
        // any information it needs it the request and in the configuration JSON object
        service.execute((PluginServiceCallbacks) callbacks, request, response);

        // Verification
        // Here you use the Java API to check if the service did what it is supposed to do with the data you gave him
        // For example let's say MyService is supposed to modify a bunch of properties on the folder we pass as param
        // Then we will check all the properties here

        // Don't forget to clean everything and delete everything you've created for or within the test to leave a clean
        // repository for other tests
    }
}

Well this is quite easy using Mockito and PowerMock, however you might face an issue when trying to mock the PluginServiceCallbacks object, because there is some dependencies we don’t have since this plug-in is supposed to run in a ICN environment. You will most certainly get this error:

java.lang.NoClassDefFoundError: com.ibm.ecm.util.DocumentContent
    at java.lang.J9VMInternals.verifyImpl(Native Method)
    at java.lang.J9VMInternals.verify(J9VMInternals.java:73)
    at java.lang.J9VMInternals.prepare(J9VMInternals.java:459)
    at java.lang.Class.getDeclaredConstructors(Class.java:535)
    at org.mockito.internal.creation.jmock.ClassImposterizer.setConstructorsAccessible(ClassImposterizer.java:75)
    at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:70)
    at org.powermock.api.mockito.internal.mockcreation.MockCreator.createMethodInvocationControl(MockCreator.java:111)
    at org.powermock.api.mockito.internal.mockcreation.MockCreator.mock(MockCreator.java:60)
    at org.powermock.api.mockito.PowerMockito.mock(PowerMockito.java:203)
    at org.powermock.api.extension.listener.AnnotationEnabler.standardInject(AnnotationEnabler.java:106)
    at org.powermock.api.extension.listener.AnnotationEnabler.beforeTestMethod(AnnotationEnabler.java:54)
    at org.powermock.tests.utils.impl.PowerMockTestNotifierImpl.notifyBeforeTestMethod(PowerMockTestNotifierImpl.java:90)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:292)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:127)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:82)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282)
    at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:86)
    at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:49)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:207)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120)
    at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:33)
    at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:45)
    at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122)
    at org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:104)
    at org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53)
    at org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:53)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.ClassNotFoundException: com.ibm.ecm.util.DocumentContent
    at java.net.URLClassLoader.findClass(URLClassLoader.java:434)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:677)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:358)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:643)
    at org.powermock.core.classloader.MockClassLoader.loadModifiedClass(MockClassLoader.java:178)
    at org.powermock.core.classloader.DeferSupportingClassLoader.loadClass(DeferSupportingClassLoader.java:68)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:643)
    ... 33 more

This is quite a complex problem, to make it easy, here is a class modifying the ByteCode of the class we want to mock to remove every method we don’t need and cause this exception. At the end you won’t have much methods left, but you will have all the ones you need, like getPluginLogger, loadConfiguration and so on. below is the class.

Warning: When using this class with java 7 and the PowerMock agent, you will have to lower the bytecode verification engine with -Xverification:none for IBM JRE, and -XX:-UseSplitVerifier for SUN JRE. This this post.

package com.ibm.idwb.common.filenet.ittesttools.bytecode;

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;

import javassist.CannotCompileException;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.NotFoundException;
import javassist.bytecode.AccessFlag;
import javassist.bytecode.ClassFile;
import javassist.bytecode.MethodInfo;

/**
 * <p>
 * This class is meant to modify bytecode for some class with compilation error when hot compilation with java assist,
 * preventing them to be mocked by Mockito or PowerMockito.
 * </p>
 * <p>
 * At the time the only class known with this problem is PluginServiceCallbacks from ICN.
 * </p>
 * <p>
 * <strong>Works only with PowerMockito 1.5.6, do not use yet with 1.6.0</strong>
 * </p>
 *
 * @author G. DELORY
 *
 */
public class ByteCodeModifier {

    private static List<String> alreadyModified = new ArrayList<String>();

    /**
     * Edit the bytecode of the class PluginServiceCallbacks from ICN so it can be mocked by Mockito. This prunes all
     * deprecated methods, non public methods, and methods related to other repository than P8. It also removes a few
     * methods with missing dependencies.
     *
     * <p>
     * <strong>This method must be called before the class PluginServiceCallbacks is loaded into the ClassLoader,
     * including any reference fron other class (Field, Method type...) or it will fail!</strong>
     * </p>
     *
     * @throws NotFoundException
     * @throws CannotCompileException
     */
    public synchronized void prunePluginServiceCallbacks() throws NotFoundException, CannotCompileException {
        String name = "com.ibm.ecm.extension.PluginServiceCallbacks";

        if (!alreadyModified.contains(name)) {
            alreadyModified.add(name);
            String[] byName = new String[] { "^getCMIS.*", "retrieveDocumentContent", "getCMPrivilegeMasks" };
            String[] bySignature = new String[] { ".*org/apache/struts.*", ".*org/apache/chemistry.*", ".*filenet/vw.*", ".*com/ibm/mm/sdk/.*",
                    ".*com/ibm/edms/od/.*", ".*com/ibm/ecm/mediator/od/.*" };

            pruneClass(name, byName, bySignature);

        }

    }

    /**
     * Allow to remove methods from the bytecode of a class, then recompile the class and load into the ClassLoader
     *
     * @param className the class to edit
     * @param byName array of pattern like string of method to remove
     * @param bySignature array of pattern like string of method with a specific signature to remove
     * @throws NotFoundException
     * @throws CannotCompileException
     */
    @SuppressWarnings({ "rawtypes", "unchecked" })
    private void pruneClass(String className, String[] byName, String[] bySignature) throws NotFoundException, CannotCompileException {
        ClassPool cp = ClassPool.getDefault();
        CtClass cc = cp.get(className);
        ClassFile cf = cc.getClassFile();

        List<MethodInfo> methods = cf.getMethods();
        for (Iterator iterator = methods.iterator(); iterator.hasNext();) {

            MethodInfo m = (MethodInfo) iterator.next();

            if (isDeprecated(m) || deleteBySignature(m, buildPatterns(bySignature)) || !isPublic(m) || deleteByName(m, buildPatterns(byName))) {
                deleteMethod(iterator);
                continue;
            }
        }

        cc.toClass();
    }

    private boolean deleteBySignature(MethodInfo method, List<Pattern> signToExclude) {
        for (Pattern pattern : signToExclude) {
            if (pattern.matcher(method.getDescriptor()).matches()) {
                return true;
            }
        }
        return false;
    }

    private boolean deleteByName(MethodInfo method, List<Pattern> nameToExclude) {
        for (Pattern pattern : nameToExclude) {
            if (pattern.matcher(method.getName()).matches()) {
                return true;
            }
        }
        return false;
    }

    private boolean isPublic(MethodInfo method) {
        return AccessFlag.isPublic(method.getAccessFlags());
    }

    private List<Pattern> buildPatterns(String[] patternsAsString) {
        List<Pattern> patterns = new ArrayList<Pattern>();
        for (int i = 0; i < patternsAsString.length; i++) {
            try {
                patterns.add(Pattern.compile(patternsAsString[i]));
            } catch (PatternSyntaxException e) {
                System.err.println("Cannot compute " + patternsAsString[i] + ". This pattern will not be used");
            }
        }
        return patterns;
    }

    private boolean isDeprecated(MethodInfo method) {
        return method.getAttribute("Deprecated") != null;
    }

    private void deleteMethod(Iterator<MethodInfo> it) {
        it.remove();
    }

}

Then you just have to call prunePluginServiceCallbacks method before making any reference to the class, because it must be the first time the class is loaded. That’s why we used the type Object for the field callbacks instead of PluginServiceCallbacks, or it sould have loaded the class before we had a chance to edit the ByteCode and the modification of the ByteCode would have fail. Here an example of how to use it:

@BeforeClass
public static void beforeClass() throws NotFoundException, CannotCompileException {
   ByteCodeModifier byteCodeModifier = new ByteCodeModifier();
   byteCodeModifier.prunePluginServiceCallbacks();
}

 

Leave a Reply