Until now I thought that Page Size was really useful for queries only. As often, I was wrong… Of course it matters a lot for queries, but it actually matters when working on one object too. I discovered this when I was writing my other post on the Property Filter and did some benchmarks.

I noticed that every 500 items I had a small pause in my execution, even with using properly the Property Filters. So my dream of one light call, hoping to get my all 5000 subfolders in one light call and then iterate super fast on it wasn’t completely true.

Page Size effect on object fetch

Actually the Page Size in Property Filter also applies to multi-valued properties, and therefore even if you have nice property filters making sure the answer is light, it won’t fetch more than Page Size items from the server. And when you will iterate on all your items, you will have some pause in the middle, when FileNet is fetching the next page.

It is good if you need a lot on properties, to avoid OutOfMemory exceptions, but if you know you are retrieving only few information, like in the example of my other post only the name of the folder, you can increase a lot this value. However you will be stuck at 1,000 which is the default maximum value for the page size configured in FileNet P8 Domain. If you use more in your Property Filter, this value will be used.

Let’s compare effect of page size on performance.

Two different tests have been made here:

• Display all 5,000 subfolders names of a folder
• A recursive display folder name where folder have all 4 subfolders, on 7 levels (which makes, if I can count (this is a sum of consecutive powers 4⁷ + 4⁶ + 4⁵ … + 4 + 1), 21845 folders)

All test have been made 11 times, and average time made on 10 last executions. Properties filter are set to fetch everything in one called, the only limit preventing to fetch everythins is the page size argument.

Here are the snippets.

int pageSize = 5000;
PropertyFilter pf = new PropertyFilter();
pf.addIncludeProperty(new FilterElement(1, null, null, PropertyNames.SUB_FOLDERS, pageSize));
pf.addIncludeProperty(new FilterElement(1, null, null, PropertyNames.FOLDER_NAME, pageSize));
Folder f = Factory.Folder.fetchInstance(os, "/MyFolder", pf);
FolderSet sfs = f.get_SubFolders();
Iterator<Folder> it = sfs.iterator();
Folder sf;
while (it.hasNext()) {
    sf = it.next();
    System.out.println(sf.get_FolderName());
}

public void displayTree(String folder){
    int pageSize = 5;
    PropertyFilter pf = new PropertyFilter();
    pf.addIncludeProperty(new FilterElement(99, null, null, PropertyNames.SUB_FOLDERS, pageSize));
    pf.addIncludeProperty(new FilterElement(99, null, null, PropertyNames.FOLDER_NAME, pageSize));
    Folder f = Factory.Folder.fetchInstance(os, folder, pf);
    displayTree(f, 0, pf);
}

public void displayTree(Folder folder, int currentLevel, PropertyFilter pf){
    System.out.println(ident(currentLevel) + folder.get_FolderName());
    FolderSet subfolders = folder.get_SubFolders();
    Iterator<Folder> iterator = subfolders.iterator();
    while (iterator.hasNext()) {
        Folder sf = iterator.next();
        displayTree(sf, currentLevel + 1, pf);
    }
}

And here are the results of the benchmarks:

We can see how page size parameter affects performance. Even on LAN. With 5000, only on call is made to the server, with 1000, 5 calls and so on until 1 where 5000 round trips are made. Please note that we had to increase the MaxQueryPageSize of the domain to use 5000, but performances are really impressive.

I excluded on purpose the page size over the Internet because its value (584.6 seconds) makes the chart hard to read. But this is 20x more than with a page size of 10, of course no one would use a page size of 1.

Again I skipped the page size 1 over the Internet because it was too long. Based on the LAN value it should be around 1500 seconds.

We can see that here page size of 5 or 5000 makes no difference. It makes sense since we have only 4 subfolders, and the page size is for each level, not in total including sublevels. That means that with a page size of 5, we will actually retrieve all 21845 folders in one round trip. That demonstrates that recursion is as important as page size, and as property filters. All three need to be carefully thought when planning a fetch from the server, especially a big one. With 1 we actually do 21845 round trip to the server!

The result with 4 is also interesting, because I would have thought that it will fetch everything in on round trip, and I did check and there really is only one page of subfolder so there is no need for a round trip. It might be because when the page size is exactly the same size than the numbers of items (or a multiple), FileNet has to do a last round trip to check if there is more. But this is just a guess.

How to change the default maximum page size value for the Domain

Via the ACCE

This is to be used with caution, because this value is used to protect the system from OutOfMemory error or huge Query. If you do decide to change this value, you have to be sure all developers using this value will choose a page size according to what they are retrieving. In our case we will work on a value retrieving only Fole Name for all our subfolders, which is not a lot of information, so we can increase this value. But if someone set a huge page size value while retrieving Object with a lot of information, you could have problem. This is the procedure anyway.

In the Administrative Console for Content Engine, click on your Domain then Properties tab. Then scroll to Subsystem Configuration property, and in the list, click on 6) Server Cache Configuration:

It will open the object, scroll to Query Page Max Size and enter your new value.

Then Go back to the domain tab and save (You cannot save the Server Cache Configuration by itself since it is a dependent object (see Knowledge center on dependent object if you are not familiar with this concept).

This will affect the QueryPageMaxSize of the ServerCacheConfiguration object, and you will be able to use a higher page size value in your property filter.

Again, be careful with increasing this value, if not all developers working on the platform are familiar with the property filter concept and are not extra careful with what quantity of data is retrieved when retrieving objects (sometimes you can think because you fetch only one object it does not need to be evaluated)

Via Java API

You can change this value programmatically if you have sufficient rights. You can set it back after a query if you have really specific needs for just one highly optimized query.

SubsystemConfigurationList ssCL = os.get_Domain().get_SubsystemConfigurations();
Iterator<SubsystemConfiguration> it = ssCL.iterator();
while (it.hasNext()) {
    SubsystemConfiguration ssC = it.next();
    if (ssC instanceof ServerCacheConfiguration) {
        Properties properties = ssC.getProperties();
        Property p = properties.get(PropertyNames.QUERY_PAGE_MAX_SIZE);
        p.setObjectValue(value);
        os.get_Domain().save(RefreshMode.REFRESH);
        System.out.println(PropertyNames.QUERY_PAGE_MAX_SIZE + " set to " + value);
    }
}