Here is how I plan to filter 10K nodes with Fancytree and still offer the user good performance on startup. Read More

Filtering 10K nodes with Fancytree

Here are my plans for managing a fancytree filter with 10K+ nodes.

I already run 1600 nodes, and it is growing fast to 10,000 or more.

For background, every node in the tree represents a different web page on a different subject.  Initially I only show a node, its parents and children.  Siblings of parents are hidden.  During search more of the tree is shown.  

How do I do this currently?  Well first I serve some html that looks like the tree.  Call it a pseudo tree.  This is also good for those without javascript.   If the user clicks on the pseudo tree before fancytree  loads,  it takes him to the web page for that node, where the process is repeated.  If he  has Javascript and if he waits a few seconds, the json tree loads, and the fancytree  is displayed and the search box is made visible.  That is okay for 1600 nodes, but when it goes to 10,000 nodes it will be too slow. 


So I want to take a different approach.  Instead of loading a json tree, I want to serve a list of json nodes with parent node  names.  For server performace, I cache that list on the server whenever  it is edited. 

When the page is first visited, I will serve the pseudo tree for browsers without Javascript.  I will also serve a tiny set of data.  Just this node, its parents, and the list of children for this node. That will display really quickly and give the user something to interact with.  If they click a child nodes, they can be lazy loaded.  If they click on a parent node, its children will be lazy loaded.  There will be a good user experience.  But what about filter?

  Meanwhile, in In the background I will load the complete list of nodes.  As each node is downloaded, it is created.   If a node has already been created, there will be  duplicates, so I have to test for that. 

So I will  serve the initially visible nodes, then load all the nodes in the background. If a filter happens, apply the filter to the loaded nodes, and to all the  new nodes as they are loaded.   

I think that this approach should allow me to serve and filter 10K nodes comfortably, without the user realizing that they are being downloaded in the background. 


Powered by Zopache, Grok, Zope and ZODB