Delay needed after layout_network in some cases

[nilsoberg]

I have identified an issue when using the py4cytoscape API. I am creating thousands of images using Cytoscape, and the basic procedure is to spin up an instance of Cytoscape, use py4cytoscape to load a network, layout, and export. Each instance of Cytoscape is reused a number of times (currently 20 times, but I’ve experimented with fewer times – 5 to be exact – to try to address the issue I am reporting) and then closed.

Some images that are exported are not correct in the sense that they are cropped, edges go into space without connecting to nodes, or the entire network is rendered as a single node. You can see this at the first link I provide (you will need to scroll down the page to see the issues). There doesn’t seem to be an error in the Cytoscape or py4cytoscape logs, and the error is not reproducible on the same file – it is random.

https://efi.igb.illinois.edu/downloads/superfamilies/status/ip95/old/cluster-1-3.html

I experimented with adding a delay after various API calls, and I identified a delay that seems to fix the issue. If I insert a delay of 10 seconds after the layout_network call (using "force-directed") the problem seems to be "fixed". I am reporting this because I wonder if there is a better fix than adding a delay. This is an acceptable solution for my needs, but if there is a better way I would like to use it. Here is the identical set of networks rendered using the "fix":

https://efi.igb.illinois.edu/downloads/superfamilies/status/ip95/cluster-l-3_20uses.html

I think this may be related to #91 and #92.

[bdemchak]

Thanks, Nils --

Regarding recycling Cytoscape instances, I assume you're referring to Cytoscape becoming very slow after long use. If so, we have investigated that and solved some memory issues. They will be addressed in Cytoscape v3.10.2, and are already partially implemented there. The developer will notify me when I can offer a preview.

Regarding delay being necessary after some layouts, the developers are looking into that but are unlikely to address it before Cytoscape v3.11. If we can address it at the py4cytoscape/RCy3 level, we'll do it, but I wouldn't expect this to be so. Bottom line: for now, the delays are the best workaround.

Have I missed anything?

[nilsoberg]

You have addressed everything, thank you. I am currently using v3.10.2 with success--I was poking around the nightly build site. I even noticed there's a v3.11. I don't remember if I tried it. Thank you!

[bdemchak]

Hi, Nils ... 3.10.2 is the best release for this. It seems solid and solves a number of problems. I'll let you know when a new version is posted and it applies to your situation.

Yes, 3.11 is under development and may be used as a preview. Builds have been largely stable for a while, now. No promises, though.

[bdemchak]

Hi, Nils ... The developers tell me that they found a memory leak in the Groups features, and identified an internal event log that grows forever. Both will be addressed in 3.11. Those fixes should help a lot with Cytoscape slowing down over time. Combined with the memory solutions in 3.10.2, Cytoscape performance should be much more stable over the long haul. Feel free to try 3.11 if you like.

[nilsoberg]

Thanks. I will try that in a new iteration of my image generation. All of this sounds very promising and I am happy.

Without coming across as asking for the moon, do you have any idea why the nodes don't get colored in the very largest clusters? If you look at https://efi.igb.illinois.edu/downloads/superfamilies/status/ip95/cluster-1-1_grouped/cluster-1-1_AS45.html you will see that the first 20 clusters or so (minus 3) use the default Cytoscape color. They take the shape style and edge color that is applied by the code, though. The clusters are ordered by the number of nodes in descending order. (When I say "large clusters" it very likely means clusters with many edges.)

[bdemchak]

Hi, Nils ... I see your point. Can I suppose that all clusters were generated by (essentially) the same code? ... and that none of them set the node color?

It seems like the colors are random after a certain point (~20 graphs, but before that, too).

Is it possible for you to get this effect via the Cytoscape GUI? ... or alternately, is it possible for you to save off a session (.cys) containing one of the mis-colored graphs? I'd like to look more closely at the session.