Toolkit > Group Add = why is it so S-L-O-W ?

I have been meaning to post this problem for ages, but have waited until updating from Toolkit 1.1.9 to 1.2.1 (to see if it was just a bug).

I'm finding that adding new Groups is really really slow - painfully so It's quick on small projects, but once the projects get a bit large (bigger than a normal house) then adding each new Group is taking me forever !

Right now I am working offline on a job with 1 x Network, 1 x Application and currently there are 82 Groups defined. I need to add about 30-40 more Groups, before I assign them to output devices. After I hit "OK" to Group Add dialog, it takes (timing it right now)...
A) 8 secs then the "Groups" title in left pane disappears
B) 4 to 5 secs to refresh the Groups listings.
- and every new Group I add makes it slower and slower.

On another project with 2 Networks and about 200 groups on each, it often takes upwards of 10 MINUTES (a long cigarette break) just to add 1 Group... sometimes it works, most times it locks up the computer indefinitely so I have to kill the Task because there is no response ! On that project I am lucky because there are some of "unused groups" - so I've given up adding new ones and just "re-define" the unused groups (basically, because I CAN'T add new groups).

As you can imagine, this problem makes things very hard and frustratingly slow. Whilst the Group is being added I cannot do anything - I've found CPU Usage is flat out at 98-99% during the Group Add process. There doesn't appear to be any hard-disk activity.

The problem occurs irrespective of whether the Network is online or offline, and whether there is a PCI or CNI connected or not. There is no activity in Cgate while CPU is at 99%, then at the very end as the Groups List refreshes a "Tag Information Change" message appears in the Cgate Console. I doubt it's my computer - it's a Toshiba Laptop with Celeron 1.8GHz and 512MB of Ram. If Toolkit needs more grunt that that just to write a couple of lines into a *.XML file, then there's a MAJOR programming issue

There are no other applications running - normal Idle Processes is around 96-98%. Rebooting doesn't help. What is annoying is that irrespective of the Project size, changing a TagName is virtually instantaneous. Yet editing a tag is doing basically the same thing as adding a new one... it stilll has to do all the checks, update the Groups Lists and write out to the *.xml file.

The ONLY thing I can think of that is even slightly odd is the length of the TagNames I'm using. On the larger projects I need to use most or all of the available characters - for example, here's the group I am currently adding :
Trk SecF Rw4 C1 - Used nr B/head
- but since that's only written to the local database, I don't believe that's causing the CPU usage.

For info - this comp has been used for Cbus for about 3 years, so it's had various V2 then about every 2nd Toolkit release since 1.1.4. installed in it as upgrades. But it's never slow, I do all manner of graphics and database work on it including very intensive SQL queries, etc and I've never seen it falter.

So - is this Toolkit slowness a known issue? Is there a way for me to minimise / fix it? Is there something I can give you to help me resolve it? I have no way of capturing *what* exactly the CPU is doing while it's at 99% Usage...

Help please - I'll never complete these projects the way I am going at the moment !

Thanks, John

 

John,

No known problems.. will dig around and see what we can find.

 

Thanks for the fast replies, guys !

Exactly the same delay, whether I use the Groups List "Add Group" Toolbar Button, or the + button on a Unit Dialog. On the project I am working on right now (80-odd groups), after clicking OK on the Unit's + Dialog it takes 14 seconds for the Group to be assigned to that unit / channel.

I just tested in PICED and a new group is added (to exactly the same project file) almost instantaneously - so it MUST be a Toolkit issue. Also, the length of TagName makes no difference. Finally, it takes a similar (LONG) time to delete a group too - I just did that to delete the test groups I just added.

Both projects I refer to above were originally created in V2, then imported into Toolkit (about v1.1.5). I am extensively modifying them, due to additional product being installed.

I have not tried creating a new Toolkit v1.2x Project with 80+ Groups, but I have definitely noticed problems on a Toolkit 1.1.9 project that has 92 groups. Just tested that one and found that it takes 19 seconds to Add a Group, and 10 secs to delete one.

That same delay was definitely evident in v1.1.9, as I was using the "Group Add" delay as a cigarette break. I nearly died of smoke inhalation that day while I was on-site

Sure - but I am unsure if this is Project related. PM me with an email address and I'll send you one to test.

Cheers, John

PS: I am leaving this company and laptop in a few weeks, and getting a Clipsal standard-issue Dell in my new position. But before I leave I have to complete a heap of programming - If it's just the Toolkit install on this laptop that is screwed up, maybe I could just un-install the whole Toolkit and do a clean install (after Project File backups) ?

 

Johnc
I get the same thing takes up to 12 seconds to add or delete a group.
So I use Piced to add new groups and like u say comes back straight away, only toolkit that is slow
dave

 

Thanks Dave.. curiouser and curiouser!

 

I wasn't going to bother replying because I don't have specific version numbers etc at hand, but since the claim "no known issues" is out there....

When I started building the project for my new place, I was experiencing exactly the same issue - up to a couple of minutes (was initially almost instant, becomming a few seconds, then tens of seconds...) to either create or delete a group address.

The only things I can add are that it was toolkit (piced wasn't out then), and that it was off-line (the equipment hadn't all been installed then, I was programming at home to get a jump on it).

The machine wasn't anything spectacular - it was my home cinema system - but it was quite fast enough for everything else. Had plenty of RAM available, and no shortage of disk space.

The problem just "went away" for no apparant reason, I can't recall exactly what the trigger was. I THINK I just blew away the project and started again in frustration - indeed, I very nearly pulled out and discarded the whole cbus installation believing this was "typical" performance - and I wasn't even half way through adding GAs at that stage.

RossW

 

We're glad you persisted. Hopefully we'll have a few projects that demonstrate the problem today, if we can roll a fix into the next release we certainly will.

 

More info, for what it's worth...

I was looking at that same project again before emailing it to Duncan, and found that there were 92 Groups but 31 of them are un-used by any devices. So I started deleting them... 91 = slow... 90=seems quicker... 89=definely faster... The final quantity is only 61 Groups, and comparing the time it takes (on exactly the same project) :

Code:

No. Groups    Add a Group    Del a Group
    92          19 secs        10 secs
    61           6 secs         3 secs

So, it is DEFINITELY related to the number of groups on the project ! At 50 Groups it's not too bad, at 60 it's very noticeable, at 90 it's incredibly frustrating, any more and it's un-manageable !

I just checked another big project which often completely locks up Toolkit when I add groups....

Code:

Network   No.Units   No. Groups    Add a Group    Del a Group
  253        46        209 *         
  254       108        209 *         > 4 mins       30 secs
* Group Names are identical (shared) on both Networks for some reason

I am thinking that perhaps it has something to do with the fact that Toolkit is continuously writing a NEW *.xml for every single change (irrespective of how small), whilst PICED is storing changes in a cached copy in memory, and only writing it out when you hit File > Save ?

Perhaps Toolkit is getting hung up storing the change, comparing from memory to determine what must be written out, renaming the existing *.xml to a *.xml.old, creating a new *.xml, writing out a new *.xml - but the second stage is also slow : reading the contents back into memory...

I am sure that if CIS can reproduce the issue, it will be trivial to work out what process is causing the huge delays on larger projects.

Let's hope they can help !

John

 

OK.. have recreated the problem... if your Groups Node is Expanded in the treeview.. ie you can see a list of groups in the tree for an application then adding groups is indeed VERRRRY SLOOOW..

Thankfully, by collapsing the Groups Node the Group Add time is near immediate..

We will try to roll a fix into our imminent 1.3 release.. depends on how involved the problem is..

Thanks for bringing this to our attention John.

 

Amazingly quick response, thanks Darren - and I never even thought to try that !

Well, at least we have a work-around . . . just have to remember it

***********************

Wierd that the Tree is taking so much resourses, but I did have a similar problem in a Perl script I wrote years ago. I was generating a Tree View (of a directory structure of a server) using a recursive loop. Every time the loop found a new "node" it would add that path to the loop, and store all the sub-items in a array of arrays.

Once all the main nodes were found, it'd then trawl thru 2nd level of each, then 3rd levels, etc. What happened to me was that if there were a lot of nodes, there was a huge number of arrays of arrays, each containing vars of the file contents... it was not scaleable, because I was holding the whole directory structure in memory while the Tree was "created".

Whilst this meant that the response was instant "after the tree was built" (as it was just read from memory), I had to change it and make it do each branch / node separately and only upon "request". That made changing nodes a tiny bit slower, but freed up Megabytes of memory usage and massive amounts of CPU time to create the array in the first place (and each time the tree structure changed).

Is that a similar problem to what is causing this?

Cheers, John

 

Darren.. Duncan, we always get mixed up... I'm the good loooking one, just remember that.. its easy

The problem with the Tree is related to a great swathe of code and classes we have managing the drawing of nodes, the provision of Toolbar choices etc.. Its caused a few problems before and I was hopeful we'd nailed it.. I guess we'll be opening it back up again

 

Oops, sorry about that... Oh, the good looking one - so that mustn't be you standing next to the Bedford pics in your blog ?

EEEK ! KISS is the key to success in solving stuff like this, but that's a hard one to solve while keeping the Tree View as it is.

Personally I hate it, would much prefer just opening ONE project (like in PICED) and working on it. Is there any reason whatsoever to have all the projects listed like that? A lot of the time the Tree gets in the way and takes valuable screen space. It's a pity that it makes things look simpler, but actually makes it harder to use...

I'd much prefer a toolbar with "Groups", "Applications", "Units" etc as buttons. For example, since the only thing under Applications > Lighting is "Application Log" and "Groups", there's no need for that to be there... just call up the Applications Button, Choose "Lighting" on a dialog, then display the Groups List. That'd do it all in one pane - delete the whole bloody Tree View and make it a easier application to code and to use at the same time !

But a change of interface is a BIG revision...

***************

OK - here's a serious suggestion to fix the problem...

After a Group is added, I assume you make a call to refresh the Tree - and that refresh is what is killing the CPU. How about using that same call to CLOSE the Groups Tree Node instead of refreshing it, but leave the Groups List open in the Right Pane?

Most of the time we (users) click the + on the Groups node, and leave it open while we work in the right pane. It's almost opened by default, but doesn't ever really *need* to have the Groups Node open !

I say that because the only reason to ever HAVE the Groups List open is to see/set the Levels under each Group. I don't know what "Levels" are (I assume some kind of preset)... but I'd reckon that 99% of people don't need them. And if they did want them, make them available from a button on the toolbar in the RIGHT pane rather than the Tree.

What i described above (if it makes sense) would prune the who Groups Node, and make it work like the Units node which is only 1 level deep, and after clicking on that you OPEN the Unit to work on it. In the same way, why not simply "open the Group" to work on it ?

Hmmm - I am not very good at explaining myself with this

John

EDIT - what I mean above is... take a look at Windows Explorer. The Left pane never shows any "details" (files), it only shows "containers of things" (folders). Toolkit's listing of every single Group in the left pane is the equivalent of listing every Folder's file listing in the left pane of Explorer. There is no need for that level of detail to be in the left pane, as the same info is already shown in the right pane (which is why it's there).

 

Thanks John, I'll bear all that in mind when we fix the prob.. thanks for taking the time.

 

Problem with the DELL

Gday John , Do a bit of research on the DELL Computers. Im Not 100% sure but i think there is a known problem with the DELL having a LOW comms port voltage that causes problems when trying to programme NIRTs .Someone may be able to Confirm or Deny.Id spew if i got a new computer and no body told me.

 

Duncan
Can confirm its the tree - played with toolkit today and found when tree closed all ok
ure on the right track

dave

 

I don't think I have a choice, it's a Clipsal standard-issue Dell, as that's were I'm moving jobs to
- If it doesn't work with Cbus, I'll just get the in-house tech-heads to fix it

 

I can confirm that my Dell Latitude D810 internal com port can't program a NIRT I have to use a Quatech PCMCIA Com Port. I don't expect this to be an issue in the future however.

 

Thanks Dave, appreciate the confirmation.