SHAC available memory reduces over time

Hi there,

I have a SHAC5500 and have noticed that the available memory reduces over time - It might start at 90Mb after reboot and reduce to say 60Mb after a couple of weeks. I have only a few groups being logged ON/OFF and NO trend logs. I have a number of resident scripts running but nothing below 2 second period. I had some trend logs in the past and removed them all but it has not helped. I've also noticed that the CPU/IO is sometimes above 1.0 but seems to reduce after a while. Can anyone suggest where to look for what might be causing these phenomena?

 

You could have a look at the system tab, then status, running processes this might give you an idea of what's going on.../ using the cpu,

in your resident script/s do you have any if statements that are setting the state of a group/object to some level? if these run every time the script runs sometimes this can create excess cbus traffic and put load on the CPU, depending on how fast the script is running and if it runs multiple times.
if the above is happening you would also expect to see lots of cbus traffic if you connected toolkit and had a look at the application Log etc

i would suggest its probably a script consuming the CPU - but it also may not be....
that's all i can think of at the moment hope this helps...

 

Running firmware 1.15.0.

I have a heap of script things always going on on my LSS5500NAC, and load rarely cross the 1.0 mark. It might peak at times to ~0.8 (and I know why), but the 15-minute is almost always well below 0.40. Memory is at 82MB free having not been restarted for ages, and I keep quite a few LUA tables in memory.

Try progresively disabling resident scripts to see if there is a load drop?

My running processes shows a few resident scripts.

 

thank you both and the suggestions make sense - I will work through over coming weeks and report back. I am now suspecting it is something do with inidividual scripts that I have "listening" to mqtt topics which are reloading each time they run but not releasing prior instances. my lack of mqtt and lua knowledge is most probably the root cause!

 

Here's how I do MQTT send/receive in a single resident plus a single event-based, @philthedill. This uses very little CPU. https://github.com/autoSteve/acMqtt.

The main loop is an infinite one, which receives packets on a socket (with short timeout) for changes to CBus levels (the event-based sends UDP messages to localhost:specificport), but the resident does not process these immediately, it adds them to an outbound queue. The loop then synchronously calls client:loop() for MQTT, again with a short timeout. This allows the MQTT client callback function to fire if anything is changed on the broker. The callback also does not actually process the message from the broker, instead adding it to a separate inbound queue. The main loop then calls functions to act on any queued messages inbound or outbound.

This allows short burts of many messages to be queued rapidly and the queue drained on each iteration of the loop. Because socket and client timeouts are so short the net result is high responsiveness, and low CPU use. (Turning a light on from HomeAssistant is near instant, and the same for sensor/level updates in HomeAssistant.) I drain each queue on each iteration of the main loop, but an approach of a slower drain could easily be done, like setting a priority for queued messges based on some measure, with high priority draining in a single iteration, leaving low priority messages to drain a single message on each iteration. I used to drain the queue progressively with this script, but found it wasn't needed, as inbound/output processing is very quick.

The script also has a lot more going on, with periodic detection of AC keyword changes, re-connection management, queued publishing of MQTT discovery topics, and a dead-man switch to have the script re-started should it lock up for any reason.

The CPU usually hovers at a 1-minute average at 0.08 - 0.2 when nothing else is going on and with all the other residents that do other things running at the same time.

 

wow - I am wondering now if I am using an early version of your work - v 1.23 which has a last update date of 4/11/22. Do you think I could be missing out for this reason. No matter what I disable, I still have this sort of situation although the available memory has improved.



I've most recently discovered that I cannot access the SHAC CNI so yet another problem! although one could be causing the other


it can see the project but not open it - I have tried an older version of toolkit 1.15 which I am pretty sure worked but am now wondering if my move to W11 might be the culprit.

In the meantime, I am moving as much stuff to HASS as I can (without attracting management attention). to reduce SHAC load

 

I have now installed the GitHub version of MQTT and MQTT send/receive which appears to be working fine. but I am still overloading the SHAC. I am loath to fiddle too much as I do not want to break it. Separately, I have 11 CBus blind controllers and have started to commission them on HASS using your program. However, the up button and down buttons will both move the blinds but the stop buttons don't seem to work - any ideas? I recall that the CBus blind controller STOP command is to send a level of 5 (2%). I have discovered that if I do not include COVER in the key words in the GA in CBus, I get good control although it behaves as a light!

 

Open an issue for the covers at https://github.com/autoSteve/acMqtt. I only have one blind, and don't tend to use up/down/stop. Mine is configured for presets open/closed/half open, so something a bit less blunt hammer might want coding...

November 2022, eh? Much commits since, including another bug fix today...

I run a LSS5500NAC, and there may be CPU differences to the SHAC. Not sure. One way to reduce load slightly when using my scripts is to set checkForChanges = false at the top of the script. Once you do so, any changes to keywords will require disable/enable of MQTT send receive script, and maybe also restart MQTT event-based, which is no biggie. You just need to remember. See if this has a positive impact on CPU use. I'd be fascinated.

 

BTW, for my blind I do not use "cover", @philthedill... "MQTT, select, sa=Bathroom 2, lvl=Closed/Half open/Open, ", so it would not surprise me if something were awry.

 

Hey @philthedill. I just committed a change that should get the L5501RBCP relays working for you in level translation mode. It's not perfect in terms of current blind status in HomeAssistant, especially where up/down/stop are used, but it should get you going for now.

It does up/down/stop, plus also setting a specific percentage open in HA.

 

Note that an additional commit has been added to deal with 1.15.0+ firmware, @philthedill. This adds an event script 'MQTT final' should you be running 1.15.0+ firmware. Plus event MQTT has been updated, as well as MQTT send receive. Currently these are experimental, but it works for me.

 

wow - thank you for the response - you have been busy! I will try the updates and report back - it might take a day or so. In the meantime, I have confirmed that SHAC load is much lower with the script not running. However, I've also noticed these errors being reported on start up. There appears to be line for each group I have marked with MQTT key word. nicer not to have it if possible but not a show stopper.

 

your instructions are excellent but I might have missed something. I have deployed the new <MQTT send receive> and get the following error. I am running 1.15.0 so assume I need <MQTT final> as well. What is not clear is to me is: does <MQTT final> replace <MQTT> or run along side it? what triggers it if it is the latter?



I just found this in the error log as well

 

I have modified the scripts further today, @philthedill, specifically for 1.15.0+. Latest commit was some time around 3-4pm AEDT, so after your post.

Use either event script 'MQTT', or 'MQTT final', with the latter preferable for 1.15.0+. The 'Not starting' message is a new thing. This is a check to see whether your event scripts are set up correctly. MQTT must be set to 'Execute during ramping', and MQTT final must NOT be set to execute during ramping. You will find this check box in the script properties (click the script name in the list, and the default is set to off when a new script is created). Exec during ramping was busted prior to 1.15.0. Well, event scripts ALWAYS executed during ramping regardless of this check box...

The errors in the log relate to the measurement application. I must have messed something up so will have a look at this now and report back.

 

Yeah, I am a dill, @philthedill. Added a commit to fix measurement app use with MQTT final.

(Also fixed the write to undeclared global warning.)

 

The messages about Panasonic are odd. I think that the only way this could occur is if a keyword 'AC' were tagged to those GAs along with 'MQTT', or something else very weird is up. What do your keywords look like for one example GA?

 

If your keywords look good, @philthedill, try a "blunt instrument" approach to suppressing the Panasonic messages by removing these three lines from MQTT send receive... (Assuming you don't use ESP Home devices.) I'd be interested if this drops your CPU. If it does, and your keywords are good then we need to talk more!

Code:

  { func = cudAc, init = true, script = 'AC' },
  { func = cudEnv, init = true },
  { func = cudAt, init = false, script = 'AT' },

 

Info from an ancient 2020 thread, go to Utilities tab, open C-Bus settings, then save. Does Toolkit then connect?

 

spot on - I had many items tagged with "AC" as they relate to my heating/cooling - I have changed them to HVAC and the "panasonic" problem has gone away

 

I think I had this set correctly beforehand - However, I have a new PC and had to start from scratch with Toolkit. perhaps just one of those things! It appears to be working now.