SHAC available memory reduces over time

Discussion in 'C-Bus Automation Controllers' started by philthedill, Dec 18, 2023.

  1. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    given this was a keyword issue, I have not done anything further with this
     
    philthedill, Jan 10, 2024
    #21
  2. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    Thank you @ssaunders we have made great progress.
    I have deployed the new "MQTT send receive" and the "MQTT final" (with my firmware of 1.15.0). Everything appears to be working as it should.
    out of interest how does the "MQTT final" event script get triggered by the "MQTT" keyword?
    I have now created a separate user name for the SHAC to connect with the MQTT broker (in my HASS box) in the hope that it would reduce to processor load but it has not - so that issue remains.
     
    philthedill, Jan 10, 2024
    #22
  3. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    In script properties. Event-based scripts can be associated with a single GA, or a bunch of GAs by using keyword. If execute during ramping is not set, then the script will only fire at the completion of any ramps.

    upload_2024-1-10_15-54-35.png

    Glad you found the AC keyword issue. I guess one would need to read the entire readme to even know that the MQTT send receive script looks at keywords 'MQTT', 'AC', and 'AT'. I might add a note near the top! Many would probably glaze over once they hit the Philips Hue section...

    For your CPU issue I'd be happy to have a poke around. Will PM.
     
    ssaunders, Jan 10, 2024
    #23
  4. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
     
    philthedill, Jan 10, 2024
    #24
  5. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    I've discovered this going on since restarting - might have been there before but seems to be chewing through memory and might be cpu user too....
    upload_2024-1-10_17-48-5.png
     
    philthedill, Jan 10, 2024
    #25
  6. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    ssaunders, Jan 10, 2024
    #26
  7. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    it is blank
     
    philthedill, Jan 10, 2024
    #27
  8. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    If there is no error being thrown, then I've no clue why you'd get 'Heartbeat initialised' spammed in the log, @philthedill. The script is restarting for some reason, and because resident/zero sleep it is restarting a lot.

    Most odd. Right after that log message it enters an infinite loop, checking for heartbeat messages with a one second timeout, then rinse/repeat. It should never exit. But it clearly does.

    Step 1: Change it to sleep 60 in the script parameters
    Step 2: If it spams the log every minute instead, disable 'Heartbeat' for now, and we'll need to dig further.
    Step 3: If it stops spamming the log, disable it temporarily and see if your CPU drops.
     
    ssaunders, Jan 10, 2024
    #28
  9. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    I think we are onto something - sleep 60 has the cpui/o down to .28 and falling - not seen for a long time!
    However, it is still spamming once/minute and sending the usage up
    I've now disabled heartbeat and seeing this upload_2024-1-10_18-49-18.png
     
    philthedill, Jan 10, 2024
    #29
  10. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    That looks to be absolutely brilliant, @philthedill, and what I would expect to see. Better than mine.

    As to what's up with Heartbeat... who the heck knows?! It runs like a rock on my NAC. Latest code version installed?
     
    ssaunders, Jan 10, 2024
    #30
  11. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    upload_2024-1-10_19-57-10.png

    I might not have update the heartbeat script - will now try that
     
    philthedill, Jan 10, 2024
    #31
  12. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    done - will advise
     
    philthedill, Jan 10, 2024
    #32
  13. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    looking good

    i've loaded the heartbeat latest version and restarted with period set to 0 for send/receive and heartbeat with the following....
    I'll check on the memory situation tomorrow

    with even more thanks

    Michael
    upload_2024-1-10_21-25-32.png
     
    philthedill, Jan 10, 2024
    #33
  14. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    this morning's reading:
    upload_2024-1-11_8-6-18.png
     
    philthedill, Jan 10, 2024
    #34
  15. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    this continues to work well but I have found a small snag. I am moving my operator interfaces from SHAC to HASS. In the process, I am keen to display the 2 SHAC unit parameters - alas without success. I have added <MQTT, sensor> to the keywords as follows:

    upload_2024-1-18_13-27-41.png

    everything looks normal and I can see the following in HASS - with measurement showing as <unknown>

    upload_2024-1-18_13-30-3.png

    any ideas please?
     
    philthedill, Jan 18, 2024
    #35
  16. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    ssaunders, Jan 18, 2024
    #36
    philthedill likes this.
  17. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    For Unit Parameters give the latest commit at autoSteve a try, @philthedill.

    Treat the 'pn=' keyword as mandatory. Despite what the AC interface shows, these have no internal name that can be defaulted to. Use keywords like 'MQTT, sensor, pn=Voltage, dec=1, '.

    I have noticed that updates to keywords were not carried through to HA for me for some weird reason. On a few occasions I needed to delete the homeassistant/ discovery topic for changes to show up in HA. The Mosquitto discovery topic had been updated perfectly well, just would not show the change in HA. Killing that discovery topic with MQTT Explorer and restarting the MQTT send receive script sorted things.
     
    ssaunders, Jan 19, 2024
    #37
  18. philthedill

    philthedill

    Joined:
    Mar 21, 2010
    Messages:
    140
    Likes Received:
    3
    Location:
    Melbourne
    thanks for the follow up. I have done as you say a couple of times....the voltage parameter now appears in HASS okay. However, the utilisation has not. The name has come through but not the value....funny. I'll experiment further later on.
     
    philthedill, Jan 19, 2024
    #38
  19. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    Huh. Same for me, @philthedill. I'll have another stare at it tomorrow.
     
    ssaunders, Jan 20, 2024
    #39
  20. philthedill

    ssaunders

    Joined:
    Dec 17, 2008
    Messages:
    242
    Likes Received:
    35
    Location:
    Melbourne
    No joy, @philthedill. I'm tipping that this may be an automation controller bug, or by design.

    How about the following. Create two user parameters, AC Load and AC Memory (as floating point). Then this, a lightweight resident sleep 60 script:

    Code:
    data = io.readfile('/proc/meminfo')
    load = io.readfile('/proc/loadavg'):split(' ')[ 1 ]
    
    memtotal = data:match('MemTotal:%s+(%d+)')
    memavail = data:match('MemAvailable:%s+(%d+)')
    memusage = math.floor((memtotal - memavail) / memtotal * 1000) / 10
    
    SetUserParam(0, 'AC Load', tonumber(load))
    SetUserParam(0, 'AC Memory', memusage)
    This will get the load and available memory (as a percentage) into the two user parameters.

    Then tag the user params with MQTT, blah, blah... to get values into HA and you're away.

    If you look at the load figures in the AC UI there are three shown at the bottom. The first would be a one minute average, the second a five, and the third a fifteen. So to change the script to get the fifteen minute average, use [ 3 ] instead of [ 1 ] in the code.

    The /proc/loadavg 'file' returns a string that looks like "0.16 0.28 0.25 2/64 2040". The first three numbers I've explained. The rest you don't want. The x/xx is the number of currently executing kernel scheduling entities plus the number of kernel scheduling entities that currently exist on the system. The number at the end is the process ID that was most recently created on the system. Yeah, you don't care.

    To interpret the load number you're seeing, head on over to https://www.tecmint.com/understand-linux-load-averages-and-monitor-performance/. Nerd alert. In English: Low number? You paid too much for your CPUs. High number? You might want more, which you can't have.
     
    ssaunders, Jan 21, 2024
    #40
    philthedill likes this.
Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.