Scripting setup help with ASCii TCP/ip socket

I am after some guidance in the implementation of a TCP/ip socket setup to third party control a Symetrix DSP (control protocol same as control4, amx, crestron, lutron).
Basically I'm just wanting to control "source select" "zone selection" and "volume" from the Shac touchscreen visualization
I have connection working between both the SHAC and the Symetrix DSP, as in I can ping it with a response through the "Network Utilities"
I have full control of Symetrix working through PuTTY terminal and now need to implement scripting to manage and receive data.
I am of the understanding that i need to place a "Resident script" to open an ip socket connection, something like this???

Code:

local socket = require("socket")
local tcp = assert(socket.tcp())

tcp:connect("192.168.1.246", 48631);

Will this connection stay alive?
If it drops will this Resident script reconnect?

Do all other scripts then get placed into the "Event based" script tab or do i keep all the code located in this "Resident Script"??
Can I use ASCii strings as is? or do i need to convert them??

any help or sample scripts from working situations would be extremely helpful and appreciated.

 

Resident script is usually best to maintain tcp connection if you need to keep open all the time

if the shac starts the connection in resident script then it will keep the connection for as long as the script is running. -also remember to start and stop the script if you make changes while it’s running etc

if the server drops the connection/goes offline then ideally add in some code to check and re connect if available.

Code Depends on what you need to achieve,

do you intend to open the connection send some commands and process a response and always have connection open? May need to have a timed open/ close tcp to keep connection alive but this depends on the server/other end usually
If you do a resident script you need a way to pass commands to itself as the tcp socket object is only available in the resident script so both send and receive need to be done here, so usually some udp server code inside the same resident script can work around this.

Or perhaps a simple event script running a function to open the connection send a command and close the connection may be suitable. But it may also be problematic if many commands are to be sent.

I have both types implemented and working well so it just depends on functionality required

As far as sending commands/ascii goes it really depends on the communication protocol formatting of the server etc, is it documented?

 

Yes i have the documentation and formatting of commands are similar to these examples.

When in PuTTY if i send the command CS 1 65535 <CR>
I get Controller Set "CS" to controller number "1" set to maximum "65535" (all as expected) with a response of ACK.

I will need a script to connect to
ip 192.168.1.246.48631

then follow 10-15 cbus GA's something like this.
-- Get level of the Volume control group in the Symetrix Control 127 application on the local network
value = GetCBusLevel (0, 127, 10)

-- Convert level to command. cbus 255 granular conversion to 65535 = 65535/255 = 257
command = 'CS 34 ' .. (value * 257) .. '\r'

this is about as far as I've gotten so far

then I'll need to receive about 10-15 commands and convert them back into cbus GA levels.

If I could get a script to process 1 send command and 1 recieve command then i can complete the rest of the GA assignments.
Any chance of some help with this.
I'm really surprised that this isn't something that I could find in the forums as there are so many other control platforms that use this protocol, ( AMX, Crestron, Lutron, Control4) as far as I know only from reading and not from experience

 

I have tried my best to provide as much info as possible in my post above, is there any further help you can give???

 

So all you want to do is to send a command to a port when each GA fires? Why not use an event script? Tag all GAs with a specific keyword. Event script fired on that tag. Then you don't need a resident to maintain a connection, which would require recovery/reconnection/keep alive/etc..

The example below uses UDP, not TCP, but you could do something similar, including waiting for an ACK with a timeout then close connection. For simplicity, add a table for event.dst GA to Symetrix address translation to send the right command. Or get even more clever and tag the GA with something like 'SYMETRIX, sym=34' (assuming you chose the tag SYMETRIX) then read the GA tag 'sym=' for the GA event.dst in the event script to get the address to avoid a translation table.

Check out https://github.com/autoSteve/acMqtt, the readme, the script 'MQTT send receive' and 'MQTT' for a comprehensive use of GA tagging.

Code:

--[[
Event-based, DO NOT execute during ramping, name: "HUE final"

Pushes HUE final level events to resident script via socket.

Tag required objects with the "HUE" keyword and this script will run whenever one of those objects reaches its target level.
--]]

logging = false
huePort = 5435
hueServer = '127.0.0.1' -- Address of automation controller to send messages to

server = require('socket').udp()

val = event.getvalue()
parts = string.split(event.dst, '/')
net = tonumber(parts[1]); app = tonumber(parts[2]); group = tonumber(parts[3])

-- Send a final level event to bridge
if GetCBusRampRate(net, app, group) > 0 then
  if logging then log('Sending final level for '..event.dst..' of '..val) end
  server:sendto(event.dst..">"..val, hueServer, huePort)
end

To get kewords in a script (written blind, not tested, but based on other code I have...):

Code:

local grps = GetCBusByKW('SYMETRIX', 'or')
local k, v, alias, net, app, group
local addr = -1

for k, v in pairs(grps) do
  alias = table.concat(v.address, '/')
  if alias == event.dst then
    net = v.address[1]; app = v.address[2]; group = v.address[3]
    local tags = v.keywords
    local t
    for _, t in ipairs(tags) do
      local tp = string.split(t, '=')
      tp[1] = trim(tp[1])
      if tp[2] then
        tp[2] = trim(tp[2])
        if tp[1] == 'sym' then
          addr = int(tp[2]) -- Get device address
          break
        end
      end
    end
    break
  end
end

if addr ~= -1 then
  -- Send the command desired... you also have net/app/group
end

This would not be very performant for many tagged groups, so if that were the case I would use a different approach, loading the translation as a table into storage done elsewhere. Sadly all tags are not passed in event.tag, so they need to be looked up.

 

Is there a way in a resident script to wait for a GA event change to then activate a new action? a bit like the "event.getvalue" in event based scripts???

In this script i am getting command to send perfectly fine but it continues to send every second and i would like to modify to only send new values when a change in value occurs.

Code:

-- socket connected
if connected then
 
 
  -- Get level of the Volume control group in the Symetrix Control 127 application on the local network
value = GetCBusLevel(0,  127, 10)
 
  -- Convert level to command
--command = 'cs 34' ..(value*257).. cr
 
command = 'CS 34 '..(value * 257).. '\r'

 
    -- send command
sock:send (command)

  -- read until one line received or error occured
  while true do
    char, err = sock:receive(1)

    -- error while receiving, timeout or closed socket
    if err then
      -- remote server closed connection, reconnect
      if err == 'closed' then
        connected = false
        alert('[tcp-sock] connection closed')
        os.sleep(1)
      end

      break
    -- end of line, parse buffer
    elseif char == '\r' then
      data = table.concat(buffer)
      log(data)
      buffer = {}

      -- wait some time before next request
      os.sleep(1)

      break
    -- other char, add to buffer
    else
      buffer[ #buffer + 1 ] = char
    end
  end
-- first call or previously disconnected
else
  -- close previous connection when disconnected
  if sock then
    sock:close()
    sock = nil
  end

  -- create tcp socket
  sock = require('socket').tcp()
  sock:settimeout(1)
  connected, err = sock:connect('192.168.1.246', 48631)

  -- connect ok, reset buffer
  if connected then
    alert('[tcp-sock] connection ok')
    buffer = {}
  -- error while connecting
  else
    alert('[tcp-sock] connection failed: %s', err)
    os.sleep(5)
  end
end

 

Yep. Create an event-based script to forward the changing value over a specific socket to 127.0.0.1. Then create a listening loop in the resident script. See my first example "HUE final" for how to do that. Examine "HUE send receive" at https://github.com/autoSteve/acMqtt down the bottom of that file, the "while true do" loop, and at line 130 to set up the listening socket using UDP. Misc constants used are near the top of that file.

Not possible with a single resident unlesss you poll every GA every second and test for differences, which is not very efficient.

Your resident is set to an interval of one second, which is why it sends the GetCBusLevel value every second. Make it zero sleep, or one, and use the infinite loop approach with a timeout of one second for the listening socket. That way it will act on received messages near instantly, instead of every second, and not hog CPU.

 

Out of interest, what was your resistance to using a single event-based script with keyword tags? Simple and flexible, and a single script.

 

Does something like this work for you as a single event-based script? Fire on keyword "SYMETRIX", and use "sym=nn" keyword to set the address.

It examines the event dest, then sets up a socket to send, waits for a response (with five second timeout), then closes the socket. Not fully tested because I don't have a 192.168.1.246.

Code:

local value = event.getvalue()
local dest = event.dst

local grps = GetCBusByKW('SYMETRIX', 'or')
local k, v, alias, net, app, group
local addr = ''
local buffer = {}

for k, v in pairs(grps) do
  alias = table.concat(v.address, '/')
  if alias == dest then
    local tags = v.keywords
    local t
    for _, t in ipairs(tags) do
      local tp = string.split(t, '=')
      tp[1] = trim(tp[1])
      if tp[2] then
        tp[2] = trim(tp[2])
        if tp[1] == 'sym' then
          addr = tp[2] -- Get device address
          break
        end
      end
    end
    break
  end
end

if addr ~= '' then
  -- create tcp socket
  local sock = require('socket').tcp()
  sock:settimeout(1)
  local connected, err = sock:connect('192.168.1.246', 48631)

  -- connect ok
  if connected then
    alert('[tcp-sock] connection ok')
  -- error while connecting
  else
    log('[tcp-sock] connection failed: '..err)
    sock:close()
    do return end
  end

  local command = 'CS '..addr..' '..(value * 257).. '\r'

  -- send command
  sock:send (command)

  local sent = socket.gettime()
  local timeout = 5

  while true do
    char, err = sock:receive(1)

    -- error while receiving, timeout or closed socket
    if err then
      -- remote server closed connection
      log('[tcp-sock] connection closed'..err)
      break
    -- end of line, parse buffer
    elseif char == '\r' then
      local data = table.concat(buffer)
      log(data)
      break
    -- other char, add to buffer
    else
      buffer[ #buffer + 1 ] = char
    end

    if socket.gettime() - sent > 5 then
      log('Timeout receiving data')
      break
    end
  end
  sock:close()
else
  log('Error: sym=address tag missing')
end

 

i felt like i had invested so much time trying to get it to work I wasn't able to just quit it and start a new version. I am struggling to get me head around this LUA scripting,
The benefit is that the more i struggle with the more I am slowly getting my head around, it was only 2 years ago that i didn't even know what an ip address was.

OK so I've decided to drop the resident script and start again with your your script and try to get it up and running.
Firstly I have added keywords "Symetrix,sym=##" to about 40 GA's that will be used to send commands to the symetrix device.
In the keywords I have also added "sym=##" with ## linked to the command address of each address in the symetrix.

I've tried running it and realized that there is an addressing issue and I'm not sure how the address table would be implemented or work. when I ran the script I got the
'Error: sym=address tag missing'

"For simplicity, add a table for event.dst GA to Symetrix address translation to send the right command."
How do I set up the keyword "sym=##" to become the "Command send" or 'CS' address.
I am very grateful for you time and help with this.

 

At line 8, add a log(grps) to see what's in the keywords.

Or better at line 13 insert, log(tags) to see the specific tags for an event.

 

I'm getting

Symetrix Keyword Control 29.10.2023 07:53:19
* table:
Symetrix Keyword Control 29.10.2023 07:53:19
* string: Error: sym=address tag missing

 

Could you post your full code, as well as a screen shot of the tags. I cannot replicate this.

If any keywords change you will need to disable/enable the event based script to pick up the new keywords. This may be behind it, although getting a empty table is weird.

 

Code:

    local value = event.getvalue()
    local dest = event.dst

    local grps = GetCBusByKW('Symtrix', 'or')
    local k, v, alias, net, app, group
    local addr = ''
    local buffer = {}
 log(grps)
    for k, v in pairs(grps) do
      alias = table.concat(v.address, '/')
      if alias == dest then
        local tags = v.keywords
     --log(tags)
        local t
        for _, t in ipairs(tags) do
          local tp = string.split(t, '=')
          tp[1] = trim(tp[1])
          if tp[2] then
            tp[2] = trim(tp[2])
            if tp[1] == 'sym' then
              addr = tp[2] -- Get device address
              break
            end
          end
        end
        break
      end
    end

    if addr ~= '' then
      -- create tcp socket
      local sock = require('socket').tcp()
      sock:settimeout(1)
      local connected, err = sock:connect('192.168.1.246', 48631)

      -- connect ok
      if connected then
        alert('[tcp-sock] connection ok')
      -- error while connecting
      else
        log('[tcp-sock] connection failed: '..err)
        sock:close()
        do return end
      end

      local command = 'CS '..addr..' '..(value * 257).. '\r'

      -- send command
      sock:send (command)

      local sent = socket.gettime()
      local timeout = 5

      while true do
        char, err = sock:receive(1)

        -- error while receiving, timeout or closed socket
        if err then
          -- remote server closed connection
          log('[tcp-sock] connection closed'..err)
          break
        -- end of line, parse buffer
        elseif char == '\r' then
          local data = table.concat(buffer)
          log(data)
          break
        -- other char, add to buffer
        else
          buffer[ #buffer + 1 ] = char
        end

        if socket.gettime() - sent > 5 then
          log('Timeout receiving data')
          break
        end
      end
      sock:close()
    else
      log('Error: sym=address tag missing')
    end

   

The keywords were "Sym=##" I'll get a chance to run again with the upper case in the script shortly

 

Typo? "Symtrix". Should it have an "e"?

All keywords are case sensitive, too.

And just noticed I failed to include trim(). I have this as a library function:

Code:

local function trim(s) if s ~= nil then return s:match "^%s*(.-)%s*$" else return nil end end -- Remove leading and trailing spaces

 

if you want to ignore case then use:

Code:

          tp[1] = string.lower(trim(tp[1]))
          if tp[2] then
            tp[2] = string.lower(trim(tp[2]))
            if tp[1] == 'sym' then

 

I've occasionally thought about this for event-based scripts, but not confirmed any issue doing so, @Pie Boy. I'd imagine that if two scripts simultaneously attempted communication over a newly created socket to the same remote host then each connection would come from a different source TCP port, e.g. 54321->443, 52134->443. The remote host will treat each conversation distinctly. For much simultanaity there may be connection limits that might slow things down, but due to the nature of TCP shouldn't break with the number of connection requests not being outrageous.

What issues have you encountered with this approach? (Assuming using all local variables in a distinct LUA table-based stack per script invocation.) I can't think of any down-side.

 

How'd you go @kojobomb?

You might appreciate this version of my code. It includes trim(), case insensitivity for sym=xx, and also a more efficient way to receive data on the socket. Basically gets entire lines at a time, not characters, and assumes the responses are short.

Code:

local logging = true

local value = event.getvalue()
local dest = event.dst

local grps = GetCBusByKW('SYMETRIX', 'or') -- The only keyword used is case sensitive
local k, v, alias, net, app, group
local addr = ''
local function trim(s) if s ~= nil then return s:match "^%s*(.-)%s*$" else return nil end end -- Remove leading and trailing spaces

for k, v in pairs(grps) do
  alias = table.concat(v.address, '/')
  if alias == dest then
    local tags = v.keywords
    local t
    for _, t in ipairs(tags) do
      local tp = string.split(t, '=')
      tp[1] = trim(tp[1]):lower()
      if tp[2] then
        tp[2] = trim(tp[2])
        if tp[1] == 'sym' then
          addr = tp[2] -- Get device address
          break
        end
      end
    end
    break
  end
end

if addr ~= '' then
  -- create tcp socket
  local sock = require('socket').tcp()
  sock:settimeout(1)
  local connected, err = sock:connect('192.168.1.246', 48631)

  if connected then
    -- connect ok
    if logging then log('[tcp-sock] connection ok') end
  else
    -- error while connecting
    log('[tcp-sock] connection failed: '..err)
    sock:close()
    do return end
  end

  local command = 'CS '..addr..' '..(value * 257).. '\r'

  -- send command
  sock:send(command)

  local sent = socket.gettime()
  local timeout = 5

  while true do
    local data, err, partial = sock:receive('*l')

    if err or partial ~= nil then
      -- error while receiving, or an incomplete line received
      if err then log('[tcp-sock] connection closed: '..err) end
      if partial ~= nil then log('[tcp-sock] warning... partial line received: '..partial) end
      break
    else
      -- entire response received okay
      log(data)
      break
    end

    if socket.gettime() - sent > 5 then log('Timeout receiving data') break end
  end
  sock:close()
else
  log('Error: sym=address tag missing')
end

 

its all really based on what functionality is required to be achieved

if i don't need feedback/response from tcp, then i do the same way as you described above but open send and close in one event script with KW and event.dst filtering- no need to wait for the response etc.

but if say you wanted to send a volume up command and the button was working like a bell press and commands were getting sent one after the other i would suggest many many many event scripts would run in parallel and eventually consume much CPU, also there will probably be a backlog of commands and after the bell press is released the commands would still be being sent... its really tricky to manage running processes with event scripts running like this, but for a few commands now and then its fine, i guess its a lightweight solution and has its limitations.

Main issue I've found is with tcp server/s that don't allow more than one tcp connection at a time, so when many event scripts run in parallel then sometimes commands can get missed as another script running in parallel has the tcp connection, but this is hardware and protocol specific.

So in this situation where for functionality feedback is important and commands sent are few, iv opted for a resident script that manages one tcp connection and pass all events (from KW script) to be sent to the resident script with event script and UDP then use a loop with fd's to make it consume less CPU and only run when there is a message to be read or sent.
events are passed to the resident script and response is processed in the resident script then applied to whatever object with whatever filtering.

Main issue with resident scripts is making them efficient so they are light on CPU...

 

Nice one. Guess I'm used to talking at servers that allow gobs of connections. Or better yet, MQ brokers.

For the AC I, too almost always utilise a resident that listens to an event script, queues, sends. This allows for incoming message examination allowing for flushing the queue part way, ignoring the rest.

So @kojobomb, there's a stretch goal for you. Get my code working as a base, then work out how to do an event script with keywords that forwards to a listening resident script for processing (best to use UDP like my example above for the event script as that's super lightweight) that queues, and works through its queued messages to then send to the Symetrix and wait for ACK. Then, like @Pie Boy pointed out, you can halt mid-way through processing the queue if some condition determines that you should.