Modbus broken in 0.108b #33754
Comments
|
Hey there @adamchengtkc, @janiversen, mind taking a look at this issue as its been labeled with a integration ( |
|
Hi. The warning is from pymodbus, and can be avoided by installing twister (pip3 install twister). I actually had a patch to do this automatically, but the reviewers (which happened to be the one who can merge the code) asked for this to be removed. The decision was that it was better to leave the warning and hope the pymodbus project will remove it soon. However it have no effect on the functionality of the code. Is the report just because of this warning, or are you having real problems ? (in which case please add the log). Thanks for your report. |
|
My sensors are not updating. |
|
Does your sensors show “unavailable” ? that would signal the integration is not loaded, if your sensors are not updating it is a different problem. You did not include any sensor configuration so I cannot test your configuration here. I have sensor included in my raspberry pi configuration (production) and it works without problems with the warning. |
|
I did come checking and all I know for now is that everything worked perfectly in 0.106.6 with previous Supervisor (213?). Since the supervisor got updated, some Modbus sensors (analog, holding registers) are unavailable randomly and the processor load is constantly high (0.4 or more), but it still mostly works. If I update to anything over 0.107, including the new beta 0.108.0b3, the same sensors stop updating altogether. |
|
I do believe that you have a problem, and that there is a bug in modbus, however in order to find it, I need to be able to reproduce the problem. Can you please update the configuration, so I can download it and test it. I need to see how you define the sensors, and how you calculate the sum. As it happens I am also using modbus to read out my Huawei sun2000 inverter, which works perfectly. |
|
my sensors are down for a few seconds every hour and so. I had a talk with the Huawei engineers and they told me, that there are a racing problem, if modbus request data in the same moment data are being updated. The old modbus implementation did not detect that, but merely skipped and update, so you actually worked with old data. |
|
@vzahradnik you have a much bigger modbus configuration than I have, if you have a spare minute, can you please take a look at this issue, and tell me if you see something identical ? |
|
@janiversen my configuration seems to work OK. I have two warnings and one error. One warning is related to this Twisted dependency, the rest is not related to Modbus. If there's a bug, our best option would be if @FrenkK provided his full config (if preferred, anonymized and/or stripped-down to a bare minimum). 2020-04-07 09:41:50 WARNING (MainThread) [pymodbus.client.asynchronous] Not Importing deprecated clients. Dependency Twisted is not Installed
2020-04-07 09:41:51 WARNING (MainThread) [homeassistant.components.lovelace] Resources need to be specified in your configuration.yaml. Please see the docs.
2020-04-07 09:41:53 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 38, in initialize
await scanner.async_scan(None)
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 61, in async_scan
await self._process_entries(entries)
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 85, in _process_entries
result for result in await asyncio.gather(*tasks) if result is not None
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 120, in _process_entry
info.update(await info_req)
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 120, in _process_entry
info.update(await info_req)
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 120, in _process_entry
info.update(await info_req)
File "/srv/git/hassio/home-assistant/homeassistant/components/ssdp/__init__.py", line 138, in _fetch_description
xml = await resp.text()
File "/srv/git/hassio/home-assistant/venv/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 1014, in text
return self._body.decode(encoding, errors=errors) # type: ignore
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 229: invalid start byte |
|
@janiversen maybe not related to this issue, but the difference between me and @FrenkK is that I don't use |
|
Sorry for the delay, I investigated a bit. It looks like there may be several problems interacting here. Here are the relevant parts of my configuration: #Modbus TCP
modbus:
- name: toplotna
type: tcp
host: 192.168.1.8
port: 502
timeout: 10
- name: lopa
type: tcp
host: 192.168.1.7
port: 502
timeout: 10Raw sensors: #Vrednosti iz razsmernika
- platform: modbus
registers:
- name: Opozorila razsmernika
hub: lopa
slave: 1
register_type: holding
register: 1
- name: Biti nacina
hub: lopa
slave: 1
register_type: holding
register: 7
- name: Surova napetost baterije
hub: lopa
slave: 1
register_type: holding
register: 188
count: 1
data_type: int
scale: 0.1
unit_of_measurement: V
precision: 1
- name: Napetost niza 1
hub: lopa
slave: 1
register_type: holding
register: 234
count: 1
data_type: int
scale: 0.1
unit_of_measurement: V
precision: 1
- name: Napetost niza 2
hub: lopa
slave: 1
register_type: holding
register: 235
count: 1
data_type: int
scale: 0.1
unit_of_measurement: V
precision: 1
- name: Tok niza 1
hub: lopa
slave: 1
register_type: holding
register: 696
count: 1
data_type: int
unit_of_measurement: A
scale: 0.01
precision: 1
- name: Tok niza 2
hub: lopa
slave: 1
register_type: holding
register: 698
count: 1
data_type: int
unit_of_measurement: A
scale: 0.01
precision: 1
- name: Tok v baterijo
hub: lopa
slave: 1
register_type: holding
register: 230
count: 2
data_type: int
unit_of_measurement: A
scale: 0.1
precision: 1
- name: Proizvodnja niza 1
hub: lopa
slave: 1
register_type: holding
register: 228
count: 2
data_type: int
unit_of_measurement: W
- name: Proizvodnja niza 2
hub: lopa
slave: 1
register_type: holding
register: 232
count: 2
data_type: int
unit_of_measurement: WCalculated sensors: #Izpeljane vrednosti iz razsmernika
- platform: template
sensors:
moc_v_baterijo:
value_template: '{{ (((states("sensor.tok_v_baterijo") | float ) * (states("sensor.surova_napetost_baterije") | float )) | int ) }}'
friendly_name: "Moč v baterijo"
unit_of_measurement: 'W'
proizvodnja:
value_template: '{{ (((states("sensor.proizvodnja_niza_1") | int ) + (states("sensor.proizvodnja_niza_2") | int )) | int ) }}'
friendly_name: "Proizvodnja"
unit_of_measurement: 'W'
kompenzirana_napetost_baterije:
value_template: '{{ ((((states("sensor.surova_napetost_baterije") | float ) - (states("sensor.tok_v_baterijo") | float )*0.001)) | round(1) ) }}'
unit_of_measurement: 'V'
#Filtrirane vrednosti iz razsmernika
- platform: filter
name: Napetost baterije
entity_id: sensor.kompenzirana_napetost_baterije
filters:
- filter: outlier
window_size: 10
radius: 2
- filter: time_simple_moving_average
window_size: 00:05
precision: 1 |
Looks like you are right. I commented them out and all the sensors updated at least once. But now they are stuck again. So my hypothesis about the integration failing and not recovering properly seems to hold. |
|
Update: I set the timeouts to (extreme value) 120 seconds and the sensors update a few times, then they die. Whatever the problem is, once it happens, the integration dies permanently. |
|
First of all @vzahradnik thanks for spending time to look at this. @FrenkK I will try to reply to all your messages in this commit:
You should use scan_interval on the sensors. The value of scan_interval is a When you see that message, requests start to queue up in HA, leading to a higher CPU usage.
The new code is async so it does not wait for a request to terminate, which can lead to a request buildup, but on the other hand is faster and more “polite” to other integrations by not blocking.
Since it worked in 1.106.6 there are no reason to add timeout for 1.108x
|
|
Thank you for working with me :)
Timeout was part of Modbus integration spec at the time I upgraded my installation, late last year, quite earlier than 0.106.6. This is why it's there, it always was. Anyway, when I removed it to test the @vzahradnik suggestion, the problem was the same. I did not change the config between upgrades/downgrades until then.
The documentation says scan_interval has a default value of 30 seconds, it is not needed to specify it unless a different value is needed. Anyway, I now set it to 60 seconds for all Modbus sensors and it worked for about 5 minutes, then stopped again. The error log is the same, just the number of seconds is longer. |
|
@janiversen I checked the code. The |
Or most likely, the documentation PR was merged into the |
|
I am sorry, I confused myself I was thinking about “delay” which is not “timeout”. @vzahradnik documentation is merged to the next branch. @FrenkK the seconds in “taking longer than” is actually the scan interval. If it worked for 5 minutes it seems you are on the right way. I actually use different scan intervals, the power meter (import/export of energy) is read every 10 seconds, the PV production is read every 30 seconds, and the rest is every 5 minutes. Just for reason of testing, try to make 2 scan_intervals, one 30 seconds with just the powermeter, and the second with the rest at something like 5 minutes. This is just to see if the problem is in the device. You read some indirect values (depending on your inverter), like battery values. The inverter typically act as a hub for such values, but need to do a modbus (rs485) call to the battery (of course assuming you have intelligent batteries). |
|
Thank you for now, I will test these in the evening.
No, I'm just reading the voltage of the battery directly from the inverter, there is no chain. |
|
Ok, I was testing some more with different scan intervals etc. While I can get it to work for minutes or even a bit longer, the simple fact is that once there is a problem with any Modbus sensor, it is all over, none of them work anymore. Any integration must have some kind of graceful recovery, there are always unpredictable events, and this is not the case currently for Modbus. |
|
please add a log where it works a while snd then stops working. There are a number of modbus devices that do not recover from failures unless the connection is broken. |
|
Before running please add these lines to configuration.yaml logger: That will provide a lot of debug, you can search for pymodbus and see e.g.: Please mail me that log (or add it here). |
|
|
Let it run for something like 5 minutes after the problem occurs, so I can see retries etc. |
|
log.txt |
|
Thanks for the log it is real helpful. Lets first be clear the transaction fails because the device do not adhere to the modbus protocol, the device should return an error or disconnect. But admitted I have seen this on a number of devices. Each hub have its own connection and own objects in the integration (actually they do not know the other exist), so when one hub fails the other hub is not affected at least not more than any other integration in your config. The transaction never ends, therefore all sensor belonging to that hub are not getting updates and eventually they will be marked unavailable. The good news is that I have an idea on how to solve the hanging (which is quite against the protocol definition, but should work) and while at it I think I can solve the problem of overlapping transactions as well. It will take me a couple of days to get it done, but you will be informed when I submit a PR. Depending on your setup, you can replace the files with the files from the PR, and test it. |
|
Just chiming in, I am using a Siemens PLC and since upgrading to 0.108, Modbus no longer works, as described here. |
Please add the logger lines (see earlier message) to your configuration.yaml, and verify your problem is the same. Thanks. |
|
@thehaxxa and of course I would like to see your modbus configuration (incl. sensors) togethers with a log showing your problem. |
|
@janiversen I have downgraded back to 0.107.7 (now its working again), I will try to get some logs sorted out as soon as I can. |
|
I can for sure understand you have downgraded, but the logs and configs are really important in for us to be able to help you |
|
@frenck, @thehaxxa I would like if you both could verify the patch makes it work for you, thanks in advance. |
|
@janiversen, it seems to work now. The processor load is still quite higher than earlier, but that could be due to new functionality in this version. Some sensors occasionally take a long time to update, but update they do :) |
|
Unfortunately, the sensors died again after a while. This time they did not die all at once, but once they stop updating, that is it. Error log looks the same. @janiversen, you did something, but it seems it is not the whole story. |
|
We are continuing in issue #33873, please let me see the logs “looks the same” cannot really be, there should be differences due to my fix. |
|
ups that was #33872, I have a fix ready for that, but will wait and see what your problem is. |
|
I'll downgrade and wait until you deal with that, I'm not in a hurry :) |
|
Already merged to dev. |
|
The 0.108.1 version seems to be working. Some sensors update quite slowly sometimes, but they do work. Thank you for your help :) |
|
my pleasure, the reason some of your sensors are slowly at times, is because your device does not respond and thus leaves a transaction open. The patch I provided secures the integrations “survives”, but I have another planned that will speed up responses in this case. |
|
0.108.2 Modbus not work
Error during setup of component modbus |
|
@volgunov-ad this is a closed issue. Your problem is known (but thanks for reporting it), you can follow the disccussion in issue #33872, which is about the serial problem. |
The problem
Modbus configuration no longer works in 0.108b3. Error message: "Not Importing deprecated clients. Dependency Twisted is not Installed"
Environment
https://rc.home-assistant.io/integrations/modbus/
Problem-relevant
configuration.yamlTraceback/Error logs
Additional information
The text was updated successfully, but these errors were encountered: