fix agc reset on SX126x, SX1276 & LR11x0 chips

[weebl2000]

• tried to simply explicitly call sleep briefly to reset AGC -> not sufficient it seems
• do more thorough reset using Calibrate (0x7F) -> success
• reset noise_floor so firmware shows accurate values
• so far it seems to work great on a Heltec v4
• should work on all SX126x based chips by using Calibrate(0x7F)
• shoudl work on all LR11x0 based chips by using Calibrate(0x3F)
• should work on SX1276 chips too, calling sleep there should be sufficient - SX1276's AGC is simpler — just LNA gain stepping. Sleep mode powers down the entire analog frontend, and re-entering RX restarts the AGC from G1 (max gain).
• default calibration is in 902-928 MHz range, made sure to follow up with actual configured frequency
• build firmware for yourself here: https://mcimages.weebl.me/?commitId=fixagcreset

fixes #1716- see discussion there for initial test results
fixes #1209

I've referenced SX1262 datasheet to implement the calibration reset.

• p25 4.3 ,64-65 table 12-1 listing ΣΔ ADC architecture
• p55 table 9-1 sleep enable vs stdby_rc enable
• p55 9.2 states which blocks can be calibrated
• p61 9.2.1 calibrate specific freq
• p66 warm sleep restores register state confirms we need more aggressive Calibrate(0x7F) reset
• p73 total calibration time is 3.5 ms and must be launched in STDBY_RC mode

For LR1110 similar implementation, the 'AGC' is completely black boxed behind firmware so you can only reset it by resetting registers.

LR1110 datasheet page 9 section 1.2.1 states:
•Air interface fully compatible with the SX1261/2/8 family

Also see LR1110 user docs

So reset is very similar.

[towerviewcams]

what boards do you think the AGC doesn't work on? On a V4, I've been watching closely at a terrible site I have with 945Mhz STL microwave and turned off agc.reset.interval and then back on. I think it "might" be working but not sold on that.

[terminalvelocity23]

what boards do you think the AGC doesn't work on? On a V4, I've been watching closely at a terrible site I have with 945Mhz STL microwave and turned off agc.reset.interval and then back on. I think it "might" be working but not sold on that.

I have reports that it gets stuck on V3 and T114, as well. My DIY node based on ESP32-S3 and E22 900M30S gets affected, too.

[towerviewcams]

I figured out why I'm not seeing this AGC problem. Its had me stumped on and off for a couple days. I set my repeaters to Zero on the int.thresh 0

Once I changes this back to default, yep, I can watch my noise floor eventually hit -120 on my V4 and be of course very desensitized.

I tried @weebl2000 test firm builder and in fact it is fixed. also, no current spike on AGC rest every 12 seconds that mine is set for the test.

[towerviewcams]

@weebl2000 Your test firm still has the 22 second powersaving delay. Can you update your base 1.13 firm with that fix? I'm testing your AGC fix now but since I'm in a busy area, I'm never sleeping and its draining battery pretty good in our rain for days weather.

[weebl2000]

@weebl2000 Your test firm still has the 22 second powersaving delay. Can you update your base 1.13 firm with that fix? I'm testing your AGC fix now but since I'm in a busy area, I'm never sleeping and its draining battery pretty good in our rain for days weather.

the fixagcreset is up-to-date with the meshcore/dev branch. Are you building fixagcreset branch? https://mcimages.weebl.me/?commitId=fixagcreset what branch is the 22 second powersaving delay branch? I can merge that one with this one for you so you can test the combination.

[towerviewcams]

@weebl2000 the 22 second delay showed up in 1.13 and @IoTThinks Kevin made a hotfix on his github. They also also have powersaving13 as well. I've been testing it on 2, V4 boards and its awesome. It is not merged into the dev branch at this time.

If you could do something temporary with ether the hot fix, that would allow me to test at 3 tower sites tomorrow for work that I'm taking an insurance agent to. All easy access repeaters in town.

The hot fix on easy sky mesh is under Firmware, Testing, PR-1687, : heltec_v4_repeater-002878e.bin and I have that running in addition to stock 1.13 and its perfect. I'm guessing that would be the simplest? All 3 of these sites go numb -120 in just under 24 hrs and this AGC reset fix should be golden to find out really quick!

[IoTThinks]

@weebl2000 22-second delay in PowerSaving is likely the OLED issue on Heltec v4 in the current MC 1.13 (main and dev).

It should not be related / affected to your PR.

#1562 (comment)

@towerviewcams The fix for 22-second powersaving delay in PowerSaving 13 is included in the bellow pending PR too.
It will sleep immediately after processing LoRa packets.

#1687

[weebl2000]

The hot fix on easy sky mesh is under Firmware, Testing, PR-1687, : heltec_v4_repeater-002878e.bin and I have that running in addition to stock 1.13 and its perfect. I'm guessing that would be the simplest? All 3 of these sites go numb -120 in just under 24 hrs and this AGC reset fix should be golden to find out really quick!

try this one: https://mcimages.weebl.me?commitId=agc-tower it includes this PR + #1687 + #1569

[towerviewcams]

@weebl2000 Trying right now!

[towerviewcams]

@weebl2000 Wow! what a difference! I'm slamming 2 boards at the same time, with 2 different companions and 2 ios phones. So im connected at the same time.

#1 V4 has the AGC/pwr save fix. This PR and 1687 and 1569
#2 has 1.13 stock

I'm slamming them both side by side and the AGC reset works every time. back down to -104 ish......... #2 is sticking and not resetting. Sometimes its slow, and others not. I'm sure if I let this run for a day we would see.

-Field Test
I'm taking this to a major high power broadcast site now where I really need AGC to work, I bounce all over the place from -86 up to -102 on my V4 there. I have to reboot for it to come back to -100's....Then eventually drops down and up.....So, I'll report back in a couple hours once its OTA updated. This looks very promising!! Thank you @weebl2000

@IoTThinks, Keven, I know you told me on FaceBook that you haven't seen where you need to reset. Let me test this and I'll know very quickly. I'm sitting next to over 1 megawatt of power and 4 cellular companies all on my 800ft tower that I manage.

[towerviewcams]

90 mins in and holding -98 to -102 on noise floor readings. This is exciting so far!!!!

[ignisf]

The FAQ may need to be updated.

[IoTThinks]

@towerviewcams Well, 1MW interference. Then it makes sense.

So your agc reset interval is 4?

Besides the noise floor number, have you observed if the repeaters can receive more packets or better SNR...?

I would love to have lower noise floor and better RX too.

My noise floor for Heltec v3s with 3 and 5dbi is -97 to even -77 recently.

[towerviewcams]

@towerviewcams Well, 1MW interference. Then it makes sense.

So your agc reset interval is 4?

Besides the noise floor number, have you observed if the repeaters can receive more packets or better SNR...?

I would love to have lower noise floor and better RX too.

My AGC reset is 8. Everything has been working great. Prior to this reset working properly, decoding packets was very intermitant and required a very strong signal as noise floor would increase to -86 or so and remain there. Now, I'm holding -97 to -102......I do see a suddon drop to -86 and then it pops back up with the AGC reset. So far so good!

[towerviewcams]

Today, I'm going to OTA this at another location that has 600,000 watts broadcast noise and 3 cellular tenants. My noise floor here drops slowly to -91 and I was going to do the LNA bypass. Not now, lets try this! Update soon!

[IoTThinks]

@towerviewcams Great.
Let me test on my Heltec v3 too.

This symptom seems to happen to many boards including RAK4631. Not just Heltec v4.

[towerviewcams]

Now running this on another V4 in the field next to 600,000 ERP broadcast transmitter and AGC is working nicely. Taking my Wio down to power -9 and can trace now just fine! Before, my noise floor was holding -89 and could not. Let this run for rest of the day and I'll know for sure. now im at a very nice -96-98 ish. reset rate 8 seconds. If works great I'm going to set my reset timer to 16 on both tests. then more as time goes on to find the sweet spot where it starts to drop before reset

[terminalvelocity23]

I've tested this on a V3, works fine, just as well as with V4.

[IoTThinks]

@towerviewcams I have tested with my Heltec v3 with 3dbi antenna for half a day.

MC 1.13:
agc.reset.interval=0, my noise floor is jumping around -8x to -76.
If agc.reset.interval=4, 8, 300, 600, the noise floor will be stuck at -120 after a day.

This branch: (commit dbb739)
If agc.reset.interval=0 or agc.reset.interval=4, 8, 300, 600, my noise floor is jumping around -8x to -76.
So yes, it has not been stuck at -120 yet.

All values for agc.reset.interval give the same noise floor.
Should I expect the noise floor to be better at -100 to -90 instead?
Your noise source should be much stronger than my cell towers.

[towerviewcams]

@towerviewcams I have tested with my Heltec v3 with 3dbi antenna for half a day.

MC 1.13: agc.reset.interval=0, my noise floor is jumping around -8x to -76. If agc.reset.interval=4, 8, 300, 600, the noise floor will be stuck at -120 after a day.

This branch: (commit dbb739) If agc.reset.interval=0 or agc.reset.interval=4, 8, 300, 600, my noise floor is jumping around -8x to -76. So yes, it has not been stuck at -120 yet.

All values for agc.reset.interval give the same noise floor. Should I expect the noise floor to be better at -100 to -90 instead? Your noise source should be much stronger than my cell towers.

@IoTThinks My noise is broadcast at 580Mhz so its far away from 910Mhz but a HUGE signal source for sure. I'm glad your seeing good results. Cellular is much closer to 910mhz and will get through your filter easier. So it doesn't surprise me.

[towerviewcams]

More then 32 hours and running perfectly on V4 still using the "agc-tower" firm build. Fantastic @weebl2000

Just as a thought, @IoTThinks Keven, will you please please put this into a test Powersaving13.1 beta test? Lets consider putting this together?

[towerviewcams]

Is that PR #1687 ? If so I've merged this PR, #1687 and #1569 together to this branch: http://mcimages.weebl.me/?commitId=agc-powersave

Maybe it's worth merging #1600, as well.

Good idea. Merged & pushed to http://mcimages.weebl.me/?commitId=agc-powersave

@weebl2000 So this now has 1569, 1600 and 1687, correct? This would be @IoTThinks Kevin's latest as I understand and the 3 PR fixes. I will flash 3, V4 boards and test......... and now we have a V4.3 still figuring out how / when we can get it.

[weebl2000]

@weebl2000 So this now has 1569, 1600 and 1687, correct? This would be @IoTThinks Kevin's latest as I understand and the 3 PR fixes. I will flash 3, V4 boards and test......... and now we have a V4.3 still figuring out how / when we can get it.

correct!

[IoTThinks]

PR 1687 is latest PowerSaving.
Go ahead and test with @weebl2000 's merge.

[towerviewcams]

PR 1687 is latest PowerSaving. Go ahead and test with @weebl2000 's merge.

@IoTThinks Ok Kevin, you got it! flashing 3, V4 boards now with @weebl2000 merge :)

[terminalvelocity23]

Good idea. Merged & pushed to http://mcimages.weebl.me/?commitId=agc-powersave

Trying this out. Looks like the powersaving feature is too aggressive for CLI to work, you have to disable it to make changes to router config :)

[towerviewcams]

Good idea. Merged & pushed to http://mcimages.weebl.me/?commitId=agc-powersave

Trying this out. Looks like the powersaving feature is too aggressive for CLI to work, you have to disable it to make changes to router config :)

What are you trying to do in the CLI when you're testing this?

[terminalvelocity23]

What are you trying to do in the CLI when you're testing this?

Getting the state of variables, like tx, txdelay and the like.

[towerviewcams]

What are you trying to do in the CLI when you're testing this?

Getting the state of variables, like tx, txdelay and the like.

Works great via the app ota. you must be trying USB or something

[terminalvelocity23]

What are you trying to do in the CLI when you're testing this?

Getting the state of variables, like tx, txdelay and the like.

Works great via the app ota. you must be trying USB or something

No, I'm using BLE. It seems to me that disabling powersaving makes the CLI more snappy in its responses to me, but maybe I'm mistaken. Or maybe it's something to do with my settings - AF 5, txdelay 0.5, rxdelay 0.5, int.thresh 10, agc.reset.interval 8.

The current draw with powersaving on is 24 mA vs 57 mA with off.

[towerviewcams]

What are you trying to do in the CLI when you're testing this?

Getting the state of variables, like tx, txdelay and the like.

Works great via the app ota. you must be trying USB or something

No, I'm using BLE. It seems to me that disabling powersaving makes the CLI more snappy in its responses to me, but maybe I'm mistaken. Or maybe it's something to do with my settings - AF 5, txdelay 0.5, rxdelay 0.5, int.thresh 10, agc.reset.interval 8.

The current draw with powersaving on is 24 mA vs 57 mA with off.

Oh, I'm sorry. I thought you were using it as a repeater my bad.

[terminalvelocity23]

Oh, I'm sorry. I thought you were using it as a repeater my bad.

I'm using it as a repeater obviously, accessing it over the LoRa remotely. With this power saving patch I can't even login half of the time now, and the app is less responsive in processing things like neighbors list and the like. It makes sense, if the ESP force-sleeps as soon as the packets leave the antenna :) Anyway, it's not a huge deal, if I need to change the settings, I can always turn power saving off.

[weebl2000]

Oh, I'm sorry. I thought you were using it as a repeater my bad.

I'm using it as a repeater obviously, accessing it over the LoRa remotely. With this power saving patch I can't even login half of the time now, and the app is less responsive in processing things like neighbors list and the like. It makes sense, if the ESP force-sleeps as soon as the packets leave the antenna :) Anyway, it's not a huge deal, if I need to change the settings, I can always turn power saving off.

Would be good if you can add this as feedback in the original PR - it definitely seems like a bug.

If you want to try all my open PRs you can try to build https://mcimages.weebl.me?commitId=dev_plus - it has some great improvements imho.

[terminalvelocity23]

Would be good if you can add this as feedback in the original PR - it definitely seems like a bug.

I'll test it more comprehensively over the next couple of days. Hate to give false positives.

If you want to try all my open PRs you can try to build https://mcimages.weebl.me?commitId=dev_plus - it has some great improvements imho.

I have most of them compiled in anyway :) Thank you for your outstanding work btw.

[beachmiles]

@terminalvelocity23 I also have been having difficulty getting repeater admin CLI commands to work even with powersaving off.
After I turned off powersaving the CLI started working way better (8/8 successfull cmds) but only for like 60 seconds and then it went back to sucking again and am only getting 1/10 cmds to work.
Logging in and the status refresh button are the only commands that seem to work good and the refresh button seems to always work.
PR1687 was having a similar CLI issues but dont think it was this bad?

The good news is I havent seen the noise floor get stuck at -120, the bad news is that I am still having terrible RX performance with this v4 node on the roof with an acasom cavity filter.
I tried on another v4 companion with a wehooper ceramic filter sitting next to the repeater on the roof and was seeing similar huge jumps and falls of the noise floor in the moving graph in the app.

Here are noise floor readings with agc.reset.inverval=0 at with about a 2 seconds delay between reads. Some crazy swings happening.
-99,-74, -72, -83, -58, -58, -90, -73, -79, -59, -71, -84, -85, -99. -100, -91, -72, -53, -96, -97, -88, -87, -93, -98, -92, -62, -85
Im getting 8.6 Rx packs/min and 7.1 Rx Pack Errs/min now on the roof where ~15 feet lower inside I was getting ~14.9 Rx packs/min and only 4.4 Rx Errs/min.

Was seeing better noise floor numbers with agc.reset.inverval=4 but still having poor trace success rates
-88,-87,-98, 94.-78,-92,-81,-79,-74,-98,-99,-91,-89

Was seeing some slightly worse noise floor numbers with agc.reset.inverval=8 pop up like -55, similar trace success rates
-68,-80,-76,-59,-55,-96,-77,-55,-88,-77,-86,-88,fail,-85

When the traces are successful I am generally seeing much higher RX and TX db numbers with it on the roof vs 15 feet below indoors. And I can occasionally can get traces through to some way further nodes that I could never hit with the unit indoors but my general trace success rate is way worse now.

Think I need to disable the boosted gain or something with this repeater installed on the roof as its having major issues dealing with the outdoor city noise / the wider audience of repeaters?

Here is the build I'm using created today at 12:33pm from here https://mcimages.weebl.me/?commitId=agc-powersave
firmware-heltec_v4_repeater-agc-powersave-1569-1600-1687.zip

[weebl2000]

@beachmiles do you have int.thresh set? If not, try setting it to anywhere between 5-10, or even lower if you have a proper filter. set int.thresh 10 is a good start if you've not tried it before. I also highly recommend trying out PR #1727 - it enables hardware channel activity detection before trying to send. I have the ACASOM running on my Heltec v4 too.

If you want to try all of my PRs you can build my dev_plus branch https://mcimages.weebl.me?commitId=dev_plus - it contains all my open PRs. I've been running it for over a month and haven't noticed any issues yet.

dev_plus is what I'm running on all my devices. You can view all changes compared to dev here: dev...weebl2000:MeshCore:e488de5d0c25c9ee08fae6eb7fc18acfc4adbef0

[IoTThinks]

For the powersaving, you are welcome to feedback on the PR #1687
And you may turn off powersaving by powersaving off to test and compare the responsiveness of CLI vs. powersaving on.
I'm quite sure you will see similar behaviors.

For the responsiveness of the CLI, last time, I patched this to improve the responsiveness .
Before that, I need to reboot my T1000-e every few days for CLI to be responsive again.
#1299

I believe this PR will indirectly improve the CLI responsiveness due to better RX.
My Heltec v3 on MC 1.13 (PS 13) has not stuck at -120 yet. The agc.reset.interval=0. However, the noise floor is constantly high at -77.
Another Heltec v3 on this PR with agc.reset.interval=8. The noise floor is actively changing from -77 to -85.

Before Lunar new year, they are all at -97. May be last time, they are on MC 11.
I have refreshed the status page many times to see the noise floor.

[beachmiles]

I may be seeing slightly more successful traces setting int.thresh to 10 or to 5. I am seeing better noise floor numbers now like -112 but prob cause its getting later in the night.
I did get 2 refresh status failures in a row and then I saw that the Debug Flags = 2. Rebooted and it went away but later came back did another reboot and Debug Flags = 2 was again shown.

Will try loading another build on Monday as its now on the roof in a steel case without an external wifi antenna.

[towerviewcams]

@weebl2000 @IoTThinks BIG UPDATE

So the AGC-Powersave that has the three merged PR and the quick sleep does have serious login responsiveness issues. @beachmiles is totally correct. So I have pulled that from my tests that ran for 24hrs. -done there

The dev_plus has a 16 second delay before power saving kicks in. current is 47mA and then will drop down to 11mA after the delay. I've done allot of testing ALLOT after it goes to sleep and it responds every single time, quickly! So, how can we get this delay down and still have it respond? That would then be perfect. In busy areas we may never sleep.

-The dev-plus also has the hardware delay, and I have 3 repeaters LOS to my house that are high level, and I must say this is just fantastic. It checks briefly to see if there is on air traffic before it transmits. very good.

[IoTThinks]

@towerviewcams Why there is 16s delay?
Dev_plus changed from 2 mins to 16s to activate the first sleep?

Then later, the repeater should sleep instantly, right?