Feature #1648
closedVerify Multi-TRX support for osmo-bts-trx
100%
Description
OsmoTRX supports multiple transceivers for a long time, but let's make sure it still works with the new phy_link/phy_instance abstraction in OsmoBTS.
A resulting test case should be part of continuous integration setup, preferrably for all BTS hardware models that support such configurations.
Files
Checklist
- set-up osmo-bts-trx locally for testing
- do some manual testing with different codecs and gprs
- verify multi-trx functionality with multiplexing (-m)
- verify multi-trx functionality without multiplexing
Related issues
Updated by laforge about 8 years ago
- Assignee set to msuraev
- Priority changed from Normal to High
Updated by laforge about 8 years ago
- Checklist item set-up osmo-bts-trx locally for testing added
- Checklist item do some manual testing with different codecs and gprs added
- Checklist item verify multi-trx functionality (with two trx) added
please make sure that Lazlo joins you for setting up a network with osmo-bts-trx, so he can use this information to subsequently integrate osmo-bts-trx into the osmo-gsm-tester setup.
Updated by msuraev about 8 years ago
- Status changed from New to In Progress
Waiting for suitable hw.
Updated by laforge about 8 years ago
msuraev wrote:
Waiting for suitable hw.
A B210 will be with you on monday.
Updated by msuraev about 8 years ago
The following issue was observed:
trx_if.c:380 transceiver (phy0.1) rejected TRX command with response: 'RSP SETTSC 1 7'
Updated by msuraev about 8 years ago
Disabling SETTSC command for all phy except 0 allows bts to start but calls do not go through.
Test commands:
sudo chrt 20 ./Transceiver52M/osmo-trx -c 2
sudo chrt 15 ./src/osmo-bts-trx/osmo-bts-trx -t 2 -c ~/.config/osmocom/osmo-bts-mtrx.cfg -d DRTP:DCC:DRSL:DL1C
Tested on USRP B210.
Updated by msuraev about 8 years ago
- Related to Bug #1750: DTXu/DTXd support for osmo-bts-trx added
Updated by msuraev about 8 years ago
- Related to deleted (Bug #1750: DTXu/DTXd support for osmo-bts-trx)
Updated by msuraev about 8 years ago
- Blocks Bug #1750: DTXu/DTXd support for osmo-bts-trx added
Updated by laforge about 8 years ago
- Status changed from New to Stalled
waiting for a response from osmo-bts-trx maintainer
Updated by msuraev about 8 years ago
There are 2 ways to run multi-trx with osmo-trx:
1) use "-c 2" option (each arfcn get it's own channel)
2) use "-m" option (arfcns are multiplexed on same channel)
Unfortunately neither works for me. Also multiplexing (or maybe both) requires latest uhd master.
It's also unclear what kind of configuration changes are necessary for both modes both for osmo-bts and for openbsc. See on-going discussion in openbsc ML "multiple phy with osmo-trx".
Updated by msuraev about 8 years ago
- File ggsn.conf ggsn.conf added
- File openbsc.cfg openbsc.cfg added
- File openbsc-mc.cfg openbsc-mc.cfg added
- File osmo-bts.cfg osmo-bts.cfg added
- File osmo-bts-mc.cfg osmo-bts-mc.cfg added
- File osmo-pcu.cfg osmo-pcu.cfg added
- File osmo-sgsn.cfg osmo-sgsn.cfg added
- File README.txt README.txt added
Attaching sample configuration files and description provided by maintainer in ML.
Updated by msuraev almost 8 years ago
- Related to Bug #1775: LC15: No PDCH allocation across two TRX added
Updated by msuraev almost 8 years ago
- % Done changed from 0 to 10
With recent UHD_003.010.000.000-release, mTRX is working using following commands:
./src/osmo-bts-trx/osmo-bts-trx -c ~/.config/osmocom/osmo-bts-mtrx.cfg -t 2
a) same-channel multiplex
./Transceiver52M/osmo-trx -m -c 2 -l INFO
1) voice is working on any of trx0, trx1 (tested by configuring TCH/F channels on single trx only)
2) gprs working (tested by configuring PDCH channels on single trx only)
b) separate channels
./Transceiver52M/osmo-trx -c 2 -l INFO
neither voice nor gprs are working - phone either fails to camp to network or loses connection after some time.
Updated by laforge almost 8 years ago
- Target version set to osmo-bts-trx refresh
Updated by msuraev over 7 years ago
- % Done changed from 10 to 50
With external 10MHz reference, using B210 and UHD_003.009.005-0-unknown on ubuntu 16.10 following was observed:
sudo ./Transceiver52M/osmo-trx -l INFO -x -c 2
sudo ./src/osmo-bts-trx/osmo-bts-trx -c ~/.config/osmocom/osmo-bts-mtrx.cfg -t 2
1) both trx have PDCH and TCH/F timeslots: both test phones camp without problems, voice call is ok, voice slightly degraded.
2) TCH ts on trx0, PDCH on trx1 - iphone camps to network, acer fails to see it
3) TCH ts on trx1, PDCH on trx0 - both test phones camp to network, voice call goes through but quality is badly degraded: delays, skipped frames.
Following error was on OpenBSC side:
<0000> abis_rsl.c:1950 (bts=0,trx=1,ts=0,ss=0) ERROR INDICATION cause=SABM frame with information not allowed in this state in state=ACTIVE
On osmo-bts-trx:
<0006> scheduler.c:270 Prim for trx=1 ts=0 at fn=966278 is out of range, or channel already disabled. If this happens in conjunction with PCU, increase 'rts-advance' by 5. (current fn=966297)
was seen multiple times (no PCU were running)
also
<0006> scheduler_trx.c:477 TCH/F Transmitting 'bad FR frame' trx=1 ts=7 at fn=963274.
<0011> lapd_core.c:1511 N(S) sequence error: N(S)=6, V(R)=7
<0006> scheduler_trx.c:477 TCH/F Transmitting 'bad FR frame' trx=1 ts=7 at fn=963313.
<0011> lapd_core.c:1511 N(S) sequence error: N(S)=6, V(R)=7
were seen often.
On osmo-trx:
INFO 140188128442112 17:00:35.7 Transceiver.cpp:1021:driveTxFIFO: new latency: 0:8
INFO 140188126574336 17:00:35.8 Transceiver.cpp:1054:writeClockInterface: ClockInterface: sending IND CLOCK 940004
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1039:check_rx_md_err: An internal receive buffer has filled at 6.74236 sec.
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1039:check_rx_md_err: An internal receive buffer has filled at 6.74236 sec.
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1039:check_rx_md_err: An internal receive buffer has filled at 6.74991 sec.
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1039:check_rx_md_err: An internal receive buffer has filled at 6.74991 sec.
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1576:write: Skipping buffer data: timestamp=1833002 time_end=1827648
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1576:write: Skipping buffer data: timestamp=1833002 time_end=1827648
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1576:write: Skipping buffer data: timestamp=1833002 time_end=1827648
ERR 140188126574336 17:00:35.8 UHDDevice.cpp:1576:write: Skipping buffer data: timestamp=1833002 time_end=1827648
INFO 140188128442112 17:00:36.7 Transceiver.cpp:1031:driveTxFIFO: reduced latency: 7:7
INFO 140188126574336 17:00:36.7 Transceiver.cpp:1054:writeClockInterface: ClockInterface: sending IND CLOCK 940223
INFO 140188128442112 17:00:37.7 Transceiver.cpp:1031:driveTxFIFO: reduced latency: 6:7
INFO 140188126574336 17:00:37.7 Transceiver.cpp:1054:writeClockInterface: ClockInterface: sending IND CLOCK 940441
INFO 140188128442112 17:00:38.7 Transceiver.cpp:1031:driveTxFIFO: reduced latency: 5:7
INFO 140188126574336 17:00:38.7 Transceiver.cpp:1054:writeClockInterface: ClockInterface: sending IND CLOCK 940658
INFO 140188128442112 17:00:39.7 Transceiver.cpp:1031:driveTxFIFO: reduced latency: 4:7
Not sure if upgrading to newer UHD will help (there are no pre-build packages available for 16.10 ATM). Also it's unclear why swapping PDCH and TCH timeslots between TRXes affect network visibility for some phones.
In all tests ARFCN distance between TRX was 4.
Updated by msuraev over 7 years ago
Same configurations with GPRS and iphone:
1) works after some initial delay due to multitude of errors on OsmoPCU
20170105173301926 <0002> tbf.cpp:593 TBF poll timeout for FN=1914254, TS=6 (curr FN 1914315)
20170105173301927 <0002> tbf.cpp:648 - Timeout for polling PACKET CONTROL ACK for PACKET DOWNLINK ASSIGNMENT.
20170105173301927 <0002> tbf.cpp:1008 - Assignment was on PACCH
20170105173301927 <0002> tbf.cpp:1016 - No downlink ACK received yet
20170105173302403 <0002> tbf.cpp:593 TBF poll timeout for FN=1914358, TS=6 (curr FN 1914419)
20170105173302884 <0002> tbf.cpp:593 TBF poll timeout for FN=1914462, TS=6 (curr FN 1914523)
20170105173303014 <0005> bts.cpp:1318 Got RACH from TLLI=0xa8d7baea while TBF still exists. Release pending DL TBF
20170105173303014 <0002> tbf.cpp:447 TBF Software error: Pending downlink assignment. This may not happen, because the assignment message never gets transmitted. Please be sure not to free in this state. PLEASE FIX!
20170105173303014 <0002> bts.cpp:1347 MS supports EGPRS multislot class 10.
20170105173310078 <0002> bts.cpp:1347 MS supports EGPRS multislot class 10.
20170105173311345 <0002> tbf.cpp:593 TBF poll timeout for FN=1916295, TS=6 (curr FN 1916356)
20170105173311345 <0002> tbf.cpp:648 - Timeout for polling PACKET CONTROL ACK for PACKET DOWNLINK ASSIGNMENT.
20170105173311345 <0002> tbf.cpp:1008 - Assignment was on PACCH
20170105173311345 <0002> tbf.cpp:1016 - No downlink ACK received yet
20170105173311826 <0002> tbf.cpp:593 TBF poll timeout for FN=1916399, TS=6 (curr FN 1916460)
20170105173312303 <0002> tbf.cpp:593 TBF poll timeout for FN=1916503, TS=6 (curr FN 1916564)
20170105173312783 <0002> tbf.cpp:593 TBF poll timeout for FN=1916607, TS=6 (curr FN 1916668)
20170105173312846 <0002> tbf.cpp:947 TBF releasing due to PACCH assignment timeout.
20170105173312846 <0002> tbf.cpp:447 TBF Software error: Pending downlink assignment. This may not happen, because the assignment message never gets transmitted. Please be sure not to free in this state. PLEASE FIX!
20170105173340681 <0002> bts.cpp:1347 MS supports EGPRS multislot class 10.
2) iphone indicates gprs connectivity but it doesn't work
3) works with errors on OsmoPCU
20170105174205658 <0002> bts.cpp:1347 MS supports EGPRS multislot class 10.
20170105174207477 <0002> bts.cpp:1392 RX: [PCU <- BTS] TBF FIXME: Packet resource request
20170105174211889 <0004> tbf_dl.cpp:423 - Restarting at BSN 83, because the window is stalled.
20170105174214633 <0004> tbf_dl.cpp:423 - Restarting at BSN 106, because the window is stalled.
acer phone fails in all configurations.
Updated by msuraev over 7 years ago
- Related to Feature #1553: Multi-TRX support of PCU added
Updated by msuraev over 7 years ago
Checked with external bts tester: all tests pass except for leakage to neighbor frequencies which is expected and should not result in phones not seeing network. Btw, ports on olifantasia enclosure were printed incorrectly.
Notable thing is huge (40 db) difference between trx in transmission power - either smth is misconfigured by osmo-trx/osmo-bts-trx or hw failure (amplifier?). Not sure if it could be the reason for the effects described above.
Updated by ipse over 7 years ago
We've independently looked at issues with the 2nd channel in master version of osmo-bts-trx and found that "power" and "rxgain" commands are not sent to the 2nd channel (TRX1), because relevant VTY configuration commands and associated with a phy and not with an instance. Ivan is currently looking at this and some other issues with osmo-bts-trx VTY configuration like failure to set maxdly from a configuration file.
Updated by laforge over 7 years ago
Hi Alexander,
On Tue, Jan 17, 2017 at 04:34:12PM +0000, ipse [REDMINE] wrote:
We've independently looked at issues with the 2nd channel in master
version of osmo-bts-trx and found that "power" and "rxgain" commands
are not sent to the 2nd channel (TRX1), because relevant VTY
configuration commands and associated with a phy and not with an
instance. Ivan is currently looking at this and some other issues with
osmo-bts-trx VTY configuration like failure to set maxdly from a
configuration file.
We are happy to work on the related code for integration with the
osmo-bts phy_link data model and vty, if you could document what kind of
setting osmo-trx needs in what sequence.
Updated by kluchnikov over 7 years ago
Hi Harald,
Actually we have multiple related issues:
1. osmo-bts doesn't set RXGAIN and POWER for second channel of osmo-trx (TRX1)
We use the following osmo-bts config:
phy 0 instance 0 osmotrx rx-gain 10 osmotrx tx-attenuation 10 instance 1 osmotrx rx-gain 2 osmotrx tx-attenuation 12 bts 0 band 900 ipa unit-id 1801 0 oml remote-ip 127.0.0.1 rtp jitter-buffer 0 paging lifetime 0 ms-power-loop -50 timing-advance-loop settsc gsmtap-sapi ccch gsmtap-sapi pdtch trx 0 phy 0 instance 0 trx 1 phy 0 instance 1
And if we start osmo-trx with INFO log level, we can see that osmo-bts sends RXGAIN and POWER commands only for the first channel, but these parameters should be set for second channel too, otherwise TRX1 is configured incorrectly and can't work properly:
2017-01-18_09:05:21.73985 INFO 139850878134016 09:05:21.6 Transceiver.cpp:740:driveControl: command is CMD POWEROFF
2017-01-18_09:05:21.73997 INFO 139850878134016 09:05:21.6 Transceiver.cpp:740:driveControl: command is CMD POWEROFF
2017-01-18_09:05:21.74007 INFO 139850878134016 09:05:21.6 Transceiver.cpp:740:driveControl: command is CMD RXTUNE 904800
2017-01-18_09:05:21.94325 INFO 139850878134016 09:05:21.7 Transceiver.cpp:740:driveControl: command is CMD TXTUNE 949800
2017-01-18_09:05:22.35494 INFO 139850878134016 09:05:21.9 Transceiver.cpp:740:driveControl: command is CMD SETTSC 7
2017-01-18_09:05:22.35508 INFO 139850878134016 09:05:21.9 Transceiver.cpp:740:driveControl: command is CMD POWERON
2017-01-18_09:05:22.35714 INFO 139850878134016 09:05:22.0 Transceiver.cpp:740:driveControl: command is CMD SETRXGAIN 2
2017-01-18_09:05:22.35724 INFO 139850878134016 09:05:22.0 Transceiver.cpp:740:driveControl: command is CMD SETPOWER 12
2017-01-18_09:05:22.35797 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 0 5
2017-01-18_09:05:22.35807 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 1 7
2017-01-18_09:05:22.35815 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 2 7
2017-01-18_09:05:22.35923 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 3 7
2017-01-18_09:05:22.35947 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 4 7
2017-01-18_09:05:22.35955 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 5 7
2017-01-18_09:05:22.35961 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 6 7
2017-01-18_09:05:22.35969 INFO 139850878134016 09:05:22.1 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 7 1
2017-01-18_09:05:22.35972 INFO 139850876937984 09:05:22.2 Transceiver.cpp:740:driveControl: command is CMD RXTUNE 910000
2017-01-18_09:05:38.09708 INFO 139850876937984 09:05:22.3 Transceiver.cpp:740:driveControl: command is CMD TXTUNE 955000
2017-01-18_09:05:38.09735 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETTSC 7
2017-01-18_09:05:38.09739 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 0 1
2017-01-18_09:05:38.09742 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 1 1
2017-01-18_09:05:38.09744 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 2 1
2017-01-18_09:05:38.09746 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 3 1
2017-01-18_09:05:38.09748 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 4 1
2017-01-18_09:05:38.09750 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 5 1
2017-01-18_09:05:38.09754 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 6 1
2017-01-18_09:05:38.09757 INFO 139850876937984 09:05:22.5 Transceiver.cpp:740:driveControl: command is CMD SETSLOT 7 1
2. Incorrect handling of osmo-trx configuration parameters on phy and instance levels
As you can see above in osmo-bts config, we set "osmotrx rx-gain" and "osmotrx tx-attenuation" for each instance, because we should be able to set these parameters for each osmo-trx channel separatly.
But if we execute show running config command from vty, we can see:
phy 0 osmotrx ip 127.0.0.1 osmotrx fn-advance 20 osmotrx rts-advance 5 osmotrx rx-gain 2 osmotrx tx-attenuation 12 instance 0 instance 1 bts 0 band GSM900 ipa unit-id 1801 0 oml remote-ip 127.0.0.1 rtp jitter-buffer 0 paging queue-size 200 paging lifetime 0 uplink-power-target -75 gsmtap-sapi ccch gsmtap-sapi pdtch min-qual-rach 50 min-qual-norm -5 ms-power-loop -50 timing-advance-loop settsc trx 0 power-ramp max-initial 23000 mdBm power-ramp step-size 2000 mdB power-ramp step-interval 1 ms-power-control dsp phy 0 instance 0 trx 1 power-ramp max-initial 23000 mdBm power-ramp step-size 2000 mdB power-ramp step-interval 1 ms-power-control dsp phy 0 instance 1 end
As you can see osmotrx rx-gain and osmotrx tx-attenuation parameters are on the phy level, but they should be moved to the instance level.
Actually I think that it could be the cause of the first issue too.
3. Setting "osmotrx maxdly" and "osmotrx maxdlynb" parameters from config file doesn't work
We get following error from osmo-bts:
2017-01-18_09:55:32.14230 There is no such command.
2017-01-18_09:55:32.14244 Error occurred during reading below line:
2017-01-18_09:55:32.14249 osmotrx maxdly 20
But we can set these parameters from vty, so it seems like configuration handling issue.
If you need more tests or feedback from our side, please let us know.
Updated by ipse over 7 years ago
Update: Ivan has developed patches to fix these issues. We will submit to Gerrit after a bit more testing and clean up.
Updated by msuraev over 7 years ago
Please add me as reviewer for those patches on gerrit once they are ready to speedup review process.
Updated by msuraev over 7 years ago
- Checklist item set-up osmo-bts-trx locally for testing set to Done
- Checklist item do some manual testing with different codecs and gprs set to Done
Updated by msuraev over 7 years ago
- File open-bsc.cfg-mtrx1-f-gprs open-bsc.cfg-mtrx1-f-gprs added
- File open-bsc.cfg-mtrx0-f-gprs open-bsc.cfg-mtrx0-f-gprs added
- File osmo-pcu.cfg osmo-pcu.cfg added
- File osmo-bts.cfg-mtrx osmo-bts.cfg-mtrx added
Re-testing after recent updates to osmo-trx and osmo-bts-trx:
sudo ./Transceiver52M/osmo-trx -l INFO -x -c 2 -m linux; GNU C++ version 6.2.0 20161005; Boost_106100; UHD_003.009.006-release ... -- Asking for clock rate 3.200000 MHz... UHD Warning: The requested master_clock_rate 3.200000 MHz exceeds bounds imposed by UHD. The master_clock_rate has been forced to 5.000000 MHz. -- Actually got clock rate 5.000000 MHz. -- Performing timer loopback test... pass -- Performing timer loopback test... pass ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:548:set_master_clk: Failed to set master clock rate ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:548:set_master_clk: Failed to set master clock rate ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:549:set_master_clk: Requested clock rate 3.2e+06 ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:549:set_master_clk: Requested clock rate 3.2e+06 ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:550:set_master_clk: Actual clock rate 5e+06 ALERT 139849313479552 11:54:38.8 UHDDevice.cpp:550:set_master_clk: Actual clock rate 5e+06 ALERT 139849313479552 11:54:38.8 osmo-trx.cpp:530:main: Failed to create radio device
Doesn't work at all, so we have to use
sudo ./Transceiver52M/osmo-trx -l INFO -x -c 2
But in both cases I've tried (tch on on trx0, gprs on trx1 and vice versa) the phone fails to see the network:
./openbsc/src/osmo-nitb/osmo-nitb -c ~/.config/osmocom/open-bsc.cfg-mtrx0-f-gprs -l ~/.config/osmocom/hlr.sqlite3 -P -d DGPRS:DGSUP:DSNDCP sudo ./src/osmo-bts-trx/osmo-bts-trx -c ~/.config/osmocom/osmo-bts.cfg-mtrx -t 2 -r 99 sudo ./src/osmo-pcu -c ~/.config/osmocom/osmo-pcu.cfg -r 99
Am I missing smth?
Updated by msuraev over 7 years ago
- Blocked by Bug #1963: multiplexing trx is not working added
Updated by msuraev over 7 years ago
- Checklist item deleted (
verify multi-trx functionality (with two trx)) - Checklist item verify multi-trx functionality with multiplexing (-m) added
- Checklist item verify multi-trx functionality without multiplexing added
Updated by msuraev over 7 years ago
- Related to Bug #1524: PACCH on the wrong timeslot added
Updated by msuraev over 7 years ago
- Checklist item verify multi-trx functionality without multiplexing set to Done
Updated by msuraev over 7 years ago
- Status changed from Stalled to Resolved
- % Done changed from 50 to 100
Verified (using latest master) that both TRX work for gprs (with issues outlined in #1524) and voice, with and without multiplexing. To test this I've set all TS on one TRX to TCH/F on other to PDCH, than swapped this TS config between TRXs.
Dynamic TS configurations are tracked via separate issue #1853.
Updated by msuraev over 7 years ago
- Related to Bug #1853: validate dynamic TCH/PDCH support in osmo-bts-trx added