Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Low speed when using DE edition #1943

Open
bobyturk opened this issue Jan 11, 2024 · 38 comments
Open

Low speed when using DE edition #1943

bobyturk opened this issue Jan 11, 2024 · 38 comments

Comments

@bobyturk
Copy link

Hi, there!
i am using softether with 2 server connecting with cascade connection . the second server ( with enabled nat and outside of country ) when installed Stable edition working with good speed and there is no problem

but when you are using Developer Edition with same as same config with the stable dition the speed of loading media and ... comes down about 60%

anyone has idea for the reasons of that ? and any solution ?
@shakibamoshiri
Copy link

How did you run the speed test ?

@bobyturk
Copy link
Author

when i am using speed test " in same conditions " with my isp the se version give 40Mb/s with ping 120-140 but in DE version give 10-15 Mb/s

but speed test is not important . the media loading is slow when using DE version in 2nd server

i hav test the inside server using SE version and outside DE version , i got good speed but the connections are so unstable and use too much of memory and also cpu in 1 server

but when i want to test deffrences of speed its not depend 1st server , 2nd server when is DE version speed is low and media loading slowly but when usin SE version you can get good speed

@chipitsine
Copy link
Member

is there anything interesting on server ? what is cpu load ?

@bobyturk
Copy link
Author

for example cpu load when using both server DE version is 20% when using 1st server DE and 2nd server SE version cpu loads above 60% but the problem is memory usage , memory use full percentage in some times it could be 1 day or 3 days but it got 100% anyway in 1st server

@chipitsine
Copy link
Member

chipitsine commented Jan 13, 2024 via email

@bobyturk
Copy link
Author

no DE version cant use in single core it use so more cpu about SE version . in this case it was 4core 2.5GH cpu

but the problem is difference of media loading and speed testing between SE and DE version , why DE version in 2nd sever give more slow internet access

@chipitsine
Copy link
Member

chipitsine commented Jan 13, 2024 via email

@shakibamoshiri
Copy link

@chipitsine
I am facing the same issue
First I posted on the forum to see if some already experienced this (yesterday)
https://www.vpnusers.com/viewtopic.php?f=7&t=68783
You can look at mine, there are more details I provided

Also asked the question on "superuser" site
https://superuser.com/questions/1825649/slow-vpn-tunnels-speed-on-high-speed-network

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 •
edited

@shakibamoshiri , I did a look at vpnusers thread.

you do not provide grafana based graphs (but based on your observation, speed limit corellate with 100% cpu utilization ?)

can you elaborate in collecting cpu profiling ? the following steps should collect gperftools profiling data (previously you need to install appropriate libs/packages)

mkdir build



export CMAKE_FLAGS="-DCMAKE_SHARED_LINKER_FLAGS=-lprofiler"
./configure
make -C build

CPUPROFILE=se.prof CPUPROFILESIGNAL=31 build/vpnserver execsvc

killall -31 vpnserver
<make some measurements>
killall -31 vpnserver
 
  
 pprof --callgrind ./build/vpnserver se.prof.0 > se.perf

it links vpnserver against "libprofiler.so", also, vpnserver is started interactively, first "31" signal starts collecting profile data, second dumps collected data. with <make some measurements> I marked your commands that you use for performance measurement

I'll spend some time tomorrow reading thread on vpnusers

I'm interested in se.perf file (which can be visualized by kcachegrind, for instance)

@chipitsine
Copy link
Member

also, I doubt that vpncmd /tools /cmd trafficserver/trafficclient is somewhat relevant for comparision.

I'd start with iperf in tcp mode - it would allow to run on both tunnel and out of tunnel.

@chipitsine
Copy link
Member

also, we do not have an answer to - does the ISP throttle tunnels ?, it is something to ask your ISP

@bobyturk
Copy link
Author

i was tested in same as same conditions . in the same servers ! first installed DE version and test it then rebuild server and installed SE version . configs are same too . everything is same . and the SE version gives very high speed than DE version

@chipitsine
Copy link
Member

@bobyturk , please provide detailed repro steps.

i.e. how to make a setup on ourselves to reproduce your problem.

@chipitsine
Copy link
Member

@bobyturk , alternatively, we can have a look at your monitoring (maybe grafana?) to identify what is a bottleneck (if there any ?)

@bobyturk
Copy link
Author

in detaile :

i have 2 server . 1st and 2nd

i installed Developer in first server , clients will connect to this one by openvpn app . then made cascade connection to 2.nd server

first time installede Developer edition in 2.nd server and made cascade with first one . got some slow speed on media and social media also low speed in sppedtest.net

then rebuild the 2nd server and installed Stable Edition on it and again made the cascade with first one . this time speed was very good and speedtest resault was high near capacity of my local internet connection

the configs in 2 test are same . first server with disabled nat and client got ip from 2nd server , dns subnet and everything was same

un fortunately i dont have any monitoring system on my servers

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@bobyturk
Copy link
Author

yeah i can give backup file from both config DE and SE in 2nd server . but really i didnt change anything ! just installed SE

softether-vpnserver-v4.41-9782-beta-2022.11.17-linux-x64-64bit.tar.gz

and built DE version from source . just edited dns fro
8.8.8.8
8.8.4.4
and enabled nat with subnet 255.255.254.0 in both tests

but if you think the backup files will be helpful tell me how could send you ?

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@bobyturk
Copy link
Author

your email was masked .
i hav attached a photo . this is only thing i hav edited after installing boath DE and SE version , i didnt make any changes at all , not routing not bridge or anything else . just installed both version and edited dhc section .

if you want simulate install DE version with build from source in 1.st server

in second server fisrt time build from source for DE version and then install latest version of SE
photo_2024-01-16_02-10-12

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@bobyturk
Copy link
Author

why ? i have installed with thease ways so many times ! install DE version with build from source file and install SE version by downloadin latest version !

i am using some codes for intsalling each of them . in you could give me a email adres i can send you what i did step by step every time

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@bobyturk
Copy link
Author

i have sent tou you my installation step by step codes

@bobyturk
Copy link
Author

please tell me if you didnt got my email

@chipitsine
Copy link
Member

chipitsine commented Jan 15, 2024 via email

@bobyturk
Copy link
Author

bobyturk commented Jan 16, 2024 •
edited

i have checked another thing

we use this feature with cascade connection for personal and in the best situation 7-8 people got online toghether . but i have another softether server using without cascade connection and just use for autenthication of students for access the files and on that server i dont need internet or cascade connection , there are some times alot of online users toghether ,

today for testing i made a cascade connection on that softether server when over 100 users was online . then i checked , problem is exists to and was same with the server max 7 people be online toghether , then it is not depend on traffic or online users !

@chipitsine
Copy link
Member

chipitsine commented Jan 16, 2024 via email

@shakibamoshiri
Copy link

@chipitsine hi , I thought it could be simply solved but it seems I was wrong
I can do both pprof or having Graphana to see CPU graphs
it seems I should open a new issue and give steps that others can reproduce

@chipitsine
Copy link
Member

chipitsine commented Jan 16, 2024 via email

@chipitsine
Copy link
Member

chipitsine commented Jan 16, 2024 via email

@shakibamoshiri
Copy link

yes, maybe opening dedicated issue with full repro steps makes sense. what do you think ? alternatively, we can stay in this current issue if you have an idea how to link your findings with findings in this issue.. вт, 16 янв. 2024 г. в 09:35, Shakiba Moshiri @.>:
…
@chipitsine https://github.com/chipitsine hi , I thought it could be simply solved but it seems I was wrong I can do both pprof or having Graphana to see CPU graphs it seems I should open a new issue and give steps that others can reproduce — Reply to this email directly, view it on GitHub <#1943 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ5KUBW5LSTWOAAWOHW4LDYOY3UHAVCNFSM6AAAAABBXIT5SGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJTGI3TSNRWGQ . You are receiving this because you were mentioned.Message ID: @.>

it is up to you , whatever you say

first step

I started testing on my LAN where I have physical machines with enough resources , but I do not have proper modem! the modem speed is 10M :))) and testing with this does not show any useful statistics . lets go next

second step

I have VPSs with 1G port and started testing then noticed that speed is 10% max of servers speed

in this step some say "have you checked CPUs?" since I did not see any high loads with uptime and I did not think the cause could be the CPU.s
But know I should take the CPU into the consideration as well.
Following your recommendation for profiling and whatever is needed to check CPUs

Since it may take more time and more discussion I thought an new issue be better than mixing with someone else issue

@bobyturk
Copy link
Author

i realized my way to install softether is not normal or standard way . by the way if you want to test my problem please do install in your way and compare the connection and media download speed between 2 methods

1 . the first server is developer and the second server is developer too , connect two server with cascade connection
2 . the first server is developer and second server is stable edition , connect tow server with cascade connection

in this 2 method secure nat and dhcp is enabled in second server but username and passwords stored in first server

please test in this way

@chipitsine
Copy link
Member

@shakibamoshiri , let start a dedicated issue.

since you do not suspect CPU utilization (but Iwould consider some monitoring as well), I'd suggest to start with gperftools profiling and comparing results

@bobyturk
Copy link
Author

net conclusion !

DE version is slower than stabe edition

forgot about cascade connection and tunneling with softether

i tested another way , using tcp tunnel , Developer is at least 50% slower than Stable edition in internet browsing and media downloading for example in telegram and whast app

test situation :

1 server DE edition and another server Se edition , both of them tunneled from inside country via reverse tunnel in same server and same ip address , the resault DE version is more slow

@shakibamoshiri
Copy link

shakibamoshiri commented Jan 26, 2024 •
edited

@chipitsine compiling gperftools had some errors and I ignored this way. Next was using Grafana and rerun the test

here are the result

cpu-load-0

cpu-load-1

the first bump is iperf3 --time 1 --parallel 1 --reverse --client 192.168.99.1
the second one is iperf3 --time 10 --parallel 1 --reverse --client 192.168.99.1
the third one is iperf3 --time 100 --parallel 1 --reverse --client 192.168.99.1

Also 192.168.99.2 acts as client (iperf runs from here) and 192.168.99.1 acts as server (grafana installed there)

client 192.168.99.2 ===> server 192.168.99.1 + grafana
iperf3 client 192.168.99.2 ===> iperf3 server 192.168.99.1
cascade connections 32 TCP from server A to server B
both servers Debian VMs with 2 cpus 2 Gig ram

here are iperf3 output saved in txt files

iperf-100-raw-ipv4-to-ipv4.txt
iperf-100-se-tap-to-tap.txt

@shakibamoshiri
Copy link

CPU benchmarking using gnutls-cli --benchmark-tls-ciphers showed the CPU can handle more than 1Gbps , here is the benchmarck sample

Testing throughput in cipher/MAC combinations (payload: 1400 bytes)
                   AES-128-GCM - TLS1.2  0.86 GB/sec
                   AES-128-GCM - TLS1.3  0.81 GB/sec
                   AES-128-CCM - TLS1.2  0.29 GB/sec
                   AES-128-CCM - TLS1.3  0.29 GB/sec
             CHACHA20-POLY1305 - TLS1.2  176.63 MB/sec
             CHACHA20-POLY1305 - TLS1.3  172.30 MB/sec
                   AES-128-CBC - TLS1.0  0.22 GB/sec
              CAMELLIA-128-CBC - TLS1.0  59.48 MB/sec
           GOST28147-TC26Z-CNT - TLS1.2  21.49 MB/sec

Testing throughput in cipher/MAC combinations (payload: 16384 bytes)
                   AES-128-GCM - TLS1.2  1.59 GB/sec
                   AES-128-GCM - TLS1.3  1.62 GB/sec
                   AES-128-CCM - TLS1.2  0.37 GB/sec
                   AES-128-CCM - TLS1.3  0.36 GB/sec
             CHACHA20-POLY1305 - TLS1.2  198.16 MB/sec
             CHACHA20-POLY1305 - TLS1.3  196.77 MB/sec
                   AES-128-CBC - TLS1.0  0.26 GB/sec
              CAMELLIA-128-CBC - TLS1.0  63.37 MB/sec
           GOST28147-TC26Z-CNT - TLS1.2  22.33 MB/sec

Then trying hystrai-2 which uses QUIC , the speed was near 1.7Gbps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@chipitsine @shakibamoshiri @bobyturk