First im going to say that I love boringproxy, this project made it finally possible for me to easily self host from home. Out of all the solutions iv tried this was the nicest to use, thank you for freeing me from my ISP’s CGNAT & lack of ipv6.
The problem
But lately iv rant into a big problem and its slowly turning into a show stopper, about a month ago I decided to self host a matrix home server, for the first few hours it worked fine, but as I joined rooms and traffic increased to it. The connections started to get bad, it got bad fast. Some requests take minutes to complete.
My understanding is that this is due to all my traffic for this matrix server being pushed through a single ssh tunnel, and that this is called the “TCP over TCP” problem. I know its not simple to solve, but I have some recommendations on how this issue could be helped.
How to solve this
The first solution I have thought of is to integrate another project into this one, there is a tool called sshuttle, and it helps solve the TCP over TCP problem. I believe adding support for it will not require a lot of changes to boringproxy and it will solve this large issue.
Another solution I will purpose that I dont think is as good, but I will still mention, is dynamically creating more then one ssh connection for busy domains and load balancing between them.
Hey @Yori, thanks for moving the post here from GitHub.
Since boringproxy is based on SSH tunnels, it actually isn’t susceptible to TCP-over-TCP issues.
However, since everything goes over a single TCP connection, that could certainly cause issues with lots of open connections, particularly if you start getting packet loss then head-of-line blocking could break everything. While creating a connection pool might help, I think the real solution would be to use a UDP-based protocol like QUIC. I’ve thought about this and want to do it eventually, but I’m not sure when I’ll be able to get to it.
I’m actually in the process of prototyping a new tunneling protocol. The first version will be based on SSH still, but it’s designed to be able work with multiple transports under the hood, so it will be much easier to add QUIC support than the current setup.
I have done a few speed tests and I pay for a symmetric connection. While my download speed is faster (about 200Mbps), I think 97.62Mbps should be plenty for a small matrix server, that interacts with a few dozen rooms and has under 5 users.
I dont know the first thing that comes to writing anything networking related and when I was googling the term “TCP over TCP” came up a few times and the symptoms seemed to match. So sorry about that.
As for Cloudflare & Ngrok, I dont like cloudflare & I feel it goes against the spirit of self hosting & Ngrok’s plans have a connection cap per minute that make it unusable.
Gotcha. Yeah I doubt your upload speed is the problem. Too many TCP connections tunneled over a single TCP connection is a good guess.
Unfortunately I’m not sure when boringproxy will have a UDP option. Not for a few weeks at least, and possibly longer. But this is an important use case. I’m glad you brought it to my attention since it gives a concrete reason to get QUIC working sooner.
As for solving your immediate problem, unfortunately you might not be able to find anything as simple as boringproxy. I think basically your only option today is to use WireGuard, either directly or another service built on it. Tailscale is great but not self-hosted (Headscale is working on that). You can also roll your own. Check the list as well for alternatives. I think frp has the option to use a TCP connection pool which might be enough for you.
The problem with WireGuard is it only sets up a VPN. You’ll still need to set up a reverse proxy on a public node. I recommend Caddy.
I’ll let you know if I’m able to get to the QUIC implementation in the near future.
In that case, I may just hold off on the matrix stuff, and just wait for when QUIC is added.
I have tried other stuff, its usually in one of the 3 camps. Too hard to understand and setup, Simple to setup but way to limited & dose what I want on paper, but too broken in practice to use.
Boringproxy is way too nice, its a full package that fully solves the problem of exposing self hosted software to the open web, it dose the https & and the reverse proxy web server to route subdomains to different ports too!
Sure setting up nginx with certbot is simple, but when its behind a layer of networking jank iv had issues with cerbot failing. I am spoiled and dont want to deal with any more networking jank, I find it soul crushing to deal with, iv lost days worth of time to that stuff.
Any way, please dont take this the wrong way, I know I am not entitled to your time and work, but ill just wait for QUIC. Also I already have domain names and dont need any thing new, so is there a way to support you that is not thro takingnames?
I hear you. Even for those of us who understand how the various pieces work, self-hosting is currently too much work. And it should be accessible to more people. You should be able to download an app on your computer, go through a quick OAuth flow to tunnel the app to a domain name, and immediately be able to give others access to the app through the web. And all this should happen inside a sandbox that limits the filesystem access, CPU/memory usage, network access, etc similar to docker but much much simpler. Basically like an inverse web browser for servers instead of clients. We have a long way to go, but I think on Linux at least this is all possible. Which means maybe we can get more people using Raspberry Pis.
Anyway since several people have asked, I finally set up a patreon, if you’re so inclined I appreciate it:
The plan is for TakingNames.io to offer a tunneling subscription similar to ngrok, but built on open protocols and focused on self-hosters instead of developers. This will eventually be the preferred way for people to support my various projects.