WSL 2.0: networkingMode=mirrored makes Docker unable to forward ports

Windows Version
Microsoft Windows [Version 10.0.22631.2338]
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2

Kernel Version
<IP_ADDRESS>-1
Distro Version
Ubuntu 20.04.6
Other Software
Docker version 24.0.6, build ed223bc
Repro Steps

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine (example)
Go to localhost:8080
Cannot connect
Try from WSL curl http//localhost:8080 same issue

Expected Behavior
To forward the port and be able to connect to my containers
Actual Behavior
Doesn't forward the port, so I cannot connect to my containers.
Diagnostic Logs
No response

Interestingly, the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.
Machine A:

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine
Go to localhost:8080
Cannot connect
Set firewall rules to allow 8080 inbound
Go to localhost:8080
Cannot connect

Machine B:

Go to A:8080
works

Same issue

Same problem here

Same problem here

In my case, Docker ports are not exposed to the host Windows machine at all after the upgrade to 2.0 even if I set networkingMode=NAT which is the default value. I can't see opened ports in the tcpview.exe utility. Previously, I was able to see them opened by the wslhost.exe process (or similar name).

same here

+1

+1

+1

@benhillis Any ideas/updates here?

@benhillis Any ideas/updates here?

same here

+1

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080 to the wslconfig, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

I think that's a workaround that would work... but I am waiting for someone from the WSL team to answer us here, did they get to know that is the issue, and if they have any plans to fix this?

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080 to the wslconfig, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

That seems to work. Nice find.

ignorePorts is misleading. What it does is making in this case port 8080 available to request inside the container only, but is not forwarded to Windows side. Look for more details at https://learn.microsoft.com/en-us/windows/wsl/wsl-config
If you run  get-nettcpConnection -LocalPort 8080 there's nothing listening on that port.
After starting the web server with docker run -d -p 8080:80 nginx:alpine requests inside the WSL container works:
zcobol@debian:~$ curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.25.2
Date: Mon, 25 Sep 2023 16:41:36 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 15 Aug 2023 19:24:07 GMT
Connection: keep-alive
ETag: "64dbd0d7-267"
Accept-Ranges: bytes

Requests from Windows side timeout with:
curl: (28) Failed to connect to localhost port 8080 after 21276 ms: Couldn't connect to server
From another server it work, as mentioned by @driver1998 :
tux@raspi:~$ curl -I <IP_ADDRESS>:8080
HTTP/1.1 200 OK
Server: nginx/1.25.2
Date: Mon, 25 Sep 2023 16:43:45 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 15 Aug 2023 19:24:07 GMT
Connection: keep-alive
ETag: "64dbd0d7-267"
Accept-Ranges: bytes

Just ran into this as well. AWS sam build --use-container doesn't work. Gives the same error
 Ports are not available: exposing port TCP <IP_ADDRESS>:5232 -> <IP_ADDRESS>:0: listen tcp <IP_ADDRESS>:5232: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

(The port number changes randomly every time)

Just ran into this as well. AWS sam build --use-container doesn't work. Gives the same error
 Ports are not available: exposing port TCP <IP_ADDRESS>:5232 -> <IP_ADDRESS>:0: listen tcp <IP_ADDRESS>:5232: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

(The port number changes randomly every time)
Based on this: https://github.com/aws/aws-sam-cli/blob/c5b9b1e399a1e5c938ef72934a14ede934e17bac/samcli/local/docker/container.py#L124-L125 I added every port from 5000-9000 to the ignoredPorts list. It's ugly, but for now it works.

It seems like it's generally a problem with WSL itself. I am looking for a fix soon, since the new network mode is so much useful but it's not useable (usefully) in the current state.

I ma having the same issue with apache. No changes other than added --experimental but now nothing works.
(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address <IP_ADDRESS>:80
no listening sockets available, shutting down
[no listening sockets available, shutting down](apache2.service: Control process exited, code=exited, status=1/FAILURE)
root@ACER-Nitro:/usr/sbin# lsof -nP -iTCP -sTCP:LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 308 systemd-resolve 14u IPv4 22507 0t0 TCP <IP_ADDRESS>:53 (LISTEN)
mysqld 385 mysql 21u IPv4 32886 0t0 TCP <IP_ADDRESS>:33060 (LISTEN)
mysqld 385 mysql 23u IPv4 27396 0t0 TCP <IP_ADDRESS>:3306 (LISTEN)
miniserv. 1867 root 5u IPv4 52863 0t0 TCP *:10000 (LISTEN)

Same problem

ignorePorts don't resolve this issue like in my mailpit can't access from windows side with ignoredPorts=8025,1025
docker run -d \
--name=mailpit \
--restart unless-stopped \
-p 8025:8025 \
-p 1025:1025 \
axllent/mailpit

ignoredPorts=8025,1025 work for me too. I had to use mailpit in docker
docker run -d \
--name=mailpit \
--restart unless-stopped \
-p 8025:8025 \
-p 1025:1025 \
axllent/mailpit

there seem to be two issues why Docker containers cannot connect from Windows.

ignorePorts works
Docker Desktop, probably

the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.
docker-ce that installed on Linux

temporary measures for 2 ....
disable iptables and use docker-proxy
/etc/docker/daemon.json
{
    "iptables": false
}

when using mirrored, the behavior seems to be different from the previous localhostforwarding.
use docker-proxy(listen on Linux)
windows  --->  linux(WSL)
  localhostforwarding (until)
ok    <IP_ADDRESS>   --->     <IP_ADDRESS>(lo)
ok    <IP_ADDRESS>   <---     <IP_ADDRESS>(lo)

  netowrkingMode=mirrored
ok    <IP_ADDRESS>   --->     <IP_ADDRESS>(loopback0)
ok    <IP_ADDRESS>   <---     <IP_ADDRESS>(loopback0)

interface is different, but the behavior remains the same.
use iptables(listeon on container)
windows  --->  linux(WSL) ---> Docker container
  localhostforwarding (until)
ok    <IP_ADDRESS>   ---> <IP_ADDRESS>(lo)    /   172.x.x.1(br-xxx)  --->  172.x.x.x(eth0)
ok    <IP_ADDRESS>   <--- <IP_ADDRESS>(lo)    /   172.x.x.1(br-xxx)  <---  172.x.x.x(eth0)

  netowrkingMode=mirrored
ok    <IP_ADDRESS>   ---> <IP_ADDRESS>(loopback0)/<IP_ADDRESS>(br-xxxx) --->  172.x.x.x(eth0)
                                          ~~~~~~~~~
ng                                                           <- - -  172.x.x.x(eth0)

via localhostforwarding(until), source address(Windows) was the docker network gateway (=pointing to linux).
via mirrored, source address is <IP_ADDRESS>.
for this reason that packet is returned to <IP_ADDRESS>(localhost on container) and does not reach Windows.
in the case of access from another node(PC), there is no problem because source is his address.

Same problem

Same problem :) was on the verge of reinstalling the whole system..

Any update?

ignorePorts  (with iptables disabled) seems to work but it doesn't allow you to connect to the container from the Windows Machine. However, it doesn't work with Redis (running inside docker for some reason, so I didn't have enough time to look it up).

ignorePorts  (with iptables disabled) seems to work but it doesn't allow you to connect to the container from the Windows Machine. However, it doesn't work with Redis (running inside docker for some reason, so I didn't have enough time to look it up).

+1 for priority. There is a large contingent of corporate devs who need docker and mirrored mode (vpn/sec reqs).
ignoredPorts really isn't an ideal solution. Hopefully we can get an official update on this item as it's already quite popular.

Came across this issue today when starting my RabbitMQ container.  Adding my +1 for visibility.

I tested with disable iptables for docker in /etc/docker/daemon.json and use docker-proxy instead. It works on my device.

Using ignorePorts does not work on my laptop.
I only installed docker in my wsl2 and didn't use docker desktop client in Windows.
Is that the reason?

使用ignorePorts 在我的笔记本电脑上不起作用。 我的意思是我仍然无法从 Windows 访问 docker 服务端口。 我只在我的wsl2中安装了docker，并没有在Windows中使用docker桌面客户端。 是这个原因吗？

添加下面配置到wslconfig
[experimental]
autoMemoryReclaim=gradual
networkingMode=mirrored
dnsTunneling=true
firewall=false
autoProxy=false
hostAddressLoopback=true

并且添加 "iptables": false 到 /etc/docker/daemon.json 就好了

@caaat233 it works indeed, thanks a lot! All it takes is to put {"iptables": false}.

Thanks for the work around. We are engaged with the Docker team to have better integration.

@keith-horton Why is this being closed when you are still actively working with Docker? Is there another issue that I can track? Is this workaround an official workaround suggested by MS? :) Are there any negative/downsides from turning iptables off in Docker w/ WSL?

Sorry, it was close for internal tracking of issues.
If it's more appropriate, we can keep this open as well.

" Are there any negative/downsides from turning iptables off in Docker w/ WSL?"
Sorry, I don't know Docker well enough to explain their various options. Do they have a forum to ask this?

Sorry, it was close for internal tracking of issues. If it's more appropriate, we can keep this open as well.

Does that mean a fix has been developed and we're just waiting on a new version? If so, awesome! Thank you!

Sorry, we are working with the Docker developers so their solution works well with WSL. We don't yet have a fix from our side.

Interestingly, the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.
Machine A:

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine
Go to localhost:8080
Cannot connect
Set Windows firewall rules to allow 8080 inbound
Go to localhost:8080
Cannot connect

Machine B:

Go to A:8080
works

After removing Docker Desktop and installing Docker directly on WSL, I got the containers to start.
However, Containers that expose ports are not accessible from the Windows machine or any device on the network
Yet, If you started a Node server for example, everything works like charm and can be accessed from Windows and any device on the network
Meanwhile lsof shows the following:
➜  ~ sudo lsof -i:8080
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
docker-pr 13090 root    4u  IPv4 189364      0t0  TCP *:http-alt (LISTEN)
docker-pr 13100 root    4u  IPv6 177996      0t0  TCP *:http-alt (LISTEN)
➜  ~ sudo lsof -i:3001
COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
node    3246 halim   26u  IPv6  21934      0t0  TCP *:3001 (LISTEN)

Not sure what does *:http-alt means but I think it's related to the problem!

Disabling iptables and using docker-proxy is a bad idea after all as docker-proxy has overhead

使用ignorePorts 在我的笔记本电脑上不起作用。 我的意思是我仍然无法从 Windows 访问 docker 服务端口。 我只在我的wsl2中安装了docker，并没有在Windows中使用docker桌面客户端。 是这个原因吗？

添加下面配置到wslconfig
[experimental]
autoMemoryReclaim=gradual
networkingMode=mirrored
dnsTunneling=true
firewall=false
autoProxy=false
hostAddressLoopback=true

并且添加 "iptables": false 到 /etc/docker/daemon.json 就好了

it works for me

So the two workarounds I see so far, both have their negatives but worth listing anyway. Correct any of my misunderstandings.
Option1 - without docker desktop
Edit %USERPROFILE%\.wslconfig file (create it, if doesn't exist) and add this section
[experimental]
networkingMode=mirrored
hostAddressLoopback=true

Restart WSL, then in your distro, edit /etc/docker/daemon.json and add
{
"iptables": false
}

Then sudo systemctl restart docker.
With this, both normal port usage like python3 -m http.server and docker port usage like docker run -p "8080:8080" --rm -t mendhak/http-https-echo:26 are accessible from Windows side. The disadvantage is that iptables is now disabled.
Option 2 - with Docker Desktop
Edit %USERPROFILE%\.wslconfig file (create it, if doesn't exist) and add this section
[experimental]
networkingMode=mirrored
hostAddressLoopback=true
ignoredPorts = 8000,8080

Restart WSL and ensure Docker Desktop is running. With this, the normal port usage stops working if it uses a port listed above. eg the Python http.server isn't accessible over 8000 anymore. Instead use a different port, like python3 -m http.server 8001. Docker ports will work as normal as long as the port is listed in the ignoredPorts above, like docker run -p "8080:8080" --rm -t mendhak/http-https-echo:26
The disadvantage here is you have to know which ports you'll be using with Docker. And it doesn't look like ignoredPorts accepts ranges of ports either so it can get pretty tedious.

Docker Desktop 4.26.0 Release Notes
For Windows
* Added support for WSL mirrored mode networking (requires WSL v2.0.4 and up).
* Added missing signatures on DLL and VBS files.

Thank you for posting @AzureZeng
Would appreciate a few more people checking to confirm, but I just did a quick test and it seems to be working better than before, hopefully no other issues appear 🤞
In %USERPROFILE%\.wslconfig I only put this and nothing else:
[experimental]
networkingMode=mirrored

and updated to Docker Desktop 4.26.0
I am now able to
python3 -m http.server

and also
docker run -p 8080:8080 --rm -t mendhak/http-https-echo:31

Both are visible from the Windows side in a browser.

Ok, that's cool, but I hope this issue does not get closed already. What about people who are not using Docker Desktop? (which is not free and neither open source)

Re the Docker update to 4.26.0 - WSL networking is working correctly for me in bridged mode. My Docker containers have stopped binding to my preferred NIC's IP though. Ports are exposed on localhost, but not my LAN IP. That will require some investigation and potentially, reconfiguration.

Ok, that's cool, but I hope this issue does not get closed already. What about people who are not using Docker Desktop? (which is not free and neither open source)

See above, set networkingMode=mirrored' and hostAddressLoopback=truein.wslconfig, then set "iptables": falsein/etc/docker/daemon.json`.
I think that there will not have more convenient solution than this, unless Docker team want to improve the experience that running Docker in WSL (without Docker Desktop).

Yes, I'm aware of that and that's what I am using. BTW, I don't need to set hostAddressLoopback=true and apparently it still works fine.
But I do expect some answer from either the WSL team or Docker as to if this is the recommended/final approach or not.

Docker version 24.0.6, build ed223bc

I always find the solution right after it's been fixed. So lucky.

Docker version 24.0.6, build ed223bc

I always find the solution right after it's been fixed. So lucky.

What's your solution? Could you share with me your configuration？

@LordHumphrey Update my quote. Just use docker desktop 4.26.0

I'm closing this issue out as it's now fixed! Please upgrade to docker desktop 4.26.0 to get the fix. Thank you all! :)

@craigloewen-msft, any words for the people who are not using Docker Desktop (which is not even free)? Is echo '{"iptables": false}' > /etc/docker/daemon.json the proper approach?

This issue should not be closed, the issue is still there if you're not using docker desktop. Iptables is more stable then userland proxy

even if iptables:true, unless you explicitly set userland-proxy: false, access from localhost will go through userlandproxy.
workaround if you don't want to set iptables:false
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

I think I'm not fully understanding the scenario. Here are the repro steps I tested in both docker-ce (So running as a Linux service in my WSL distro) and on Docker Desktop:

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine (example)
Go to localhost:8080 in Ede browser
Test to see if it's succeeding

This worked for both Docker Desktop and docker-ce when using mirrored mode. If you're still seeing this issue could you give me the repro steps where it's coming up for you?

That's the exact test scenario I have, and it's not working here:
https://github.com/microsoft/WSL/assets/29582865/8f89f3f5-4472-4cc4-a686-a909fedd5985

Im running docker 24.0.7 with docker desktop version 4.26.1 and i'm still experiencing the issue:
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.57
MSRDC version: 1.2.4485
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25880.1000-230602-1350.main
Windows version: 10.0.22631.2861

https://github.com/microsoft/WSL/assets/6466560/cb659445-dc55-45a9-ac1a-d9b4388bf8bd
Not sure if i'm doing something wrong but it feels very much not solved 😂 happy to help diagnose ;)

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine (example)
Go to localhost:8080 in Ede browser
Test to see if it's succeeding

Doing exactly the above (with Firefox instead of Edge 😉) results in a "The connection has timed out"-message.
It works fine if I either set "iptables": false or if I add --network host to the command (and then connect to localhost:80 instead of 8080).
Running docker-ce/jammy,now 5:24.0.7-1~ubuntu.22.04~jammy amd64 on Ubuntu 22.04 with kernel <IP_ADDRESS>-microsoft-standard-WSL2.

I'll reopen this then! For all the folks still seeing this, please collect WSL logs ( instructions here: http://aka.ms/wsllogs ) and then run these repro steps:

Change networkingMode to mirrored
docker run -d -p 8080:80 nginx:alpine (example)
Go to localhost:8080 in Ede browser
Test to see if it's succeeding

Then please attach the logs in a GitHub comment on this thread. Lastly please specify whether you are using Docker Desktop or docker-ce! Thank you!

Here are my logs.
I'm using docker-ce 24.0.7, docker desktop is not even installed.
WslLogs-2023-12-21_17-09-50.zip

Using Docker Desktop 4.26.1 / reproduced using @craigloewen-msft's provided steps:
WslLogs-2023-12-22_09-54-49.zip
Happy holidays tho 🤗

One Patch Tuesday later and it's now working for me (using Docker Desktop 4.26.0). 🤔 Windows 11 23H2 (build 22635.2771).
λ wsl --version
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22635.2771

I also re-tested docker-ce, but unfortunately I have the same result still.

@felipecrs , thank you for the trace. It looks like the wsl service is crashing shortly after startup. Can you follow these steps so that the wslservice.exe crash dmp is collected locally?
https://learn.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps?redirectedfrom=MSDN
(and stack overflow questions if you hit issues: https://stackoverflow.com/questions/30457570/windows-does-not-produce-full-crash-dumps-even-though-registry-is-set-correctly)
Thank you!

@MeikTranel , thanks for your trace. This is failing to reserve the TCP port for <IP_ADDRESS>:80 in the Linux container. Is that port already used on the Windows side? You can check it via Powershell.
e.g. this shows I have one listener on the IPv6 ephemeral address on port 80.
PS C:\Users\khorton> Get-NetTCPConnection -LocalPort 80
LocalAddress                        LocalPort RemoteAddress                       RemotePort State       AppliedSetting

::                                  80        ::                                  0          Listen
We have updated logging that is specific to Networking: https://github.com/microsoft/WSL/blob/master/diagnostics/collect-networking-logs.ps1
If you could stop WSL (wsl --shutdown), start the trace and repro, that would be very helpful.
Thanks!

However, I capture network logs as you mentioned:
WslNetworkingLogs-2024-01-03_10-40-57.zip

@MeikTranel , thanks for your trace. This is failing to reserve the TCP port for <IP_ADDRESS>:80 in the Linux container. Is that port already used on the Windows side? You can check it via Powershell.

The output is the same for me - i have process id 4 (aka the "system" process) listening on IPv6 :: Port 80 but you can see in the video i uploaded before that i switch between mirrored and not mirrored and the binding error inside httpd only happens after switching to mirrored networking here: https://github.com/microsoft/WSL/issues/10494#issuecomment-1864202014
I collected another set of logs with the updated scripts of yours - a note i'd like to make is that i changed the exposed ports in my compose to use ports that don't show up under Get-NetTCPConnection and it seems the failed binding happens within the container - its failing to bind 80 from within - windows ports do not seem to matter at that point. I'd understand that it'd fail to bind on the host port but the containerport should not be affected by the hostsystem having IIS running or something in a way that it doesnt even let me bind a non-standard host port - that would make working with other 3rdParty container images very awkward.
WslNetworkingLogs-2024-01-08_10-31-32.zip

Here are my logs.
I'm using docker-ce 24.0.7
WslLogs-2024-01-17_10-58-42.zip

Seems to work fine for me.
Nothing special has to be done.
Except according #11034, one of the settings "hostAddressLoopback = true" shouldn't be enabled.

I tested again, with latest Win11 and WSL. Problem remains the same here.
@bayeslearner are you using mirrored networkingMode? Which distro? iptables enabled in docker daemon?

I tested again, with latest Win11 and WSL. Problem remains the same here.
@bayeslearner are you using mirrored networkingMode? Which distro? iptables enabled in docker daemon?

Yes, mirrored networking mode.  Ubuntu 22.04.  I'm using docker-desktop , not docker that can also be directly installed on Linux.

I see. And it was already known that docker desktop was working. :)

After run the following command in PowerShell window with admin privileges to Configure Hyper-V firewall settings to allow inbound connections:
Set-NetFirewallHyperVVMSetting -Name ‘{40E0AC32-46A5-438A-A0B2-2B479E8F2E90}’ -DefaultInboundAction Allow
I can not only connect to the container from Windows using localhost, but also from other computer in the same lan using the ip address of the Windows.
Reference：
mirrored-mode-networking

When I set networkMode=mirrored, I found that the service on wsl can be accessed by windows through <IP_ADDRESS>, but cannot be accessed through LAN ip. At this time, docker forwarding port is normal, but docker cannot be accessed through LAN ip. to services on wsl. So I set hostAddressLoopback=true. At this time, Windows can access all services on WSL through LAN IP, but the docker port cannot be forwarded, and WSL services cannot be accessed through LAN IP. In both cases, docker can ping the LAN IP.
I use docker desktop 4.26.1

Hi there. By default, you can only communicate to/from the host and container with <IP_ADDRESS>. But if you set [experimental] hostAddressLoopback = true, then you can also use the host IP address.  https://learn.microsoft.com/en-us/windows/wsl/wsl-config#experimental-settings

None of these workaround works.   Juse change back to NAT and waiting for fix.

Just to update, this issue is still present with WSL <IP_ADDRESS> and docker 25.0.2.

After updating, the docker container still cannot access services open on wsl.

This is still an issue.

temporary measures for ....2 disable iptables and use docker-proxy
/etc/docker/daemon.json
{
    "iptables": false
}

it works for me,but this causes me to not be able to use iptables and I'm still waiting for a real fix. Also, I've heard that Docker Desktop 4.26 and later adds support for mirrored network mode.

temporary measures for ....2 disable iptables and use docker-proxy
/etc/docker/daemon.json
{
    "iptables": false
}

it works for me,but this causes me to not be able to use iptables and I'm still waiting for a real fix. Also, I've heard that Docker Desktop 4.26 and later adds support for mirrored network mode.

docker iptables: false confirmed working.
For anyone trying to connect services in WSL2 to connect to Android Emulator running in windows and have access to docker containers, this workaround works. Make sure to unset ADB_SERVER_SOCKET if set previously in WSL2 guest.

https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0
how about this?

This worked!!

@LeorGreenberger
what distribution are you using?

@LeorGreenberger what distribution are you using?

Debian 12

@LeorGreenberger
i cannot reproduce it.
however, there were some parts where the escape was missing, so I fixed it.
could you please try this?
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

@shigenobuokamoto  This works! I reloaded, stopped and started a bunch of times and I don't see any escape errors. On stopping the service, I get the following warning, but I am guessing this is expected:
Mar 07 13:57:29 REMLT28981 systemd[1]: Stopping network-mirrored.service...
Mar 07 13:57:29 REMLT28981 sh[221292]: # Warning: table ip nat is managed by iptables-nft, do not touch!
Mar 07 13:57:29 REMLT28981 systemd[1]: network-mirrored.service: Deactivated successfully.
Mar 07 13:57:29 REMLT28981 systemd[1]: Stopped network-mirrored.service.

@shigenobuokamoto Your fix works!!! I ran your service start command as root at the command line. Not tested the stop service yet. Our use case is a windows fat client connected to a backend  composed of a bunch of containers started with docker compose in WSL2.
You saved my life!

You saved my life!

I wonder why iptables: false would not be enough for you.

@felipecrs

I wonder why iptables: false would not be enough for you.

most issues can be resolved by modifying daemon.json. however, the usability may change by not using iptables.
bigger issues is that cannot use docker unless setting daemon.json.
my script is just a sample implementation workaround, but if it has a reasonable effect without any negative effects on everyone, i hope that i will be able to propose its inclusion in /init.
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0
my script is very simple, just apply the following rules to nftables.
table ip nat {
    chain WSLPREROUTING {
        type nat hook prerouting priority dstnat - 1; policy accept;
        iif "loopback0" ip daddr <IP_ADDRESS> counter dnat to <IP_ADDRESS>;
    }
}

this chain is processed immediately before PREROUTING, and will DNAT destination of the packet that arrives from loopback0 interface to <IP_ADDRESS> to <IP_ADDRESS>. it may seem like a wasteful process, but adding this action breaks the prerouting hook and disables any Docker rules set in the PREROUTING chain.
using userland-proxy only for access from localhost is the same as Docker's default behavior.
to: who have already used it
i have made some changes from the previous one,
before
  add chain   ip nat PREROUTING { type nat hook prerouting priority dstnat; policy accept; };
  insert rule ip nat PREROUTING iif loopback0 counter accept comment mirrored;

after
  add chain   ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; };
  insert rule ip nat WSLPREROUTING iif loopback0  ip daddr <IP_ADDRESS> counter dnat to <IP_ADDRESS> comment mirrored;

both are methods to prevent the PREROUTING NAT rule.
there is no change in behavior when using with Docker, but i think this is safer because it does not affect the chain managed by iptables.
if you like, please try this as well.

@felipecrs  When you disable iptables manipulations by docker, you can't have isolated networks for your docker compose stacks.
It has the same effect as adding --network host to each container. This is acceptable for simple use cases where you start a few unrelated containers.
Imagine that I have an application composed of two containers: a web server (Tomcat port 8080) and a database (oracle port 1521). I create a compose.yaml file containing two services : tomcat and oracle. And I start the pair of containers with docker compose up.
In my compose.yaml, I only expose the web server port to the outside world. The application running in the Tomcat container refers to the database as "oracle port 1521" (oracle is the name of the service in compose.yaml).
When iptables manipulation is disabled, the Oracle database binds to <IP_ADDRESS>:1521. If you try to start a second compose stack, Oracle will complain that the port is already in use.
If iptables manipulation is enabled (default), each stack will have its own network ; e.g. "<IP_ADDRESS>/16" and "<IP_ADDRESS>/16" and Oracle in the first stack will bind to <IP_ADDRESS>:1521 wile the one in the second stack will bind to <IP_ADDRESS>:1521. In the tomcat container of the first stack, the oracle host name refers to <IP_ADDRESS> while in the Tomcat container the oracle host name refers to <IP_ADDRESS>.
The fix proposed by Shigenobu NAT the packets arriving on the loopback0 interface (the one the Windows packets come from) before forwarding them on the docker0 interface (the one Docker bridges its networks). See his comment of Sep, 30 2023 for the explanation of the problem.
If you want more details on the networking of Docker, see https://docs.docker.com/network/network-tutorial-standalone/ and https://docs.docker.com/network/network-tutorial-host/.
So, disabling iptables manipulation is fine for some simple use cases but you will quickly get stuk.

Thanks both, very useful information.

Although networkingMode=mirrored is great, I hope it does not get promoted to default without having this issue resolved first. 😅

Just to renew this issue, I tested again with the just released WSL 2.2.1, but same problem still occurs.

Still an issue with v4.28.0

Hello,
We're trying to have a setup of WSL working behind a corporate VPN (Cisco Any Connect).
On Win11/WSL2, so far we have to use networkingMode=mirrored to get WSL to be able to access the Internet. (works fine)
But we're also trying to use docker inside WSL. Now, with this setup we can't access our containers that exposes specific ports from our Windows machine. For example portainer on port 9000.
This issue cause us to not be able to use Docker on WSL behind a corporate VPN.
[wsl2]
vmIdleTimeout=-1
autoProxy=false
networkingMode=mirrored
dnsTunneling=true

Version WSL : <IP_ADDRESS>
Version du noyau : <IP_ADDRESS>-2
Version WSLg : 1.0.60
Version MSRDC : 1.2.5105
Version direct3D : 1.611.1-81528511
Version de DXCore : 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Version de Windows : 10.0.22621.3296

If there is any news on this issue I'll be greatful to hear from you guys.
watching :eyes:

Hello,
We're trying to have a setup of WSL working behind a corporate VPN (Cisco Any Connect).
On Win11/WSL2, so far we have to use networkingMode=mirrored to get WSL to be able to access the Internet. (works fine)
But we're also trying to use docker inside WSL. Now, with this setup we can't access our containers that exposes specific ports from our Windows machine. For example portainer on port 9000.
This issue cause us to not be able to use Docker on WSL behind a corporate VPN.

@jsayer101 I don't understand your configuration.  Could you draw a picture of it?
If you want to access ports on your WSL from other hosts, does your windows firewall authorize it?
Have you tried the startup script propose above?

Yea something is out of order. Corporate firewall wouldn't prevent you from accessing WSL on your own computer.
Anyway, add this to your WSL OS (https://github.com/microsoft/WSL/issues/10494#issuecomment-1951848642) and add the following settings to your .wslconfig. It's similar to what you have but with additional flags. This is working fine across all of my Win11 PCs
[wsl2]
networkingMode=mirrored
firewall=false
dnsTunneling=true

[experimental]
useWindowsDnsCache=true
hostAddressLoopback=true

@shigenobuokamoto

could you please try this?
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

Your hack works fine for me. Thank you so much!.. Just one small issue.. it breaks other IPv6 access going to internet for me. My docker is not configured for IPv6. So.. could you help me understand what parts of the gist should be removed so that it does not mess with ipv6 routing stuff?
❯ ssh -6<EMAIL_ADDRESS>-vv
OpenSSH_8.9p1 Ubuntu-3ubuntu0.6, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /home/vj/.ssh/config
debug1: /home/vj/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolving "xxx.com" port 22
debug1: Connecting to xxx.com [2406:0000:000:0000:000:0000:0000:c711] port 22.
debug1: connect to address 2406:0000:000:0000:000:0000:0000:c711 port 22: Connection timed out
ssh: connect to host xxx.com port 22: Connection timed out

With normal mirrored mode, this connectivity used to work. I can still connect fine via same command from windows host. just not from WSL.

@dharapvj

Your hack works fine for me. Thank you so much!.. Just one small issue.. it breaks other IPv6 access going to internet for me. My docker is not configured for IPv6. So.. could you help me understand what parts of the gist should be
removed so that it does not mess with ipv6 routing stuff?

if you exclude the line containing "ip6", IPv6 support will be disabled.
$ grep -v ip6 network-mirrored.service > mirrored-noip6.service

however, since this support controls the source port for sources other than WSL container (= the traffic routed by WSL container), it is strange that access via SSH is no longer possible.

Waiting for WSL to fix this issue.

same issue

same issue, please MS, help us!

Hi there.
We have a few questions as we narrow down what is going on.
The below is referencing the troubleshooting docs in https://github.com/MicrosoftDocs/WSL/blob/main/WSL/troubleshooting.md
Sadly not all changes have been replicated yet to https://learn.microsoft.com/en-us/windows/wsl/troubleshooting :(

can folks who are hitting this confirm they do not have a separate Network Manager running?
see "Docker container issues when its Network Manager is running" in the above troubleshooting article.

have folks who have hit this looked at the troubleshooting article we wrote? Can you give us feedback if that works for you, or if that cannot be done for a reason with your scenario?
see: "Docker container issues in WSL2 with Mirrored networking mode enabled when running under the default networking namespace"

Thanks for your help!
We're trying to understand exactly what is going on.

Hi there.
We have a few questions as we narrow down what is going on.
The below is referencing the troubleshooting docs in https://github.com/MicrosoftDocs/WSL/blob/main/WSL/troubleshooting.md
Sadly not all changes have been replicated yet to https://learn.microsoft.com/en-us/windows/wsl/troubleshooting :(

can folks who are hitting this confirm they do not have a separate Network Manager running?
see "Docker container issues when its Network Manager is running" in the above troubleshooting article.
have folks who have hit this looked at the troubleshooting article we wrote? Can you give us feedback if that works for you, or if that cannot be done for a reason with your scenario?
see: "Docker container issues in WSL2 with Mirrored networking mode enabled when running under the default networking namespace"

Thanks for your help! We're trying to understand exactly what is going on.

In my scenario, I don't have network-manager.service:

I use docker-ce, not docker-desktop. My case:
From WSL:  curl localhost:8080 OK (service 8080 running in windows host)
From container docker: curl localhost:8080 = Couldn't connect to server - OK localhost here is the container loopback.
From container docker: curl <IP_ADDRESS>:8080 = Couldn't connect to server.  - <IP_ADDRESS> is IP of WSL and Windows Host.

seems like a new issue that is different from the current flow.

From container docker: curl <IP_ADDRESS>:8080 = Couldn't connect to server. - <IP_ADDRESS> is IP of WSL and Windows Host.

this is the WSL  ip address from the container's perspective, not WSL and Windows Host.
then, i tried to think of a way to implement this method.
with these steps, you can connect <IP_ADDRESS>:8080 from the container to 8080 on the Windows host.

enable the target port to overlap with Windows Host

.wslconfig
[experimental]
ignoredPorts=8080

<IP_ADDRESS>:8080 -> <IP_ADDRESS>:8080 with socat

$ sudo socat TCP-LISTEN:8080,bind=<IP_ADDRESS>,reuseaddr,fork TCP:<IP_ADDRESS>:8080

seems like a new issue that is different from the current flow.

From container docker: curl <IP_ADDRESS>:8080 = Couldn't connect to server. - <IP_ADDRESS> is IP of WSL and Windows Host.

this is the WSL ip address from the container's perspective, not WSL and Windows Host.
then, i tried to think of a way to implement this method. with these steps, you can connect <IP_ADDRESS>:8080 from the container to 8080 on the Windows host.

enable the target port to overlap with Windows Host

.wslconfig
[experimental]
ignoredPorts=8080

<IP_ADDRESS>:8080 -> <IP_ADDRESS>:8080 with socat

$ sudo socat TCP-LISTEN:8080,bind=<IP_ADDRESS>,reuseaddr,fork TCP:<IP_ADDRESS>:8080

Sorry @shigenobuokamoto , I mixed up the problems.

Regarding the <IP_ADDRESS> ip, it's from WSL and Windows Host. It's different because it's the IP when I'm connected to the VPN. I also tried the IP of the Wi-fi network interface but to no avail.

But I had the same port forwarding problem with my docker-ce and networkingMode=mirrored.
docker run --rm -it -p 8082:8082 alpine nc -p 8082 -l
Trying to connect via the browser or curl on the Windows machine doesn't work (http://localhost:8082). However, if I connect via WSL, it works.
I set {"iptables": false} in daemon.json and it worked. Without the iptables modification it only worked with --network-host.
Now I think we're talking about the same thing in this issue.

@AlencarGabriel, sorry if I offended you
mirrored network has a rather special structure. it would be nice to have an official document, but i have written my thoughts below, so please use them as a reference.
https://github.com/microsoft/WSL/discussions/11326#discussioncomment-8874149
for connections from Windows Host to docker on WSL, you can control traffic to the container using daemon.json or nftables. currently, it is not possible to create the same behavior as native linux without controlling it.
<IP_ADDRESS> is its own address to linux. since it processes the traffic that comes here by itself, it rejects connect to 8080 that is not open. but Linux itself can connect to <IP_ADDRESS>:8080 (on Windows Host). this is where the magic of mirrored network.
this issue and similar docker-related issues pointed to connect from Windows host to container service.
in that sense, i think that connecting from container to Windows poses a new issue. and a different (ad-hoc) solution is required than what has been shown so far.

Interestingly, the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.
 I turned off the firewall and forwarded the ports, but I just don't know why the telnet connection won't connect. Later I saw what you said and found out that it doesn’t work locally.

I have the following in my .wslconfig
[wsl2]
memory=20GB

networkingMode=mirrored
dnsTunneling=true
firewall=true
autoProxy=true

[experimental]
sparseVhd=true
# requires dnsTunneling
bestEffortDnsParsing=true
useWindowsDnsCache=true

I have a wireguard VPN running on my Windows host and I have no extra config in my wsl or docker, except in my /etc/sysctl.conf (because certain apt repos are utterly unusable using ipv6, at least over my Wireguard vpn):
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1

I have no extra Windows Firewall config and I have absolutely no issues accessing the web from my host or wsl, and docker behaves exactly how I would expect it too (I can access everything from both the host and wsl). It works perfectly whether I turn the VPN on or off.
I am using vanilla docker installed from the instructions https://docs.docker.com/engine/install/ubuntu/. This also works perfectly with the new Ubuntu 24.04 LTS (Ubuntu Preview in the Store).

What is your distribution? I'm tried configuring my WSL like yours, but I still can reproduce the issue.
https://github.com/microsoft/WSL/assets/29582865/38338c55-979d-4a25-a7d5-57b57497b343
❯ wsl --version
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.3447

BTW I can confirm that @shigenobuokamoto's workaround works like a charm.

To follow-up on this thread, we are actively working to address this. We are considering solution such as that shown by @shigenobuokamoto  earlier in this thread. We are currently considering options and possible side-effects for the multitude of ways Linux might be configured by users.
Thanks everyone for the detailed feedback!

Wow, I couldn't get it to work in mirrored mode for 4 hours, then it randomly worked after restarting services, killing processes, restarting wsl, running docker desktop as admin etc.
The next day I couldn't open docker desktop, the engine would just not start, for 2 hours.
Modifying my config finally worked:
[wsl2]
I added:
firewall=false
dnsTunneling=true

so my config now looks like this:
networkingMode=mirrored
# localhostForwarding=true
firewall=false
dnsTunneling=true

[experimental]
autoMemoryReclaim=gradual
sparseVhd=true

I'm using docker-ce, behind a corporate proxy/VPN with mirrored networking (no other wsl settings) and the only thing that has worked so far that allows access to docker containers ports from the windows host side is to not use iptables on the dockerd side.

Just got this running recently with @underlines config.
Forgot that mirrored mode requires a signed in user, which fails if offline (even signed in and online before reboot), which is beyond frustrating, and will cause me to recommend against docker desktop version of docker.

Network Mode mirrored does not work for me with docker. Looks like port 2375 is only reachable via IPv6 (netstat - nl | grep 2375 only returns a mapping for 'tcp6').
Why I can reach http://localhost:2375/version via Browser, e. g. Testcontainers does not find a docker environment (despite DOCKER_HOST set to 'tcp://localhost:2375').
Even after 8 years, docker is just a nightmare with WSL(2).I am literally the only dev using Windows anymore on my team, everyone else is using Macs now where it just works.

I keep coming back to this and @shigenobuokamoto's solution is the only thing that works for me.

After updating WSL I'm running into the same issue that networkingNode=mirrored no longer works.  If I set it to NAT I can get to the WSL/docker container from the host machine but I can no longer get to it from the network at all.

This issue seems to have been resolved for me with WSL <IP_ADDRESS>. Can anyone else confirm this so we can close the issue?

This issue seems to have been resolved for me with WSL <IP_ADDRESS>. Can anyone else confirm this so we can close the issue?

No. I still need to disable iptables to make it work.
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-2
WSLg version: 1.0.61
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26091.1-240325-1447.ge-release
Windows version: 10.0.22631.3737

hello, thanks for your patience while we investigated this
we are talking with Docker to address this in their product, but in the meantime, you can run these commands to update the Docker nftables rules to allow localhost port forwarding to work
step 1) Run "sudo nft -a list chain ip nat DOCKER". Example output shown below.
table ip nat {
chain DOCKER { # handle 4
iifname "docker0" counter packets 0 bytes 0 return # handle 7
iifname != "docker0" meta l4proto tcp tcp dport 8080 counter packets 0 bytes 0 dnat to <IP_ADDRESS>:80 # handle 12
}
}
step 2)
using the handle of the second rule in the DOCKER chain from step 1 (the rule that includes the "dnat" keyword), run the following command:
(please make sure to use the handle from your output in step 1, as the handle number can change)
sudo nft -a delete rule ip nat DOCKER handle 12
step 3) Run the following command. Make sure that the ports and IP address match your output from step 1
sudo nft add rule ip nat DOCKER iifname != "docker0" iifname != "loopback0" meta l4proto tcp tcp dport 8080 counter dnat to <IP_ADDRESS>:80

This issue seems to have been resolved for me with WSL <IP_ADDRESS>. Can anyone else confirm this so we can close the issue?

I guess I should also specify I am not using docker desktop. I installed Docker directly in WSL.

@CatalinFetoiu
it may not be practical to modify the nft rules set by Docker every time.
instead, if you run the following two lines, traffic from the Windows host will be handled by <IP_ADDRESS> (= docker-proxy).
sudo nft  add chain   'ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; }'
sudo nft  insert rule 'ip nat WSLPREROUTING iif loopback0 ip daddr <IP_ADDRESS> counter dnat to <IP_ADDRESS>'

i have been using this rule for a long time and it does not seem to have any side effects.

@shigenobuokamoto's solution continues to be rock-solid over many months of 10h/day usage. Write yourself a personal VM setup guide and just copy paste their stuff.

I thought I was only having this issue with docker but its really any wsl instance.  I can connect between WSL instances but windows to wsl does not work.  I am running in mirrored mode.  This works fine on 2 of the 3 computers I have available to me.
I've tried a clean reinstall of wsl on this problem machine but nothing works. there is no configuration/version difference between these machines.

I have no idea how to explain this one... but I have mirrored mode working again after playing around with wireshark in an effort to see where the TCP/IP stack was breaking down.  after resetting promiscuous mode on all of the network devices the windows to wsl communications started working again.  This was after multiple reinstalls of wsl, hyper-v, docker as well as all of the different firewall scripts in the related issues and resetting the ip stack in windows.  My best guess is when I switched from cabled ethernet to my wifi enabled hotspot after a network outage something in the network stack was flipped breaking the communications between windows and wsl.  But after toggling promiscuous mode on all of the devices somehow it was set back to the manner required to make "mirrored" work correctly.
It would be nice to have more information but I have no idea how to reproduce the breaking issue or the fix.  I was actually rather annoyed when I found the stack was working again as I did not make any intentional changes.  I was just playing in wireshark trying to find the curl http://localhost with it timing out... just to try it again and have it start returning results.
Good luck to the person that gets to hunt down the issue in the TCP/IP stack.

More interestingly... it only works when I have wireshark doing a local loopback capture.

@CatalinFetoiu it may not be practical to modify the nft rules set by Docker every time. instead, if you run the following two lines, traffic from the Windows host will be handled by <IP_ADDRESS> (= docker-proxy).
sudo nft  add chain   'ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; }'
sudo nft  insert rule 'ip nat WSLPREROUTING iif loopback0 ip daddr <IP_ADDRESS> counter dnat to <IP_ADDRESS>'

i have been using this rule for a long time and it does not seem to have any side effects.

seems still have side efforts:

win host --> service in wsl, ok
wsl --> service in win host, ok
win host --> service in wsl docker, ok through localhost/<IP_ADDRESS>
wsl docker --> service in win host, no

above bullet 4 in WSL2 NAT workable.

@itispip

wsl docker --> service in win host, no

https://github.com/microsoft/WSL/issues/10494#issuecomment-2050817875
this is the way I thought about it before.
it seems difficult to handle this with nftables, so i am using socat to relay traffic.

Hi folks, we believe we've found a fix for this that needs to be added to the Docker engine. We've filed a bug for it and are working with the docker team to investigate adding it in: https://github.com/moby/moby/issues/48056

Hi folks, we believe we've found a fix for this that needs to be added to the Docker engine. We've filed a bug for it and are working with the docker team to investigate adding it in: moby/moby#48056

As a note, this is not just docker.  Even without docker and resetting WSL entirely this still occurs. mirrored mode does not work correctly with regular WSL images such as Ubuntu.

There are several here all describing the same issue.  Opening more is not going to fix it any better.

But I can tell you this issue will very likely be closed as soon as the Docker CE issue is resolved.

As already noted above I have cross mentioned the issue to try to get an actual fix instead of just a pile of work arounds.
https://github.com/microsoft/WSL/issues/11600

Reference

Well it looks like the issue wasn't actually fixed for me. Not sure what I did or what changed to fix and then unfix the issue.
Also when running the sudo nft -a list chain ip nat DOCKER I get a new rule for each port I forward in the container.
Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
chain DOCKER { # handle 10
iifname "docker0" counter packets 0 bytes 0 return # handle 29
iifname "br-c360e1083765" counter packets 65 bytes 3981 return # handle 13
iifname != "br-c360e1083765" tcp dport 8085 counter packets 0 bytes 0 dnat to <IP_ADDRESS>:54002 # handle 30
iifname != "br-c360e1083765" tcp dport 10001 counter packets 0 bytes 0 dnat to <IP_ADDRESS>:10000 # handle 32
iifname != "br-c360e1083765" tcp dport 9001 counter packets 0 bytes 0 dnat to <IP_ADDRESS>:9001 # handle 34
iifname != "br-c360e1083765" udp dport 1024 counter packets 0 bytes 0 dnat to <IP_ADDRESS>:1024 # handle 36
}
}
Where it looks like the c360e1083765 is the ID of the container. Not sure if this is different because I'm using the docker engine over docker desktop.

although this issue occurs with Docker, i believe it is a compatibility issue with WSL.
the following parameters set by mirrored networking are unexpected from Docker.
net.ipv4.conf.loopback0.route_localnet = 1

my nftables rules partially cancels the effect of this parameter.
Docker can run even with the above parameters set, but we will encounter another issue where the Windows host cannot route to the networks added inside WSL.
if the Windows host could reflect the routing to the network inside WSL, the fourth point raised by @itispip would be easily solved.

I just tested WSL 2.3.11 and unfortunately the problem remains there.

@felipecrs : right - <IP_ADDRESS> fixes a variety of Linux container scenarios. This in particular will require Docker to change their route - tracked here: https://github.com/moby/moby/issues/48056.
We are in contact with them and the fix should hopefully be released soon.

@keith-horton
(https://github.com/microsoft/WSL/releases/tag/2.3.11)[WSL 2.3.11]

Update mirrored mode loopback routing to support connections between Windows host and Linux Docker/Podman containers (solves #11136,#11468,#11758,#10926)

this improvement gives us more options.
in the daemon.json

"userland-proxy": false

this now works as well.
but
https://github.com/moby/moby/pull/48075
this change is made, 'userland-proxy: false' may no longer work.
i think it needs a little adjustment.

this may seem unnecessary since moby takes care of it, but this is a new improvement plan.
WSL 2.3.11 brings several improvements to communication with Windows host.
therefore, you can now use the following rules to improve communication from Windows host to Docker containers:
$ nft add rule ip nat WSLPOSTROUTING iif "loopback0" ip saddr <IP_ADDRESS>/8 ip daddr != <IP_ADDRESS>/8 counter masquerade
$ sysctl -w net.ipv4.conf.all.route_localnet=1

@shigenobuokamoto with WSL pre-release 2.3.11 would you still recommend your systemd script workaround?
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

@shigenobuokamoto with WSL pre-release 2.3.11 would you still recommend your systemd script workaround?
https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

As an user, I can it still works fine.

@codeart1st
work on this issue is ongoing in moby (=Docker Engine), so until that is released you can use my script to mitigate the issue.
essentially, the issue is that network within WSL behaves differently from normal Linux, so i think it would be best to deal with it on the WSL side.
WSL 2.3.11 includes some improvements to mirrored networking, so i am looking into how to use them to improve the experience. this is the prototype new network-mirrored.service. please try this too.
https://gist.github.com/shigenobuokamoto/540c5f09a03eb07149501e99a6c8d82b

taking advantage of the fact that connect from Docker container network to Windows host is now possible, i wrote a more natural nftables.
added support for communication between Windows and Linux using <IP_ADDRESS>/8, so i worte the missing route.

the issue is that network within WSL behaves differently from normal Linux, so i think it would be best to deal with it on the WSL side.

Also, this would mean that whatever application that relies on the same feature as docker would have the same issue. It could be an old application that doesn't receive more support for example. Or some application that refuses to implement a fix just for WSL.
So, yeah, I totally agree. The ideal would have been the fix to happen on WSL side.

When will this fix be built into wsl?Now we can only use service script to make it work, I don't think this is a long term solution.
Besides, I use nixos-wsl, this is my example module that can be used as a reference for nixos user, from @shigenobuokamoto

{pkgs, ...}: {
    systemd.services.network-mirrored = {
      description = "network-mirrored";
      enable = true;
      wants = ["network-pre.target"];
      wantedBy = ["multi-user.target"];
      before = ["network-pre.target" "shutdown.target"];
      serviceConfig = {
        User = "root";
        ExecStart = [
          ''
            /bin/sh -ec '\
            [ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
            echo "\
            add chain   ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; };\
            insert rule ip nat WSLPREROUTING iif loopback0  ip daddr <IP_ADDRESS> counter dnat to <IP_ADDRESS> comment mirrored;\
            "|${pkgs.nftables}/bin/nft -f -\
            '
          ''
        ];

        ExecStop = [
          ''
            /bin/sh -ec '\
              [ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
              for chain in "ip nat WSLPREROUTING";\
              do\
                handle=$(${pkgs.nftables}/bin/nft -a list chain $chain | sed -En "s/^.*comment \\"mirrored\\" # handle ([0-9]+)$/\\1/p");\
                for n in $handle; do echo "delete rule $chain handle $n"; done;\
              done|${pkgs.nftables}/bin/nft -f -\
            '
          ''
        ];
        RemainAfterExit = "yes";
      };
    };
}

here is a really dirty solution, I just wrote a nodejs tcp reverse proxy server to map the docker port to another port, so we can using another port because its created by a non-docker process.
import net from 'net'
let map = {
  15432: 5432,
  13000: 3000,
}
for (const from in map) {
  const to = map[from]
  net
    .createServer((server) => {
      const client = net.createConnection(to)
      server.pipe(client)
      client.pipe(server)
    })
    .listen(from)
    .on('listening', () => {
      console.log(`tcp-proxy: ${from} -> ${to}`)
    })
    .on('error', (err) => {
      console.error(err)
    })
}

@zaaack, if you are looking for a temporary solution, this one is the best you'll find: https://github.com/microsoft/WSL/issues/10494#issuecomment-2241498340

@zaaack, if you are looking for a temporary solution, this one is the best you'll find: #10494 (comment)

thanks, but my wsl  is older and doesnt set systemd

thanks, but my wsl is older and doesnt set systemd

You can use iptables -t nat -I DOCKER -i loopback0 -d <IP_ADDRESS>/8 -j RETURN¹ or any of the variants² that have been mentioned above.

Has anyone had success configuring WSL to allow connections from containers in WSL to the windows host?
I made some attempts on my own to forward used ports, 5966 in this case, from WSL to the windows host through the nft but was unsuccesfull.
Maybe some one can tell me what I did wrong.
sudo nft add chain ip nat prerouting { type nat hook prerouting priority 0 \; } sudo nft add chain ip nat postrouting { type nat hook postrouting priority 100 \; } sudo nft add rule ip nat prerouting tcp dport 5966 counter dnat to <IP_ADDRESS>:5966 sudo nft add rule ip nat postrouting ip saddr <IP_ADDRESS>/8 counter masquerade

@jweaston
you can access from Docker container in WSL to Windows host address, WSL 2.3.11 and later.
.wslconfig
[experimental]
hostAddressLoopback=true

Windows host                  WSL Linux                         Docker Container
<IP_ADDRESS>    --- loopback --- <IP_ADDRESS>   --- not connected --- <IP_ADDRESS>(same address but unrelated)
<IP_ADDRESS> --- loopback --- <IP_ADDRESS>
                                  | routed
     (unknown network)        <IP_ADDRESS>  ----- connected ----- <IP_ADDRESS>

docker container (in WSL) can connect to <IP_ADDRESS>:5966.
if you try connect to <IP_ADDRESS>:5966, the packet is sent to container's <IP_ADDRESS>. so cannot connect to windows host.

hello. Docker completed the PR that fixes this issue - https://github.com/moby/moby/pull/48514
This was included in the following Docker release - https://github.com/moby/moby/releases/tag/v27.3.0
Please try Docker version 27.3.0

It works in my test. :)

My favorite flavor of Linux is Ubuntu via Docker via Ubuntu via WSL via Windows 11.

Just to add to this...
I recently set up Ubuntu 24.04.1 with Docker and it works fine in Mirrored mode (Docker 27.3.1)
But on my Alpine install (Docker 26.1.5) it refuses to work unless I set it back to NAT.
Was driving me crazy

Just to add to this...
I recently set up Ubuntu 24.04.1 with Docker and it works fine in Mirrored mode (Docker 27.3.1) But on my Alpine install (Docker 26.1.5) it refuses to work unless I set it back to NAT.
Was driving me crazy
EDIT: Does anyone have any advice on manually installing latest version in Alpine without using Dockers setup script that tries to force mt to use Docker Desktop?

can i see all of your relevant settings because i still cant get it to work. I am trying to run a simple gradio app inside of a docker instance.

Just to add to this...
I recently set up Ubuntu 24.04.1 with Docker and it works fine in Mirrored mode (Docker 27.3.1) But on my Alpine install (Docker 26.1.5) it refuses to work unless I set it back to NAT.
Was driving me crazy
EDIT: Does anyone have any advice on manually installing latest version in Alpine without using Dockers setup script that tries to force mt to use Docker Desktop?

@BolteDev can i see all of your relevant settings because i still cant get it to work. I am trying to run a simple gradio app inside of a docker instance. this used to work fine and since i've started trouble shooting everything i have no idea what are the proper settings.

Just to add to this...
I recently set up Ubuntu 24.04.1 with Docker and it works fine in Mirrored mode (Docker 27.3.1) But on my Alpine install (Docker 26.1.5) it refuses to work unless I set it back to NAT.
Was driving me crazy
EDIT: Does anyone have any advice on manually installing latest version in Alpine without using Dockers setup script that tries to force mt to use Docker Desktop?

@BolteDev can i see all of your relevant settings because i still cant get it to work. I am trying to run a simple gradio app inside of a docker instance. this used to work fine and since i've started trouble shooting everything i have no idea what are the proper settings.

I haven't done anything different than following normal installations and setting up docker, other than my wslconfig having networkingMode=mirrored.
I was setting up an Alpine install and nginx just wasn't working until I disabled it and went back to regular NAT along with localhostForwarding=true, I thought the docker just wasn't working until I did a curl against localhost from within the instance and realised it was working, which lead me here.
I'm still pretty new to Linux side of things, but I would say if you are lost with it, start from scratch. I would say try and get something simple like Nginx working then at least you know the networking side works and it will then be a docker image issue or not.
But basically all I did was install openrc (I'm using the 3mb rootfs Alpine) do the basic setup to start it in /etc/init-wsl:
# OpenRC
mount --make-rshared /
mkdir -p /run/openrc
touch /run/openrc/softlevel
sysctl net.ipv4.ip_unprivileged_port_start=80 > /dev/null 2>&1

openrc
rc-service networkmanager start
rc-service docker start

Then ran the standard nginx image as a test on 80:80

If you're seeing issues, please be sure you have the latest Docker + latest WSL.
Some applications may resolve the name of the local device to an assigned address - instead of using loopback. In those cases you'll want to set [experimental] hostAddressLoopback=true.
https://learn.microsoft.com/en-us/windows/wsl/wsl-config
hostAddressLoopback (bool - default == false)
Only applicable when wsl2.networkingMode is set to mirrored. When set to True, will allow the Container to connect to the Host, or the Host to connect to the Container, by an IP address that's assigned to the Host. The <IP_ADDRESS> loopback address can always be used,this option allows for all additionally assigned local IP addresses to be used as well. Only IPv4 addresses assigned to the host are supported.

If you're seeing issues, please be sure you have the latest Docker + latest WSL. Some applications may resolve the name of the local device to an assigned address - instead of using loopback. In those cases you'll want to set [experimental] hostAddressLoopback=true.
https://learn.microsoft.com/en-us/windows/wsl/wsl-config
hostAddressLoopback (bool - default == false) Only applicable when wsl2.networkingMode is set to mirrored. When set to True, will allow the Container to connect to the Host, or the Host to connect to the Container, by an IP address that's assigned to the Host. The <IP_ADDRESS> loopback address can always be used,this option allows for all additionally assigned local IP addresses to be used as well. Only IPv4 addresses assigned to the host are supported.

@keith-horton so i want them both to true? i run other apps withing WSL that work fine, but when i try to host the docker containers from the run command inside of WSL it does not connect to a forwarded port.

Right: some apps may want to connect to an assigned address instead of loopback. In those cases, it might help those solutions end-to-end to also enable hostAddressLoopback
(hostAddressLoopback is only applicable to Mirrored mode)

Right: some apps may want to connect to an assigned address instead of loopback. In those cases, it might help those solutions end-to-end to also enable hostAddressLoopback
(hostAddressLoopback is only applicable to Mirrored mode)

@keith-horton brother, you just changed my life. thank you

docker --version
Docker version 27.3.1, build ce12230

wsl.exe --version
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.64
MSRDC version: 1.2.5326
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.22631.4169

Disabled and stopped the systemd service helper for https://gist.github.com/shigenobuokamoto/540c5f09a03eb07149501e99a6c8d82b.
Propably most things work, but for no real reason I can't connect with my Oracle DB Container anymore. Reenabled the systemd service helper and my Oracle DB Container works again.

closing since the issue is resolved
if you encounter the issue after using Docker version 27.3.0 or later, please open a separate issue


Need to agree the detailed structure of the geographic tree

Question: what should come below region?
Obviously we start with:

The North
Region

but what comes next?
Should it be county and combined authorities? If so, there will be some small unitary areas (UTLAs) that aren't caught in this (e.g. City of York, Cheshire East).
We could to Counties and UTLAs, but that would give us, e.g. all 5 West Yorkshire districts on the same tier as North Yorkshire. Is this what we want? It feels that CAs (or Metropolitan Counties) should be represented as an equivalent level to counties, so that lower-tier LAs (e.g. Blackburn with Darwen, Blackpool) appear on the same layer as districts within West Yorkshire, Greater Manchester etc.
I'll build a quick mock-up of how this might work

Example leaflet interactive here
One thought: if we go regions > counties/CAs, then some unitaries are not visible. Also, that will mean some "layers" of geographies might be missing in the tree. Question: does this matter?
E.g.:

Region
County
CA
LA

Yorkshire & Humber
North Yorkshire
NA
Scarborough

Yorkshire & Humber
NA
West Yorkshire
Calderdale

Guess we could code an
if (is.na([next_level_of_geog_down)) {
  [go_to_next_level_down]
}

Could do with @gilesdring and @luke-strange thoughts

Update version is now here

[ ] Need to remove *NM fields from geography_tree and create a separate .csv with *CD, *NM as lookup

Is this basically done?


Bring up FAT filesystem on external Flash

A FAT filesystem on external Flash- assuming it is possible (block size, wear leveling concerns, etc)- would be useful for myriad reasons not limited to storing games, settings, save states and Lua scripts without requiring an SD card.
It would require a new implementation of user_diskio.c:
https://github.com/pimoroni/32blit-beta/blob/d788e63013a37b7eac0cd19d76398844fcb624fd/32blit-stm32/Src/user_diskio.c
And linking into the FATFS system: https://github.com/pimoroni/32blit-beta/blob/master/32blit-stm32/Src/fatfs.c#L33
There's a - possibly useful - implementation available under the MIT license here: https://github.com/amvasil/stm32-qspi-fat-usb/blob/master/Src/fat_qspi_drv.c

You may want to consider using LittleFS https://github.com/ARMmbed/littlefs
Just for external flash, not suggesting SD Card, as the one downside is that LittleFS is not lingua franca as FAT is (should 32blit support USB MSD in the future)

How do we format, partition etc?

It kinda depends on how the 'standard'/default model of  32blit external flash usage is envisage/designed.
In one scenario the flash could be 100% system managed and direct access is hidden from the enduser's OS File managers    e.g. there is code in the firmware to prepare LittleFS and copy files SD External to/from flash.
In another scenario, with a bit more work, the firmware could present a FatFS via USB MSD but have it backed by LittleFS and remap on the fly.
The external flash could be split into, for example, 50% LittleFS (e.g. binaries) / 50% FatFS (e.g. assets), for convenience and if that really makes sense, lots of options.
The benefits of LIttleFS wear levelling and resiliency seem to be worth using it in some way.

The library could be bundled with the 32blit SDK just as a optional component for the devs who go medieval, although they could obviously make it part of their own project and no need for the extra fat in 32blit code base.
It could have a 32blit specific C++ FS wrapper and it could then have a LUA binding too.
I’m rambling, just thought I’d mention it’s existence.
All the best
Wayne
(Sponsored by LittleFS :) )

I think if we do support a filesystem on QSPI flash it's likely it will be read-only and appended as a binary blob to the end of the user code. It would be useful where user code expects a local filesystem (perhaps in a port where converting all of the assets would otherwise be painful), but wants the throughput of QSPI over that of the SD card.

Closing for now since we've currently got our eyes on the QSPI flash for storing multiple games simultaneously and have made... some significant progress- see: https://github.com/pimoroni/32blit-beta/issues/220
I believe storing/running multiple games, and having a FAT filesystem are somewhat mutually exclusive.


380 BTR spawn issue 14.0.1

General information:
Project issue is for:
build 14.0.1 as of 01/27/24 0144 EST
Expected result:
BTR to move into playable area
Actual result:
BTR spawns outside of playable area and doesn't move
(Add a description of the issue here)
How to reproduce:

git clone 3.8.0 launcher, modules, server
copy live ETF files
create new 380 folder and paste latest EFT files
copy latest bepinx files and extract into 380 folder
build 3.8.0 SPT
copy 3.8.0 build into new 3.8.0 folder

Log files:
Server: (found in /user/logs/server-yyyy-mm-dd-xx.log)
BepinEx: (found in /BepinEx/LogOutput.log)
Client: (found in /Logs/log_<date>_<version>/<date>_<version> traces.log)
Profile:
You can optionally include your game profile if you think it would help us reproduce the error easily (found in /user/profiles/xxxxxxxxxxxx.json)
The profile id can be found in the top left of the launcher.
Screenshots:
(Optional, only add them if you think it adds value)

Attachments

380_debug_logs_pt2.zip (85 KiB)
380_debug_logs_pt1.zip (80 KiB)
LogOutput.log (55 KiB)

Originally written by naterade

Thanks to NGST, NVIDIA Reflex seems to be the problem as BTR works with it disabled

Originally written by naterade

this.moveSpeed / (float)Application.targetFrameRate;
We use the server component of the Vehicle, the server will never have Reflex enabled, and will almost always have a static targetFrameRate (The server tickrate)
However, if you enable Reflex, that number is set to -1
I'm working on a fix

Originally written by DrakiaXYZ

Spawning issue fixed here: https://dev.sp-tarkov.com/SPT-AKI/Modules/commit/c01bd92c7871567a81633a9ab568d0b367527d60
There's still a variability in the speed that's being looked into

Originally written by DrakiaXYZ

as stated , spawn/movement issue is fixed and tested. speed still is an issue but can be placed into separate issue

Originally written by naterade

Speed should be fixed as of https://dev.sp-tarkov.com/SPT-AKI/Modules/commit/eb443aaa5e01bf06ca4f6efbd23e8cd6abd411cb and is no longer tied to the framerate

Originally written by DrakiaXYZ


Files from /tmp/ are shown in quick picker even after the path is excluded in files.exclude setting

Issue Type: Bug
In quick picker files from /tmp directory are shown even with the following setting.
"files.exclude": {
	"/tmp/**": true
},
"files.watcherExclude": {
	"/tmp/**": true
},
"search.exclude": {
	"/tmp/**": true
},

Steps:
In a workspace, make changes in a file tracked by git. Open the changes in the file by using command "gitlens.externalDiffAll".
Close the diff editor and all open files.
In the quick picker (ctrl + p), type the edited file name. Same file from /tmp is also shown. Expected is this file should not be shown in recent files.
VS Code version: Code 1.69.2 (3b889b090b5ad5793f524b5d1d39fda662b96a2a, 2022-07-18T16:12:52.460Z)
OS version: Windows_NT x64 10.0.22000
Restricted Mode: No
Remote OS version: Linux x64 5.4.0-122-generic

System Info

Item
Value

CPUs
11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz (16 x 2496)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
31.73GB (17.50GB free)

Process Argv
--crash-reporter-id 85c323a1-5d98-45e5-8818-74519d9e3b95

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: <IP_ADDRESS>

OS
Linux x64 5.4.0-122-generic

CPUs
Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (4 x 900)

Memory (System)
62.70GB (41.09GB free)

VM
0%

Extensions (18)

Extension
Author (truncated)
Version

project-manager
ale
12.6.1

docs-view
bie
0.0.11

arm
dan
1.7.4

jupyter-keymap
ms-
1.0.0

remote-ssh
ms-
0.84.0

remote-ssh-edit
ms-
0.80.0

remote-wsl
ms-
0.66.3

linkerscript
Zix
1.0.2

atlascode
atl
2.10.12

codealignment-vscode
cpm
2.0.19

doxdocgen
csc
1.4.0

gitlens
eam
12.1.2

vscode-clangd
llv
0.1.21

cpptools
ms-
1.10.8

hexeditor
ms-
1.9.7

tmlanguage
ped
0.12.0

vscode-commons
red
0.0.6

vscode-yaml
red
1.9.1

(1 theme extensions excluded)

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonvspyt551cf:30345471
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
pylanb8912:30529769
vsclangdf:30486550
841h4636:30531698

I was able to workaround this by adding the entry following entry in files.exclude
"files.exclude": {
	"/tmp/**": true,
	"/tmp/git-blob**": true
},

But please fix this properly

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


chore(package): update minami to version 1.2.3

https://greenkeeper.io/
Description of change
Version 1.2.3 just got published.
Your tests are still failing with this version. Compare the changes 🚨

Commits
The new version differs by 2 commits .

b654ddb 1.2.3
1f9ad17 fix(style): move tutorials above code, fix readme

See the full diff.

Pull request (PR) check-list

:white_check_mark: Please review and check the appropriate items.

Type of change

[ ] BREAKING CHANGE (with documentation) Please explain if selected
[x] Chore
[ ] Defect (bug) fix (with regression tests)
[ ] Documentation
[ ] Feature/enhancement (with JSDoc and/or Wiki article(s))
[ ] Hotfix (with documentation)
[ ] Performance improvement (with benchmarks)
[ ] Refactoring (with catalog reference)
[ ] Reversion
[ ] Tests

Code standards compliance

[x] Yes, eslint passes.
[ ] N/A (not applicable): This is a dependency update

Code quality. Do the quality gateways pass with an "A" grade?

[x] Complexity
[x] Duplications (0.00%)
[x] Issues (0 issues or documentation for "won't fix" cases)
[x] Maintainability/Technical Debt
[x] Reliability
[x] Security
[ ] N/A (not applicable). Please explain if selected

Test coverage

[x] The source code is 100% covered with passing specs.
[ ] N/A (not applicable). This is a dependency update

:information_source: These tasks are not required to open a PR and can be done afterwards, or while the PR is open.

Coverage remained the same at 100.0% when pulling 6f3c3bcf78873c44e6247ff551c605d7926713b8 on greenkeeper/minami-1.2.3 into 65a7191f6b6a4269c4efc03353bd339b854c2e85 on develop.

Coverage remained the same at 100.0% when pulling 6f3c3bcf78873c44e6247ff551c605d7926713b8 on greenkeeper/minami-1.2.3 into 65a7191f6b6a4269c4efc03353bd339b854c2e85 on develop.

Coverage remained the same at 100.0% when pulling 6f3c3bcf78873c44e6247ff551c605d7926713b8 on greenkeeper/minami-1.2.3 into 65a7191f6b6a4269c4efc03353bd339b854c2e85 on develop.

Coverage remained the same at 100.0% when pulling 6f3c3bcf78873c44e6247ff551c605d7926713b8 on greenkeeper/minami-1.2.3 into 65a7191f6b6a4269c4efc03353bd339b854c2e85 on develop.


(BOLT-185) Add bolt functions from puppet

This adds the execution functions from Puppet into an embedded module in Bolt. The unit tests for the functions have also been copied over.
At the moment those tests does not run as they require 'puppet' and sublibraries thereof. (Need help to fix those).

@adreyer sure, not good if there is a confusion over which module is which. The name of the embedded module will only show up if there is an error - or if bolt needs to include things in the puppet namespace that require prefixing with the module name. Then users would also see that name. Is one of boltcore, boltlib, boltpp, or boltmodule  acceptable perhaps?

CLA signed by all contributors.

Of those I like boltlib but they're all fine

I renamed the embedded module to boltlib and dropped the metadata.json.
Still need help with how to get the unit testing of the functions to work - they cannot just require 'puppet' as things are configured now.

@hlindberg I think you just need to require 'vendored/require_vendored'. We'll also need bolt's spec_helper.rb to register before_each, etc hooks in puppet's lib/puppet/test/test_helper.rb. Something like:
require 'vendored/require_vendored'
require 'puppet/test/test_helper'

RSpec.configure do |config|
  Puppet::Test::TestHelper.initialize

  config.before :all do
    Puppet::Test::TestHelper.before_all_tests
  end

  config.before :each do
    Puppet::Test::TestHelper.before_each_test
  end

  # etc
end

Alternatively, we could use puppetlabs_spec_helper...

I'm working on updating this. There's a bit more work needed than Josh outlined.

@MikaelSmith I think we probably need to rewrite the tests to use normal module testing tools instead of requiring directly from puppet/spec

Closing in favor of https://github.com/puppetlabs/bolt/pull/161.


videocomposer: change relative import .. in dataset

A minor change to avoid the error of not finding annotator when export MS_COMPILER_CACHE_ENABLE=1. This bug will be fixed in future mindspore version.

Does export MS_COMPILER_CACHE_ENABLE=1 also work in the inference part?
https://github.com/mindspore-lab/mindone/blob/master/examples/videocomposer/vc/infer_engine/modules/extractor.py#L11
https://github.com/mindspore-lab/mindone/blob/master/examples/videocomposer/deploy/data/datasets.py#L10


A typo(?) causing crashes in newer versions of apify + crawlee[parsel]

I am running an Actor using the CrawleeParsel.
After starting the actor, it immediately crashes because of this error:
TypeError: Requested global configuration object of type <class 'apify._configuration.Configuration'>, but <class 'crawlee.configuration.Configuration'> was found

This is my requirements file:
apify
beautifulsoup4[lxml]
httpx
types-beautifulsoup4
crawlee[parsel]
I managed to fix the error specifying  apify==1.3.0

Hello, and thank you for your interest in Crawlee! Could you give us an executable code snippet that reproduces the error?

Hi @Rigos0, the crawler initialization has to be wrapped inside async with Actor. If you'd like to define your request handlers at the top level, you can use the following structure:
import asyncio
from apify import Actor
from crawlee.parsel_crawler import ParselCrawler, ParselCrawlingContext
from crawlee.router import Router

router = Router[ParselCrawlingContext]()

@router.default_handler
async def request_handler(context: ParselCrawlingContext) -> None:
    context.log.info(f'Processing URL: {context.request.url}...')
    ...

async def main() -> None:
    async with Actor:
        input_data = await Actor.get_input()
        start_urls = input_data.get('startUrls', [])
        url_to_crawl = start_urls[0]['url']

        crawler = ParselCrawler(request_handler=router)
        await crawler.run([url_to_crawl])

if __name__ == '__main__':
    asyncio.run(main())

I believe this should resolve your issue, so I'll close it. Feel free to re-open if this won't help.


RfcommDeviceService null exception

Windows.Devices.Bluetooth.Rfcomm.RfcommDeviceService.FromIdAsync("Bluetooth#Bluetoothb8:27:eb:39:ae:e9-98:d3:32:30:c2:2f");
returns null which is used in Microsoft.Maker.Serial.BluetoothSerial.Begin();
But i can get service with this method
                Windows.Devices.Bluetooth.Rfcomm.RfcommDeviceService device_service = null;
                var bluetoothDevice = await Windows.Devices.Bluetooth.BluetoothDevice.FromIdAsync("Bluetooth#Bluetoothb8:27:eb:39:ae:e9-98:d3:32:30:c2:2f");
                var rfcommServices = await bluetoothDevice.GetRfcommServicesForIdAsync(Windows.Devices.Bluetooth.Rfcomm.RfcommServiceId.SerialPort);
                if (rfcommServices.Services.Count > 0)
                {
                    device_service = rfcommServices.Services[0];
                }

are they different things or this is an issue?
Package.appxmanifest
  <Capabilities>
    <Capability Name="internetClient" />
    <Capability Name="privateNetworkClientServer" />
    <Capability Name="internetClientServer" />
    <iot:Capability Name="systemManagement" />
    <iot:Capability Name="lowLevelDevices" />
    <DeviceCapability Name="109b86ad-f53d-4b76-aa5f-821e2ddf2141" />
    <DeviceCapability Name="00001101-0000-1000-8000-00805F9B34FB" />
    <DeviceCapability Name="bluetooth.rfcomm">
      <Device Id="any">
        <Function Type="name:serialPort" />
      </Device>
    </DeviceCapability>
    <DeviceCapability Name="serialcommunication">
      <Device Id="any">
        <Function Type="name:serialPort" />
      </Device>
    </DeviceCapability>
  </Capabilities>

After uninstall application and restart raspberry pi, problem solved.


cdml-kinesis-analyzer to 0.0.8

Promote cdml-kinesis-analyzer to version 0.0.8

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


New Software - DynamicsSDK

Hi,
I am using Azure pipelines with the agent spec of vs-2017-win2016.
Getting error:  "No agent found in pool Azure Pipelines which satisfies the following demand: DynamicsSDK. All demands: DynamicsSDK, Agent.Version -gtVersion 2.103.0"
Can you please install DynamicsSDK?

@s-abram , it is prohibited to use demands with Hosted agents, please remove all demands and try to rerun builds.
From Azure DevOps documentation:

Demands and capabilities are designed for use with self-hosted agents so that jobs can be matched with an agent that meets the requirements of the job. When using Microsoft-hosted agents, you select an image for the agent that matches the requirements of the job, so although it is possible to add capabilities to a Microsoft-hosted agent, you don't need to use capabilities with Microsoft-hosted agents.

All Hosted VMs have identical configuration and you shouldn't use demands

thank you, I will try that.

I did remove the demand but the build failed as it is looking for the software.
. : The term 'D:\a\10\s$(DynamicsSDK)\PrepareForBuild.ps1' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:3 + . 'D:\a\10\s$(DynamicsSDK)\PrepareForBuild.ps1' -DatabaseBackupToRes ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (D:\a\10\s$(Dyn...areForBuild.ps1:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException

So looks like this tool is not installed on images. Could you please modify your first message to follow tool-request template: https://github.com/actions/virtual-environments/blob/main/.github/ISSUE_TEMPLATE/tool-request.md

Close, the new thread: https://github.com/actions/virtual-environments/issues/1910


Azure function failed to deploy with the error ERROR_CONNECTION_TERMINATED

Environment
Visual Studio
Issue Description
I am trying to deploy the azure function using visual studio and getting the error: Web deployment task failed (Complete error logs are given below).
I have followed all the steps mentioned in the article http://techgenix.com/video-publish-azure-functions-azure-using-visual-studio-tools-azure-functions/ but still facing the issue.
Please help me as I have spent a lot of time on this issue and didn't get any solution.
Error logs
1>------ Build started: Project: TestFunction, Configuration: Release Any CPU ------
1>TestFunction -> C:\Repo\TestFunction\TestFunction\bin\Release\net461\bin\TestFunction.dll
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
Publish Started
TestFunction -> C:\Repo\TestFunction\TestFunction\bin\Release\net461\bin\TestFunction.dll
TestFunction -> C:\Repo\TestFunction\TestFunction\obj\Release\net461\PubTmp\Out
C:\Program Files\dotnet\sdk\2.1.300-preview1-008174\Sdks\Microsoft.NET.Sdk.Publish\build\netstandard1.0\PublishTargets\Microsoft.NET.Sdk.Publish.MSDeploy.targets(139,5): error : Web deployment task failed. (Web Deploy experienced a connection problem with the server and had to terminate the connection.  Contact your server administrator if the problem persists.  Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_CONNECTION_TERMINATED.) [C:\Repo\TestFunction\TestFunction\TestFunction.csproj]
C:\Program Files\dotnet\sdk\2.1.300-preview1-008174\Sdks\Microsoft.NET.Sdk.Publish\build\netstandard1.0\PublishTargets\Microsoft.NET.Sdk.Publish.MSDeploy.targets(139,5): error :  [C:\Repo\TestFunction\TestFunction\TestFunction.csproj]
C:\Program Files\dotnet\sdk\2.1.300-preview1-008174\Sdks\Microsoft.NET.Sdk.Publish\build\netstandard1.0\PublishTargets\Microsoft.NET.Sdk.Publish.MSDeploy.targets(139,5): error : Web Deploy experienced a connection problem with the server and had to terminate the connection.  Contact your server administrator if the problem persists.  Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_CONNECTION_TERMINATED. [C:\Repo\TestFunction\TestFunction\TestFunction.csproj]
C:\Program Files\dotnet\sdk\2.1.300-preview1-008174\Sdks\Microsoft.NET.Sdk.Publish\build\netstandard1.0\PublishTargets\Microsoft.NET.Sdk.Publish.MSDeploy.targets(139,5): error : Root element is missing. [C:\Repo\TestFunction\TestFunction\TestFunction.csproj]
Publish failed to deploy.

Please open a thread here for VS related issues.


Add PackageManagement nuget package as a depedency to Microsoft.PowerShell.SDK nuget package

PackageManagement should be added as a dependency to Microsoft.PowerShell.SDK
Steps to reproduce
NA
Expected behavior
PackageManagement should be a dependency.
Actual behavior
PackageManagement is missing as a dependency.
Environment data
NA

@bmanikm
PackageManagement NuGet should be available on NuGet.org to add this dependency.

This is not needed anymore


Solve intermittent failing jobs on hosted runners with heavy loads

The chaos pipeline is currently plagued by failing tests owing to the following error (which also applies to other hosted runners):

The hosted runner: ubuntu-latest-8-cores_999a951508e7 lost communication with the server. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.

When running the test locally on a 32GB M2 MacBook, there is never a problem with CPU/Memory so it is likely that GH's hosted runners are not capable of running these tests in their entireties. Such breaks in the chaos pipeline significantly reduce dev velocity as engineers must manually inspect the pipeline for these flakes. Moreover, it causes large bottlenecks in the release process due to wasted time waiting on and rerunning flakey tests.
We should therefore look at alternative methods by which these jobs can be executed either through provisioning our own dedicated self-hosted GH runners or looking at different cloud CI providers for this specific repo.

Fixed by disabling PolarSignals: https://github.com/weaviate/weaviate-chaos-engineering/commit/52b0d00ea486881f5779eed3fb5a70deb714560e


Update fs2-core, fs2-io, ... to 3.4.0

Updates

co.fs2:fs2-core
co.fs2:fs2-io
co.fs2:fs2-reactive-streams

from 3.3.0 to 3.4.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "co.fs2" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "co.fs2" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #819.


Python 3

I suggest changing the header of EVERY file to the following:
# -*- coding: utf-8 -*-

"""
Copyright (c) 2017, University of Southern Denmark
All rights reserved.
This code is licensed under BSD 2-clause license.
See LICENSE file in the project root for license terms.
"""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

# original script below

As discussed e.g. here:
http://python-future.org/quickstart.html
https://www.digitalocean.com/community/tutorials/how-to-port-python-2-code-to-python-3

Yes, thanks, I'll do it this week.

Fixed: https://github.com/sdu-cfei/modest-py/pull/39

I believe the latest pyFMI is also Py3 compatible, but they did not yet make an official release, same for Assimulo. Maybe ask for a new release on their forum?
I assume you are aware of https://github.com/CATIA-Systems/FMPy

Python 3 compatibility would be great, I'll check their forum.
I know about FMPy, but did not have time to test it yet. Have you tested it? It looks promising.

I have not yet tested FMPy, I only heard about it and it is Py2 and Py3 compatible.

OK, I have it on my TODO list :)

http://jmodelica.org/10417#comment-6642

http://jmodelica.org/27800

Also, we got the answer: http://www.jmodelica.org/10417#comment-6643


Failed to install PowerToys app version 0.58.0

Microsoft PowerToys version
0.58.0
Running as admin

[X] Yes

Area(s) with issue?
Settings, Installer
Steps to reproduce
This started happening after updating PowerToys to version 0.58.0
✔ Expected Behavior
The settings app opens after clicking the settings button in-tray.
❌ Actual Behavior
Nothing happens when clicking the settings button in-tray.
Other Software
No response

/dup #18015
Can you please try  0.59 ?


PLA-91 Fetch features data from es and setup database model for persisting features in db

Proposed Changes
The feature is a script that will help get device features from elasticsearch and persist said data in the database. This PR contains

The initial script setup
The es aggregation query to fetch device features from
The model for storing features in db
Adding the database column for features to devices_integration

Impacted Routes

Caveats

It makes the most sense to me to do the PR that adds the table with all the signals first, and then to drive these queries from that.

Let's also update readme with a new section on how to run this locally. The tricky thing will be test data - but you can look at example in dimo-explorer or ask @Allyson-English how she managed with that. I think it may be a problem we have solve before.

Let's also update readme with a new section on how to run this locally. The tricky thing will be test data - but you can look at example in dimo-explorer or ask @Allyson-English how she managed with that. I think it may be a problem we have solve before.

I actually have my local test environment setup and I was able to test this before pushing. We might need to write something more comprehensive in a central location on how to setup local testing with es and how to generate test data.

Let's also update readme with a new section on how to run this locally. The tricky thing will be test data - but you can look at example in dimo-explorer or ask @Allyson-English how she managed with that. I think it may be a problem we have solve before.

I actually have my local test environment setup and I was able to test this before pushing. We might need to write something more comprehensive in a central location on how to setup local testing with es and how to generate test data.

That's great.
For central docs, we can use the internal docs repo - there's already some elastic stuff in there or start a separate one specific to this. Anything repo specific is fine in the Readme.


8413.15 add section to show super users in UI

https://github.com/Shippable/pm/issues/8413
add a section in addons titled System super users
this should show an empty text box followed by a list of accounts that are returned by GET /api/superusers route
when user clicks Install, the value in text box is POST-ed in /api/superusers route if it is non-empty
if its empty, skip the post
all items in the list should have a x mark after the id
clicking it should mark the account id for deletion from account roles table
when user clicks Install, all the accounts which have been marked for deletion should be removed from account roles table

complete. closing.


[azure-iot-sdk-c] Fix passing hsm_type_symm_key

A bug in the port file resulted in hsm_type_symm_key not being defined to ON when using the feature use-prov-client
+@ewertons for review

/azp run


Upload functionality in 1.0.0-beta.17

How can I implement file upload functionality with custom type?

You cannot upload files with graphql like you can do with REST APIs, you have to either convert your files to base64 on the frontend and then use those to upload files or have to go for REST endpoint for file uploads. If you are uploading files to S3 then generate signedurl via graphql and then use that signed url on the frontend to upload files.

I am trying to use "graphql-upload" for creating custom upload mutation but for that, I have to create custom type "Upload". How can I create a custom type?

here you can find an example: https://github.com/almost-full-stack/graphcraft/blob/feature/refactor/tests/sequelize-graphql-schema.spec.js
look at types

if you want to create a custom type within a model, you can also do same in there by using type property.

Resolved.


Dynamic filtering for semi-joins: execution and testing

Follows #4942
Supersedes #2190

merged, thanks!

Thanks a lot!

@lxynov got any before vs after performance stats on this?