4.1 and Liteqube

[kennethrrosen]

I manually unpacked and installed the mirage fw and kernel into a liteqube setup on 4.1 (I realize liteqube also offers it as its own fw). The VM seems to function, but it cannot supply firewall rules. There is a red banner notice reading "Net qube does not support 'qubes-firewall'. Firewall restrictions will not be applied."

[palainp]

I see this too, but it will work as expected (to validate, you can try to restrict to one domain and you won't be able to get traffic outside that domain).

The message may come from a missing signaling part in the communication protocol between Qubes and the unikernel. We can probably leave this issue open until this message is fixed.

[palainp]

Update: I probably checked the issue on an old mirage-firewall installation, but with the new installation procedure, we should run qvm-features mirage-firewall qubes-firewall 1 which sets up the firewall property and removes this red message.

Can you tell me if this solves the problem for you?

[kennethrrosen]

I did this at install, as per the README file instructions. No joy. I've setup the same configuration on a vanilla 4.1 with firewall rules in the mirage VM, and it acts as a perfect vpn killswitch, allowing only VPN traffic. but here it allows both clearnet and VPN traffic, even with the rules.

[palainp]

Would you mind to give the result of qvm-firewall AppVM list?

[kennethrrosen]

[user@dom0 ~]$ qvm-firewall fw-mirage list
NO  ACTION  HOST  PROTOCOL  PORT(S)  SPECIAL TARGET  ICMP TYPE  EXPIRE  COMMENT
0   accept  -     udp       1194     -               -          -       -
1   accept  -     tcp       5995     -               -          -       -
2   accept  -     tcp       8443     -               -          -       -
3   accept  -     udp       4569     -               -          -       -
4   accept  -     udp       5060     -               -          -       -
5   accept  -     -         -        dns             -          -       -
6   accept  -     icmp      -        -               -          -       -
7   drop    -     -         -        -               -          -       -

[palainp]

Thank you for that output.

These rules (set on mirage-fw in Qubes Manager but not dealt inside mirage-fw, it's up to it's netvm to filter the netflow) should be visible in the netvm with iptables -L (it may worth to check there if something is missing), and I agree that any flow not matching an accept line should be dropped in netvm.

It may also worth to try to set the rules on your vpn VM tab in Qubes Manager to check how mirage-fw deals with that.

[kennethrrosen]

They don't seem to populate in core-net, but of course the rules work when set through the Qubes Manager in sys-vpn, though I'm stumped as to why I can't set all necessary rules in fw-mirage.

[palainp]

Yes I think you should be able to restrict the vpn netflow by setting rules on the mirage-fw tab in Qubes Manager. However this leads into rules being checked by sys-net or core-net which is untrusted, and that sounds odd to me.

Given it seems to work as intended with a vanilla Qubes, is it possible to try with liteqube and a normal linux firewall: setting rules on that VM and see if it works as vpn killswitch? If it works we should fix something into mirage-fw, and if not it's probably something relevant in liteqube (and we may open an issue there with this one as background).

[kennethrrosen]

I created a new default sys-firewall qubesctl state.apply qvm.sys-firewall and attached the qubes: core-net <--> sys-firewall <---> sys-vpn <---> appqube and applied the firewall rules to sys-firewall. This worked as intended.

When I set the rules in sys-vpn, the appqube, and mirage-fw, those rules don't appear to be visible in core-net.

[palainp]

The rules should only be visible in the uplink VM where you add some rules (so core-net when rules are set on mirage-fw or sys-firewall, mirage-fw or sys-firewall when set on sys-vpn, etc.).
EDIT: In fact it's very hard to see the rules with iptables inside sys-net, the output of iptables refers to vif+ with few details :(

As it works fine with a linux firewall, I assume the issue comes from mirage-fw, but I don't see what it could be so far :(

[palainp]

To summarize the tests, am I right with the following?

• standard Qubes, firewall rules on mirage-fw: can limit traffic to vpn only
• standard Qubes, firewall rules on sys-firewall: can limit traffic to vpn only
• Qubeslite, firewall rules on mirage-fw: let vpn and non vpn traffic to go through
• Qubeslite, firewall rules on sys-firewall: can limit traffic to vpn only
• Qubeslite, firewall rules on sys-vpn with mirage-fw as netvm: can limit traffic to vpn only

So the situation is that we can't filter traffic outside vpn with mirage-fw directly after core-net, but we can't reproduce it with standard Qubes distribution.

[kennethrrosen]

Correct. Happy to test any other configurations. Is the red text appearing only since the latest Qubes release?

[palainp]

Hi, I found an easy way to check firewall rules in core-net, would you mind to type:

• before applying fw rules to qubes-mirage-fw
• after applying fw rules to qubes-mirage-fw
• before applying fw rules to sys-firewall
• after applying fw rules to sys-firewall

You should get something referring chain qbs-10-137-0-XX which stands for the IP the qubes and see the rules there.

[palainp]

Sorry I forgot the command to type in core-net : sudo nft list table qubes-firewall.

[kennethrrosen]

Hmmmm, that's bizarre: core-net and debian-core are both mssing nftables and thus I can't run nft.

[palainp]

Oh that's probably because liteqube uses the -minimal templates. That may (or may not) be a root cause for this issue.

I'm also thinking about the vpn script (https://github.com/a-barinov/liteqube/blob/main/4.VPN/default/debian-core/etc/protect/template.core-vpn/usrlocal/bin/liteqube-vpn), please, if that's what you use as vpn, can you check that the iptables rules are correctly up ? (this should be visible in the core-vpn VM with sudo iptables -L -v).

[kennethrrosen]

I don't use the liteqube VPN, but rather a variant of this: https://forum.qubes-os.org/t/how-to-setup-openvpn-fedora-appvm-for-ovpn/3354

[palainp]

Ok thanks ! Anyway this is probably not the root cause.

It seems that iptables in debian stands for nft, so you still should be able to see the rules for mirage-fw inside core-net (but the syntax could be different and I havn't liteqube for testing :( ), would you mind to share that?

[kennethrrosen]

Sorry I forgot the command to type in core-net : sudo nft list table qubes-firewall.

root@localhost:~# sudo iptables -L

target               prot       opt   source                                      destination
DROP               all         --     anywhere                                  anywhere             state INVALID
DROP              udp        --     anywhere                                  anywhere             udp dpt:bootpc 
ACCEPT           all          --     anywhere                                  anywhere            ctstate RELATED,ESTABLISHED
ACCEPT           icmp      --     anywhere                                  anywhere  
ACCEPT           all          --     anywhere                                  anywhere  
REJECT            all         --     anywhere                                  anywhere             reject-with icmp-host-prohibited
DROP               all         --     anywhere                                  anywhere  

Chain FORWARD (policy DROP)
target               prot       opt   source                                      destination
DROP               all         --     anywhere                                  anywhere            state INVALID
ACCEPT           all         --     anywhere                                  anywhere            ctstate RELATED,ESTABLISHED
QBS-FORWARD     all         --     anywhere                                  anywhere
DROP           all         --     anywhere                                  anywhere
ACCEPT           all         --     anywhere                                  anywhere
DROP           all         --     anywhere                                  anywhere

Chain OUTPUT (policy ACCEPT)
target               prot       opt   source                                      destination

Chain QBS-FORWARD (1 references)
target               prot       opt   source                                      destination

[palainp]

So I have exactly the same (classical Qubes, fedora sys-net). So there's probably a way to check into core-net the filter rules you set for mirage-fw. Sadly I didn't find something useful yet :(

[palainp]

@arkenoi, @froeb I recently saw that you've been trying to adapt liteqube to Qubes 4.2.

I'm throwing a bottle into the sea in case you might have an idea of what could cause issues with mirage-fw (not) acting as a killswitch when used in a VPN context.
TLDR: on a classic Qubes 4.2 mirage-fw correctly acts as a killswitch as well as sys-fw, on liteqube mirage-fw doesn't but core-fw does.