Wrong Gateway address pingable and ssh

  • I used Global Pro to update a NS box, but have the wrong gateway address. For some reason I am able to ping, and SSH to the NS, but not able to ping out from the NS.

    Usually if the gateway IP is wrong, I will loss connectivity to the box. For whatever reason I am able to ping and SSH.

    Does the netscreen remember where packet coming in by interface and sent it back out?

  • Engineer

    Yep Steve, as you point on your post, dynamic routing has been introduce to 4.x screenOS but not designed for it at the first time. It seems that we are wainting a lot of screenOS 5 for several amelioration.

    You tell that you have always-on-dest command on your default configuration template …
    Why do you choose to use it by default ?
    Do you note some performance dgradation with it ?

    cheers …

  • Hi Flo,

    Yep, I enable that command on all my firewalls and have that as part of the standard template. However, as far as I’ve seen it does not help in this issue :(.

    – steve

  • Engineer

    Thanks to correct me Steve …
    sorry for the mistyping but as you well know, set mac does not exist …
    Concerning this problem, it is more related to route table update however just want to point on the possibility to change the behavior of not sending mac request once session is established. Helpful on some case

  • Are you sure about that? There is no ‘set mac always-on-dest’ command that I’m aware of unless it is more recent than 4.0.3r2. There IS a ‘set arp always-on-dest’ but that doesn’t help this problem.

    – steve

  • Engineer

    However you can force the NS to make MAC request even if the session is already established by issuing the set mac always-on-dest.

  • Not to my knowledge. It was going to require a rearchitecting of the session table code so they may be waiting until 5.0 comes out.

    – steve

  • Engineer

    Is this already fixed?


  • Yep. Should have more info on how soon this will be resolved later on today.

    – steve

  • Historically, NetScreen’s cached the source and destination MAC’s in the session table. This was done for performance improvements. Before the recent addition of dynamic routing protocols, this worked great. Looks like there was an oversight, and the MAC addresses aren’t being re-evaluated when there is a route table update.

    This is not addressed in r3, which is already out. I’m not sure how fundamental this change is, but it might be painful. I’d not bet on an r4 fix, personally.

    Incidentally, a ‘get session’ shows the MAC addresses stored for that session:

    ns208-> get session
    alloc 17/max 128000, alloc failed 0
    id 83047/s**,vsys 0,flag 00000040/00/00,policy 320002,time 2
     4(01):10.yyy.yyy.yyy/137->10.xxx.xxx.xxx/137,17,00085b19af37,vlan 0,tun 0,vsd 0
     4(20):10.yyy.yyy.yyy/137<-10.xxx.xxx.xxx/137,17,00c02b000080,vlan 0,tun 0,vsd 0

    Right after the port and protocol number.

  • Case# 49249 assigned.


    – steve

  • I know of no other way currently short of clearing the active session in question:

    ns5xp-> clear session ?
    <return>all clear all sessions
    dst-ip destination ip address
    dst-mac destination mac address
    dst-port destination port number or range
    id clear id specified session
    protocol protocol number or range
    src-ip source ip address
    src-mac source mac address
    src-port source port number or range
    vsd-id clear vsd-id specified sessions
    ns5xp-> clear session

    – steve</return>

  • I can confirm Gillsr’s observations and have experienced similar issues on a 5XP/ADSL. Like gillsr I could’nt believe it so I have been looking at other possible causes, but this explaind the issues :idea:
    When this happens is their a way to force the box to re-evaluate the next-hop??

  • cheers for the lab work…let us know what NS say…here comes r3 probably 😉


  • I just confirmed this issue in the lab. It appears as if NS 4.0.0r2 code does NOT re-evaluate the next-hop for active sessions even though there is a routing change!!!

    • I had a redundant OSPF path set up through two Netscreens (a NS50 and NS208).
    • I removed the primary path, and my established session (through both firewalls) was disconnected even though there was a secondary path. This is not good.
    • I established a session over the secondary path while the primary was down.
    • I reinserted the primary cable and the primary path once again became the best next-hop.
    • I removed the cable for the secondary path and my session was disconnected. This is not good.

    Essentially established sessions do not re-evaluate next-hops when the routing table changes either for the better or for the worse. I would presume that all of the following fall in the above category:

    VPN Group Failover
    Dyanamic Route Updates
    Manual Updates
    Receipt of ICMP Redirects

    I’ll be reporting this to NS shortly.

    – steve

  • Engineer

    If that is what happen, You are completly right steve !!

    I don’t have check this with dynamic routing or VPN failover but I through that the box don’t manage his proper connexion like other session. And that NS looks at his routing table if he need to connect to an VPN endpoint with new IP !!

    Have you check this ?


  • Actually it is a problem. VPN failover would potentially not check the routing table again because the session would already be established and located in the session table. As we have established, a change in routing table SHOULD warrant a change in forwarding path. The same could be true for dynamic routing failures/changes. This is actually quite important. I’m sure it improves performance, but it does NOT improve reliability.

  • Engineer

    You are right, it’s hard to believe since I don’t see other firewall proceed like that. Routing table is checked for each packet passing the firewall.

    But I am not sure this is a bug … it must improve perf and no security concern.

    VPN failover are not a problem : since the VPN endpoint change, the NS will check his route table.
    With manual routing, no often change to the route table.
    Concerning dynamic routing or ICMP redirect, that new and it will be interresting to see if some problem can be caused by this behavior.
    However, I don’t think that this will cause lots of problem of connectivity.


  • It is still unclear as to whether or not other situations in addition to manual route changes do not trigger a recalculation of the session’s next-hop.

    – steve

  • FLO, it seems you are correct. I have seen this behaviour during a firewall cutover before but I refused to believe it. I just briefly tested this and noticed that it is indeed the case where the routing table is not being consulted despite the fact that routing has changed on the box. A session remains reachable even though the route to that destination is no longer there.

    I would probably consider this a bug which has become even more important now with the addition of dynamic routing whereby paths to given destinations can change more easily.

    Cached information ceases to be correct in instances including but not limited to: manual route changes, dynamic routing changes, receipt of redirects, VPN failover, etc…

    If a route changes over which a session is being forwarded, then the firewall SHOULD re-evaluate the path it chooses to forward those packets. If it doesn’t, this is very bad practice.

    The routing table should not only be consulted during the beginning of a session setup, but also during session flow if there is a change.

    On the CLI you can simply type “get session” to view the session table. Some of the entries are fairly self evident, others aren’t. Most session tables keep track of socket 4 tuples, special flags, timers, protocol, and a few odds and ends.

    Keep in mind that the fact that an entry makes it into the session table does not guarantee that a packet is being forwarded. IE you may find that a firewall builds a session table entry, but drops a packet before actually forwarding it based on a variety of factors. Do not use the fact that an entry makes it into the session table proof that a packet has been forwarded.

    At the risk of overgeneralizing things, the process for making it into or out of the session table is:

    1. If the flow is not already in the session table and it matches a policy, and there is room in the session table, add it to the session table. Mark the session as embryonic with a smaller timeout.

    2. If the flow completes TCP three way handshake (or gets established for other protocols), it moves to the established state and the timeout gets increased to the timer configured for that service.

    3. If the flow timer expires before any new packets arrive, mark it for deletion.

    4. If the flow is terminated (FIN/RST/etc…), mark it for deletion.

    5. Flow garbage collection runs at 10 second intervals.