For the IT gurus [Network routing issue to AFF]

[NM]

[Uber Geek Hat On]
ICMP hasn't been blocked, that is obvious since a trace route can be done all the way to the endpoint from other addresses. The ISP has not blocked ICMP either as trace routes will still get as far as 100ge4-1.core1.lax2.he.net (184.105.65.9).

But the trace stops at that point, so the non-delivery of the ICMP message may be from that point?

If it was an MTU problem chances are that many people would be experiencing issues, not just a select few coming from a specific range of IP addresses. (My MTU is set to 1500 right now, and I can access AFF no issues).

And if there is a link along your path that is less than 1500 byte MTU then it is either fragmenting correctly or if Do Not Fragment bit is set (common for SSL traffic) then the ICMP Type 3 Code 4 (Destination Unreachable, Fragmentation Needed and Don't Fragment was set) is getting back to your client. That does not change the MTU, just the frame size for the TCP session.

Note also that the supplied traceroute output was from an iMac, which runs an Apple version of Unix operating system. The Unix traceroute command uses UDP rather than ICMP EchoRequest (basically PING) used by Windows devices, but still looks for the same ICMP Type 3, Code 12 or Type 11 (TTL Expired) response. The UDP based Traceroute is generally more reliable at getting through, but the responses can still be blocked.

On a Mac, you can force it to use ICMP Echo for Traceroute by adding the -I option to the command.

As I said above, my money would be an administrator corrupting a routing table (easy to do), or an overzealous security program blocking an entire range of IP's after someone launched an attack against them (possible, they do host some other pretty important sites which the world would notice if they went down).

Certainly could be the case. But if this is the case, the administrator needs serious education and change control practice needs to be reviewed.

Quick and easy way to test on a Windows PC is to use the NETSH command to temporarily reduce the MTU on the client to say 1400 bytes and test the connection again.

 

[JohnM]

What language are they speaking .

 

[harvyk]

Since the original problem has now been diagnosed and I believe fixed, this can now be a more general discussion.

But the trace stops at that point, so the non-delivery of the ICMP message may be from that point?

Correct, from JB's (and others) address, it hits that router (or more likely firewall / IDP) and gets blocked at that point. I'm coming from a different IP address when I do my trace and thus I'm allowed through.

And if there is a link along your path that is less than 1500 byte MTU then it is either fragmenting correctly or if Do Not Fragment bit is set (common for SSL traffic) then the ICMP Type 3 Code 4 (Destination Unreachable, Fragmentation Needed and Don't Fragment was set) is getting back to your client. That does not change the MTU, just the frame size for the TCP session.

Note also that the supplied traceroute output was from an iMac, which runs an Apple version of Unix operating system. The Unix traceroute command uses UDP rather than ICMP EchoRequest (basically PING) used by Windows devices, but still looks for the same ICMP Type 3, Code 12 or Type 11 (TTL Expired) response. The UDP based Traceroute is generally more reliable at getting through, but the responses can still be blocked.

On a Mac, you can force it to use ICMP Echo for Traceroute by adding the -I option to the command.

Re: unix traceroute using UDP - You are correct (and I've learnt something today). I was always under the impression that trace routes where pure ICMP, and in the Windows world (where I've done most of my work) they are ICMP's.

That said my traceroute tests where done from a Linux machine, so I was getting through using UDP packets, the same as what JB was trying.

Certainly could be the case. But if this is the case, the administrator needs serious education and change control practice needs to be reviewed.

Quick and easy way to test on a Windows PC is to use the NETSH command to temporarily reduce the MTU on the client to say 1400 bytes and test the connection again.

The more I think about it, the more I'm thinking overzealous IDP rather than any human activity. We've certainly experienced problems in the past with overzealous IDP's assuming normal looking traffic was an attack. I could imagine an IDP blocking a range of IP's if it detects an attack from a network.

Mistakes in changes do slip in, even with strong change control culture, but it's unlikely that an administrator would add in a blocking rule, or a bad route for a specific range of IP's from a completely external network, or that such a mistake is made.

What language are they speaking .

Geeklish...

 

[mjt57]

Jeez, JB, look what you've started....

 

[TheRealTMA]

Since the original problem has now been diagnosed and I believe fixed, this can now be a more general discussion.


Correct, from JB's (and others) address, it hits that router (or more likely firewall / IDP) and gets blocked at that point. I'm coming from a different IP address when I do my trace and thus I'm allowed through.



Re: unix traceroute using UDP - You are correct (and I've learnt something today). I was always under the impression that trace routes where pure ICMP, and in the Windows world (where I've done most of my work) they are ICMP's.

That said my traceroute tests where done from a Linux machine, so I was getting through using UDP packets, the same as what JB was trying.



The more I think about it, the more I'm thinking overzealous IDP rather than any human activity. We've certainly experienced problems in the past with overzealous IDP's assuming normal looking traffic was an attack. I could imagine an IDP blocking a range of IP's if it detects an attack from a network.

Mistakes in changes do slip in, even with strong change control culture, but it's unlikely that an administrator would add in a blocking rule, or a bad route for a specific range of IP's from a completely external network, or that such a mistake is made.




Geeklish...

This is a more sensible solution. As I suggested above, the issue seems to be more like one where the specific ip range was mapped out or blocked because of some mistake in blocking an ip range due to DDOS or legal takedown order. Traceroute is very unreliable since many routers will block the ping (whether UDP or ICMP) for the same reasons or prior DDOS attacks.

 

[jb747]

Well, I did ask the IT gurus...and found we have a few.

 

[mjt57]

And guess what, folks? The problem has returned. At least for me, it has.

 

[jb747]

And me too. I guess another complaint tomorrow....

 

[mjt57]

And me too. I guess another complaint tomorrow....

I still have the original email from them responding to my fault report. I replied back to that.

 

[jb747]

New trace route...

1 192.168.178.1 (192.168.178.1) 0.746 ms 0.405 ms 0.452 ms
2 loop1801501200.bng1.mel.aussiebb.net (180.150.120.1) 23.704 ms 24.303 ms 23.990 ms
3 tengige0-1-0-21.win20.melbourne.telstra.net (139.130.190.61) 23.811 ms 24.332 ms 24.682 ms
4 bundle-ether12.win-core10.melbourne.telstra.net (203.50.11.111) 26.237 ms 25.369 ms 25.683 ms
5 bundle-ether12.ken-core10.sydney.telstra.net (203.50.11.122) 36.004 ms 36.672 ms 36.055 ms
6 bundle-ether1.pad-gw11.sydney.telstra.net (203.50.6.61) 35.199 ms 37.458 ms 35.979 ms
7 bundle-ether1.sydp-core04.sydney.reach.com (203.50.13.90) 36.263 ms 37.490 ms 36.666 ms
8 i-0-1-0-47.sydp-core03.bi.telstraglobal.net (202.84.222.25) 37.325 ms
i-0-1-0-45.sydp-core03.bi.telstraglobal.net (202.84.222.17) 36.578 ms
i-0-1-0-47.sydp-core03.bi.telstraglobal.net (202.84.222.25) 36.378 ms
9 i-10701.paix-core02.telstraglobal.net (202.84.138.50) 223.448 ms 223.129 ms 223.408 ms
10 i-92.paix02.telstraglobal.net (202.84.247.41) 222.850 ms 222.857 ms 222.680 ms
11 comcast-peer.paix02.pr.telstraglobal.net (134.159.63.158) 222.580 ms 222.588 ms 222.620 ms
12 be-11083-cr02.sunnyvale.ca.ibone.comcast.net (68.86.84.13) 223.142 ms 224.754 ms 223.108 ms
13 be-11015-cr02.losangeles.ca.ibone.comcast.net (68.86.86.98) 225.030 ms 225.066 ms 224.518 ms
14 be-11580-pe02.losangeles.ca.ibone.comcast.net (68.86.82.34) 222.372 ms 222.158 ms 222.363 ms
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
And so on

 

[harvyk]

New trace route...

1 192.168.178.1 (192.168.178.1) 0.746 ms 0.405 ms 0.452 ms
2 loop1801501200.bng1.mel.aussiebb.net (180.150.120.1) 23.704 ms 24.303 ms 23.990 ms
3 tengige0-1-0-21.win20.melbourne.telstra.net (139.130.190.61) 23.811 ms 24.332 ms 24.682 ms
4 bundle-ether12.win-core10.melbourne.telstra.net (203.50.11.111) 26.237 ms 25.369 ms 25.683 ms
5 bundle-ether12.ken-core10.sydney.telstra.net (203.50.11.122) 36.004 ms 36.672 ms 36.055 ms
6 bundle-ether1.pad-gw11.sydney.telstra.net (203.50.6.61) 35.199 ms 37.458 ms 35.979 ms
7 bundle-ether1.sydp-core04.sydney.reach.com (203.50.13.90) 36.263 ms 37.490 ms 36.666 ms
8 i-0-1-0-47.sydp-core03.bi.telstraglobal.net (202.84.222.25) 37.325 ms
i-0-1-0-45.sydp-core03.bi.telstraglobal.net (202.84.222.17) 36.578 ms
i-0-1-0-47.sydp-core03.bi.telstraglobal.net (202.84.222.25) 36.378 ms
9 i-10701.paix-core02.telstraglobal.net (202.84.138.50) 223.448 ms 223.129 ms 223.408 ms
10 i-92.paix02.telstraglobal.net (202.84.247.41) 222.850 ms 222.857 ms 222.680 ms
11 comcast-peer.paix02.pr.telstraglobal.net (134.159.63.158) 222.580 ms 222.588 ms 222.620 ms
12 be-11083-cr02.sunnyvale.ca.ibone.comcast.net (68.86.84.13) 223.142 ms 224.754 ms 223.108 ms
13 be-11015-cr02.losangeles.ca.ibone.comcast.net (68.86.86.98) 225.030 ms 225.066 ms 224.518 ms
14 be-11580-pe02.losangeles.ca.ibone.comcast.net (68.86.82.34) 222.372 ms 222.158 ms 222.363 ms
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
And so on

That's a bit strange, could you also include the first line of the trace which will say something like
traceroute to www.australianfrequentflyer.com.au (192.69.193.66), 30 hops max, 60 byte packets

Again, this is nothing you can really do from your end. But including the first line will give enough information to let the right people know where the problem might be.

 

[jb747]

traceroute to australianfrequentflyer.com.au (192.69.193.66), 64 hops max, 52 byte packets

 

[Franky]

Thanks ... I am still confused as I am not aware of an AFF app.

Do you have a browser shortcut icon?

Serfty - I don't know what a browser shortcut icon is, perhaps it is what I am calling an app? I have taken a screen shot of my iPad screen, please advise me how to load it or an email address to send it to you.

 

[jb747]

Would those of you who are having problems mind posting your IP address?

I've been moved, but still in the 180.150 address range. Problem persists.

 

[sudoer]

Would those of you who are having problems mind posting your IP address?

I've been moved, but still in the 180.150 address range. Problem persists.

180.150.65.x here but no issues.

 

[mjt57]

Would those of you who are having problems mind posting your IP address?

I've been moved, but still in the 180.150 address range. Problem persists.

I’m at work. I’ll check tonight when I get home.

 

[mjt57]

Would those of you who are having problems mind posting your IP address?

I've been moved, but still in the 180.150 address range. Problem persists.

My WAN IP4 address is 180.150..*.

I don’t know what it was before the problem originally surfaced.

 

[serfty]

There is indeed work going on in an effort to rectify the situation.

ABB have been in contact with AFF regarding these issues.

 

[TheRealTMA]

There is indeed work going on in an effort to rectify the situation.

ABB have been in contact with AFF regarding these issues.

So? What technical info have you been given which explains the blocking of AFF IP address? This would immensely help those affected, especially when the situation happens again and they have to talk to tech support at ABB.

 

[sudoer]

ABB have been in contact with AFF regarding these issues.

This is one of the reasons I'm with ABB - they actually care. Reminds me of the good old Swiftel ADSL days.