Forum Discussion

PMilne87's avatar
PMilne87
On our wavelength
2 years ago
Solved

Virgin Media DNS servers completely dropped from our website DNS zones

Hi there, 

I work at Marys Meals International and as of late Thursday evening, 1st Feb, Virgin Media and another US ISP appear to have been completely dropped from our website's DNS Zones. What should happen is if it doesn't have a zone, then it should query the root DNS servers globally to pull down a copy of the zone and cache it. But that's not happening, so I suspect their DNS zones have got corrupted somehow at Virgin (or in their upstream provider). That is what happened with Sky last year.

This has probably been a maintenance/update on your side which has gone awry.

Our DNS Provider has website called DNS Hospital that you can use to query what otherDNS servers return. I'll past the link below. It doesn't have a valid SSL certificate (never has had) so you will get a warning that anything entered in the site isn't secure 

If I input your Virgin DNS servers:

No results at all

https://www.dnshospital.com/cgi-bin/dig.cgi?d=marysmealsusa.org&n=194.168.4.100&q=ANY

Google DNS shows full listings: https://www.dnshospital.com/cgi-bin/dig.cgi?d=marysmealsusa.org&n=8.8.8.8&q=ANY 

Thousand Eyes Outages map shown that "Level 3 - A Major USA DNS Provider Has Issues". Are these incidents related and can you please re-run your DNS Servers to be added back onto our Zones?

Affected sites:

  • Robert_P's avatar
    Robert_P
    2 years ago

    Hi MarysMeals

     

    Sorry to hear of the issues experienced, we have raised this and passed the details provided to our team to investigate further. Once we get an update on this will let you know here.

     

    Rob

17 Replies

  • PMilne87's avatar
    PMilne87
    On our wavelength

     

    Please see above CMD Prompt and below email from Blackbaud Online

     

  • Client62's avatar
    Client62
    Alessandro Volta

    The nslookup.io Authoritative tab may be empty because the specified Name Servers are not responding.

    If I try these commands they both time out :-

    nslookup marysmeals.fr ns2.bb-online.org

    nslookup marysmeals.fr ns1.bb-online.net


    • MarysMeals's avatar
      MarysMeals
      Joining in

      That's very odd. I get responses from both these servers when querying them directly using a local nslookup tool.
      I wonder if there's any DDOS issues

  • Client62's avatar
    Client62
    Alessandro Volta

    What is also unusual is looking at nslookup.io for marysmeals.fr the public DNS are showing Revalidate in 5mins,  just as if they have all expired and are re-trying a refresh at short intervals.

    Below are the timeout I see :

    C:\Users\Philip>nslookup marysmeals.fr ns2.bb-online.org
    DNS request timed out.
    timeout was 2 seconds.
    Server: UnKnown
    Address: 185.206.123.20

    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    *** Request to UnKnown timed-out

    C:\Users\Philip>
    C:\Users\Philip>
    C:\Users\Philip>nslookup marysmeals.fr ns1.bb-online.net
    DNS request timed out.
    timeout was 2 seconds.
    Server: UnKnown
    Address: 185.206.123.1

    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    *** Request to UnKnown timed-out

    • MarysMeals's avatar
      MarysMeals
      Joining in

      The TTL on that domain has been lowered pending a website migration which could account for that revalidation you're seeing, but I'm still unsure why those BB-Online name servers are not showing as unreachable (same output below from 2 different ISPs outwith Virgin Media):

      ===========(Trooli DNS)

      C:\Users\Stephen>nslookup marysmeals.fr ns1.bb-online.org
      *** Can't find server address for 'ns1.bb-online.org':
      Server: UnKnown
      Address: 2a04:204:291f:c700::1

      Non-authoritative answer:
      Name: marysmeals.fr
      Address: 52.18.15.4


      C:\Users\Stephen>nslookup marysmeals.fr ns2.bb-online.org
      Server: ns2.bb-online.org
      Address: 185.206.123.20

      Name: marysmeals.fr
      Address: 52.18.15.4
      ===============

      ===============(Cisco DNS)

      C:\Users\Stephen>nslookup marysmeals.fr ns1.bb-online.org
      *** Can't find server address for 'ns1.bb-online.org':
      Server: dns.sse.cisco.com
      Address: 208.67.222.222

      Non-authoritative answer:
      Name: marysmeals.fr
      Address: 52.18.15.4


      C:\Users\Stephen>nslookup marysmeals.fr ns2.bb-online.org
      Server: ns2.bb-online.org
      Address: 185.206.123.20

      Name: marysmeals.fr
      Address: 52.18.15.4

      ===================

      Are you querying these nameservers from within the Virgin Media network?

  • legacy1's avatar
    legacy1
    Alessandro Volta

    if I run nslookup on o2 I get

    C:\Users\_>nslookup marysmeals.fr ns2.bb-online.org
    Server: ns2.bb-online.org
    Address: 185.206.123.20

    Name: marysmeals.fr
    Address: 52.18.15.4

    C:\temp>tracetcp ns2.bb-online.org:53

    Tracing route to 185.206.123.20 [ns2.bb-online.org] on port 53
    Over a maximum of 30 hops.
    1 2 ms 2 ms 1 ms 192.168.255.247
    2 19 ms 20 ms 18 ms 192.168.8.1
    3 * * * Request timed out.
    4 * * * Request timed out.
    5 * * * Request timed out.
    6 * * * Request timed out.
    7 Destination Reached in 46 ms. Connection established to 185.206.123.20
    Trace Complete.

    on VM

    C:\Users\_>nslookup marysmeals.fr ns2.bb-online.org
    DNS request timed out.
    timeout was 2 seconds.
    Server: UnKnown
    Address: 185.206.123.20

    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    DNS request timed out.
    timeout was 2 seconds.
    *** Request to UnKnown timed-out

    C:\tracetcp_v1.0.3>tracetcp ns2.bb-online.org:53

    Tracing route to 185.206.123.20 [ns2.bb-online.org] on port 53
    Over a maximum of 30 hops.
    1 10 ms 7 ms 8 ms 10.112.32.133
    2 10 ms 10 ms 10 ms 80.1.81.185 [basl-core-2b-ae63-650.network.virginmedia.net]
    3 * * * Request timed out.
    4 * * * Request timed out.
    5 * * * Request timed out.
    6 19 ms 19 ms 19 ms 62.254.85.86 [86.85-254-62.static.virginmediabusiness.co.uk]
    7 * * * Request timed out.
    8 31 ms 21 ms 45 ms 130.117.51.73 [be2348.ccr41.lon13.atlas.cogentco.com]
    9 30 ms 27 ms 34 ms 154.54.56.94 [be12194.ccr41.ams03.atlas.cogentco.com]
    10 35 ms 33 ms 89 ms 130.117.0.122 [be2813.ccr41.fra03.atlas.cogentco.com]
    11 43 ms 40 ms 41 ms 154.54.36.54 [be2959.ccr21.muc03.atlas.cogentco.com]
    12 45 ms 45 ms 42 ms 130.117.0.17 [be3072.ccr51.zrh02.atlas.cogentco.com]
    13 44 ms 44 ms 46 ms 154.54.39.158 [te0-0-1-3.agr11.zrh02.atlas.cogentco.com]
    14 46 ms 44 ms 44 ms 154.25.7.66 [te0-0-2-2.nr11.b046110-0.zrh02.atlas.cogentco.com]
    15 49 ms 46 ms 48 ms 146.228.52.155
    16 46 ms 43 ms 52 ms 94.177.122.250 [vbgp01.4b42.net]
    17 * * * Request timed out.
    18 * * * Request timed out.
    19 * * * Request timed out.
    20 * * * Request timed out.
    21 * Destination Reached in 45 ms. Port closed on 185.206.123.20
    Trace Complete.

  • The auth servers ns1.bb-online.net & ns2.bb-online.org are responding to ICMP ping, meaining it's definitely reachable and routing OK from my VM connection. However DNS querying the servers using port 53 is timing out.

    Very weird indeed.

     

    Maybe this has something to do with it: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy-geo.html

    "Geolocation works by mapping IP addresses to locations. However, some IP addresses aren't mapped to geographic locations, so even if you create geolocation records that cover all seven continents, Amazon Route 53 will receive some DNS queries from locations that it can't identify. You can create a default record that handles both queries from IP addresses that aren't mapped to any location and queries that come from locations that you haven't created geolocation records for. If you don't create a default record, Route 53 returns a "no answer" response for queries from those locations."

    • MarysMeals's avatar
      MarysMeals
      Joining in

      Does anyone from VM networking support see forum posts, or is there anyway for customers / miderators to elevate support cases for their review? I'm unable to reproduce these issues outside of the VM network and not a customer so have no direct access to business support. Our users who are on VM residential services are simply having their line tested by support engineers and being informed there is no fault.

      • PMilne87's avatar
        PMilne87
        On our wavelength

        I second this. Virgin Media have offered no help when I have tried to speak with them via Live Chat. We really need a moderator to escalate this case asap.