• Tag Archives network
  • Network issues at home – the kill packet

    Non-tech description first…….
    The network is ‘glitching’ every 1 to 3 days.
    I have no idea what it is, or why it is doing it. I just know it happens at any time it wants.
    When it happens, I lose network connectivity to the Internet and between devices at the house. This means that if my website is down, sorry. Send me an email. I do not have anything automatic set up to let me know when it goes down, because when it goes down, I cant send emails!
    It is driving me nuts!

    Techo version.
    The Asus router has a lot going for it. I really like a lot of its features and it seems to be powerful enough to do the job when things are going smooth.

    The frustrating thing is that I have not had more than 2 days of a working network since all this started. I have not been blogging about it because I have been spending my time trying to find and fix it. (The other generic blogs are all scheduled blogs that I wrote ages ago).

    It is annoying because once it goes down, the only way I can find to get things back up is to reboot the router.
    Here is the thing, I am not 100% sure it is even the router that is causing the issue.
    It seemed like it was at first. Like there was a memory leak or something (keeping an eye on the status page of the router does not show anything like that).
    I then setup a cronjob to reboot the router at 3:14am every morning….. When whatever it is that glitches does the same thing as when the router reboots. It will take out random devices around the network. Not just take them out, but sometimes services on those devices.
    For example. It will glitch one or more Linux PC’s. Sometimes they refuse to ping, sometimes they ping fine, but the web server service does not respond, or it pings fine, but the SNMP service goes south. Sometimes it is the Linux boxes, sometimes the Opto controllers, sometimes the Arduino, sometimes a combination of devices, sometimes a single device. The best way to get them back is to just unplug their network connection, count to three, plug it back in and they are fine.

    Why is it not the router? Because I have had times when the router stays up, but that sort of glitch goes through the network and takes out one or more devices. I can not seem to find a way to log the system from the firewall on the Asus, so all the traffic data is bound up in the core Linux router log data. Having some issues finding a clear way to log it to the USB stick rather than RAM, so I have not been able to view any logs.
    Sometimes it happens when I am at home, and sometimes when I am at work.

    Is is like there is a kill packet floating around taking out stuff at random.

    The worst is when it happens and no one is at home.

    It reminds me of what was happening to Ipcop. Where it just glitched and went weird. I blamed Ipcop or the PC it was running on, but now the Asus is doing the same things.

    To top it all off, I am trying to monitor the network traffic to see what is going on and I have found that my network is circulating around 90 gigabytes of traffic every 24 hours… does that seem like a lot to you, because it seems to be an insane amount to me!

    I thought it might be the Verizon ONT, so we got the guy to come out and change it. Nice side effect is that we now have speeds over 100Mbs, so when it’s working, the Internet fairly screams at our place.
    I also changed out the two switches in the place for gigabit versions and that did not change the glitches, but again, when its working, everything hauls ass big time.

    Blood pressure was up again this morning. 132/93
    I do not have any clue what is going on. I do not have a plan to move forward.

    In other news, a blog reader, David, very kindly sent me a Ubiquiti Wifi AP to evaluate and integrate into the system. I have been using it because one of my frustrations with the Asus is that I can not seem to get more than 30Mbs speed from the Wifi…. The UBNT is doing the same speed…. it is nice to have something to compare against, so now I know it is my wifi devices… Only they get ~50+ at work, so I know it’s not the devices…..


  • Asus networking devices

    Mixed feelings about this one, but want to throw it out there and have the discussion rather than pretend it did not happen or does not happen.

    I am NOT going to relive it (far too stressful), but I had a very bad week (and a bit) of networking trouble at my house mid February 2016. Long story short, we went through 5 different routers before we found one that a) worked and b) that I could live with.

    The router I ended up with is an Asus RT-AC3100.
    Not that long ago, there is no way anyone should have used an Asus router on any network…..

    http://www.engadget.com/2016/02/23/asus-ftc-settlement-router/

    the FTC found that the Taiwanese manufacturer’s routers had critical security flaws despite its promise to consumers that the devices can “protect computers from any unauthorized access, hacking and virus attacks.”

    Hackers could easily exploit one of those bugs to access users’ web-based control panels and change their security settings. If the user isn’t exactly tech-savvy, someone with malicious intentions doesn’t even have to hack the device. He simply has to use ASUS’ default log-in credentials: username “admin” and password “admin.”

    So, like most home router manufactures, they cut a LOT of corners on security and, well, pretty much ignored it.
    They got caught. They got fined. And here is the interesting bit…..

    Over the next two decades, ASUS’ routers and their firmware will undergo an independent security audit once every two years.

    From here on, they have to hand over the code that runs on their (my) router and have someone poke under the covers.
    If you are a regular reader, you will know that I am not running stock Asus firmware. This means that the code I am running has already been inspected three times. Once by Asus, once by the auditors and once by open source community that work on the firmware I use.

    So yeah, mixed feelings. I wish all companies took security more seriously, but that is a hope wish dream…. So in the meantime, getting caught and having to show your cards every 2 years for the next 20 is better than nothing.


  • Network issues at home – the last 36 hours

    Wow. Just. Wow.
    I am spent.
    What a mess…..

    I have proof. I can show you the trend line on groov. I was in church.
    There is no way I had anything to do with it. Nothing!

    Let me back up a little…… Friday afternoon, I put the IP cop in.
    Wrote the last blog. (No idea why it did not publish then, but it was probably me – I schedule a lot of my blogs). Had Friday night dinner with the family thinking it was all behind me.
    Saturday morning. All was right in the bald geeks network.
    Church. Streamed the service and it went smooth.
    After church spent some time with Terry.
    After dinner tried to visit my website….. Nothing…. Whaaaaaa.
    Did some little digging. All outbound connections were fine. All inbound that went to my IP address worked, but anything that went to thebaldgeek.net was getting rejected.
    Doing a ping of my URL worked and showed the correct IP address.
    As such, some of my aircraft tracking stuff was working, some wasn’t.

    Thinking it was a caching issue, I got an old laptop out that had not been used for at least a week, tried to get it on the network……
    IPcop would not give it an IP address to get it on the network.
    Link light, but no way I could get it to pick up an IP address and gateway.
    It was around 9:30pm by this stage.
    Rebooted IPcop for the 3rd time.
    Nothing.

    By 10:30pm I give up. My site is down. Had been for many hours (it stopped working around 11:50ish am – like I said, while I was ‘in’ church)…. and I have no idea why or how to fix it.
    I thought about putting the edge router back on, but as I said, I was spent.

    Went to bed.
    Did not sleep much.
    Got up around 4am.
    First thing in the morning as usual, took my blood pressure, it came up in big red numbers for the first time ever… waited a few minutes, took it again. Better, not good. Gave it a few more minutes, and got my highest reading (128 over 84) ever.
    Went for a run (it’s my thinking time).
    Came back and did some reading….

    There were plenty of people that said ‘why don’t you just put in the same router as you had in the first place?’… Two main reasons, it ran hot. Really hot. (This is why they have been failing for others, and probably was the cause for mine to fail).
    The other reason is because it only allowed 32 port forward rules and 32 static IP address reservations. Both limits I bumped up against often.

    Asus’s second latest router, the RT-AC3100 solved the heat issue and in my Googling found that there was some third party ‘WRT’ like firmware for it that removed both of the limits (and added some sweet traffic logging). It also added OpenVPN (as well as the stock PPTP VPN option).

    2-22-2016 9-19-10 AM

    So, that was that then.
    I went down to BestBuy and got one.
    Once I got back, about 20 min later I was up and running. 2-3 hours latter, I had things dialed in pretty sweet.

    Given that IPcop ran fine for some 14-18 hours before it fell over, I am not counting my chooks just yet…. But, I am hopeful.
    I have a VOIP phone, I can skype and my RSS feed is now working again. (Another weird thing that stopped working a few routers back).

    Freddy wanted to know why I did not just get this router last Sunday……

    So, in case you are like my wife and have been keeping track. A total of five (yes FIVE) routers have been abused the past week.

    I had no idea how much I depended on my network to do my job. I had no idea how fragile it was. I had zero clue how hard it would be to get it setup the way I needed.

    I am spent. But we seem to be back up and running.


  • Network issues at home – A new hope

    My Aussies mates are great.
    32 emails and a solid phone call latter and they almost have me sorted… well, ok, my network sorted….
    Dan and Gary really were a huge help in helping me see through the fog and see my options and a way forward in a much better light.

    Long story short, we ended up modifying a small computer, installing Linux on it with a software package called IPcop.
    At this stage, we have it in and running things. There is some mopping up to get the last of the control devices readdressed, but the point is, things are back up and running.

    There are a few quirks with the new router software. For example, a lot of the daily tests that I do with networking equipment and services are on non-standard port numbers, this means in the new software, I have to first make a ‘service’ for that port number, then I can open the firewall for it and do the port forward. It is surprising how annoying it is. I am still trying to find a way to label the service so that it makes sense, is quick and easy for me to find and makes it easier to read my firewall rules. (You can not rename your services once they are in use, so it has been a drag getting things working).

    The wifi is also a work in progress. The new router does not include wifi which was deliberate. I was not happy with the wifi for the past, well, 5+ years, the whole time we have been in this house it has been hit and miss with the coverage and speeds, so it has been on my to-do list to fix for that whole time… In the past it has been tied to my main router, so was hard to get interested about fixing it, but now that it is totally separate, I am actually excited about fixing it… AND since the router was free, I hope I can use a bit of the budget for that on wifi access points.

    Backing up the network is another issues I wanted to address. If it crashes (or breaks) again, I DO NOT (can not) go another week like the past one…… I need a solid disaster recover plan in place to get the network back to where it was. The new software and hardware should offer this, and it is something I will be testing once the dust settles.

    Lastly, I do not have as much visibility of my network traffic as I hoped. A lot of my testing can sometimes get out of control…. I can have a service spew network traffic loud, far and wide and it has been hard to see that in the past, I wanted to address this in anything I implemented, so to that end, I hope to find some plugins or tools to help me see the invisible packets of traffic on the network (hint, TRON).

    Anyway, we are not totally out of the woods, but the fog, doom and gloom has lifted and I can breath (and sleep) again.

    Half a planet away, and my mates are still digging me out of holes.
    Auwsom.


  • Network issues at home – cant phone home

    From almost the first week we moved to The States, every Tuesday night I have called my Dad. We used the VOIP (phone over Internet) phone since it was registered in Ballarat, it is a local call for me.
    Keep in mind that we have been here seven point something years, so it has been a pretty regular occurrence…..

    Last night, pretty sure for the first time ever, I could not phone home because of technical difficulties.
    Seems that the new (very very temporary) router is blocking the SIP protocol that is required for my phone to reach the server in Ballarat.

    Here is a screenshot of my network as it currently stands….
    network down

    Two things, yeah, I have a software package that monitors all the devices on my network for me…. What? You don’t?… wow… weird man….
    Second thing, red is bad, green is good… The current state of my network is a shambles.

    The current plan is that my mate Dan is going to rescue my sorry mess.
    We are looking at putting a Mikrotik router into play…. bit of a learning curve, but I think the flexibility and tools it offers is going to set me up for a very long time and give me the features that I have been wanting for ages.

    In time, I should even be able to phone home. Sorry I missed the call Dad, but we are working on it as hard and as fast as we can….