This post is a plea for help from smart network administrators/geeks and/or Bob Metcalfe:
Like a lot of dads, I’m the on-site technical support for my household and while that job used to just involve keeping software updates fresh on a single desktop computer, dads like me now have a whole clutch of computers and other gadgets (DVD players, refrigerators, tablets, laptops, TVs, cameras) — all of which have IP numbers — and all of which are used by any one of four family members to gobble up bandwidth. Like a like lot of Canadians who get their Internet service from a major cableco/telco, I have a bandwidth cap. Exceed the cap and the Akin household pays through the nose. That makes the Akin household’s Accounts Payable Manager (my wife) very unhappy with On-Site Technical Support (that would be me.)
So I need help. Let me describe the genesis of today’s plea for help:
Checking the daily log my ISP keeps of the bandwidth consumed by our household we found that yesterday we gobbled up a whopping 17GB of incoming data. 17GB in one midnight-to-midnight period! For the life of me, I can’t figure out which device (or household member) was the big hog. So far as I can tell, the household members have fessed up to:
- One child watched about two hours of YouTube videos on his iPod.
- Two children played about six hours worth of Minecraft (Is Minecraft a bandwidth hog?)
- On-site tech support (me) updated printer drivers (250MB tops)
- There was the usual e-mail/Web surfing/social media participation which is usually good for about 2 GB
But we used 17GB! (Our previous single day high was 9 GB. Our 24-day average is about 5.5 GB) No one downloaded a movie or a single MP3. There are no servers running in the household. The wireless network is locked up good-and-tight. The network name and password are changed twice monthly.
So why and how did 17GB of data get into our household?
What would make solving this puzzle easy is a software tool that could be installed at the router (Cisco DPC3825 DOCSIS 3.0 Gateway, if you were curious)) that would do this:
1. Log inbound Internet packets/bytes and identify target device of that inbound traffic.
2. Alternatively, some utility on each Internet-capable device in our household that could track how much data it receives from the Internet (as opposed to how much data it receives from other devices on the wireless LAN) would also be useful. That said, we have, for example, an Interent-enabled television, Internet-enabled DVD players, and Internet-enabled gaming consoles and I’m not sure I would even know how (let alone have permission) to modify the software on my DVD player.
So the router route seems the best one. So that at the end of the day I can get my router to provide me with this report:
SUMMARY FOR [DATE]-------------------------DEVICE INBOUND MBDVD 1 892DESKTOP 1 3133TABLET 1 1202SMARTPHONE 1 808LAPTOP 1 1445
And so on … This way, I could track which device was responsible for sucking down most of that 17GB and then, in turn, probably triangulate the user…
So geeks: Anything out there that can help me track how much Internet is being gobbled up by each device on my local WAN?
I’m no expert, but having a similar situation of seven Mac laptops, 2 iOS devices, one Android tablet on a 20GB/month satellite connection.
I run “NetUse Traffic Monitor” on my Mac which logs hourly, daily, weekly, monthly totals and displays the current average for the last 30 seconds. It works with most routers that have SNMP function enabled (Airport Extreme Base Station).
Unfortunately it does not provide data for client usage. So far I have not found anything that would log WAN traffic only, separated by client use. I don’t think there is any way to combine all devices together in one application.
To see the client traffic we use the Network Monitor function of “Little Snitch”, one of the first programs we install on every Mac. It logs usage of each client since loaded and can display usage per application.
I have a script running that restarts this app at midnight to give me a one day view of activity.
On the iPad/iOS we run “DataMan”. It logs total usage in various ways and per App but has a problem with displaying what Apple’s Push Service and Sync Service uses and I suspect these to be the big bandwidth hogs on our network. We have some really high unexplainable usage since updating to iOS 6. I’ve been trying to shut off anything that could be running in the background, or telling the gamers to close their games or play in Airplane Mode as much as possible.
I have tried “Rubbernet”, which can collect usage from clients, but it had high memory usage doing so.
There is no easy way getting a grip on data usage with so many different devices.
In order to get some sort of reporting like you mentioned on a per device basis you’d probably have to reflash the firmware on the router which is not something I recommend for a non power user.
If you are a power user and want to learn more: http://www.dd-wrt.ca/site/index
It’s entirely possible that iTunes could have re-downloaded the entire library if you use Apples cloud based services.
David,
What you are looking to do is a good start, but you also need to look at what has changed within your home network. You don’t identify when your previous single-day high of 9GB occured and what has been added to your home network since that time. What items were purchased over the holidays that would have been connected to your network prior to your recorded high of 17GB usage? Could be one of those new items, initiated an internal update from the internet, which would be outside of your control. Your print driver updates, did you need to install a software update to your Operating System in order for the updates to install? If so, that update could have been quite large. You state you do not have a server, how do your print-driver updates get “pushed” to the other PC’s/Laptops on your network, are they updated individually, in which case you must download the print-drivers multiple times? Good luck as this same type of investigation would take place daily in most business environments and our toolsets are quite expensive and out-of-reach for most home-based Tech teams.
Just get service from an ISP that does not put a cap on bandwidth for the same if not lower cost than a major ISP. Then it won’t matter and you will never have to stress about it again, which in turn will keep your wife happy.
While there may technical ways to accomplish what you want, IMHO I think you’re going about this the wrong way: first off, I’m going to go out on a limb and assume that you’re with one of the big incumbent telcos.
You should not be the one that has to change his behavior for simply using a service “normally”. I think the problem is not that the household is using too much of your quota, but rather your cap is too low. Instead of jumping through hoops to stay under, I’d put the effort into finding alternative service providers.
A work colleague is with TekSavvy (in the GTA), and he has a 300 GB cap. Of course he does go over that on a semi-regular basis, but he’s never actually been charged for it. TekSavvy also allows you to pay an extra few dollars to have a plan without any cap. You may want to check them out:
http://teksavvy.com/en/residential/internet/cable
I know two other people who use their DSL service, and are quite happy. I’m currently stuck with one of the incumbent DSL providers for various reasons, so I don’t have a horse in this race.
There are other ISPs available as well:
http://www.canadianisp.ca/index.php
Another work colleague is with Acanac, and also fairly happy with them. (I’m hoping to move away from my current Big Telco DSL company to TekSavvy or Acanac as soon as I can.)
Bandwidth caps are complete BS, and jumping through hoops to stay with-in them is encouraging the wrong behavior. Don’t reward the your current company with your business: do a bit of digging to see what alternatives are available in your area, and see what their prices and caps are. Then call up your current company and ask if they can match the other company’s plans. If not, start proceedings to switch. (After which you may be getting a call from a retention specialist.)
Seriously: you’re not the one that should be jumping through hoops.
I am not sure if you can get anything to work with the router itself as I imagine the config files are locked down by the ISP (I assume you are renting it through them)
For this (and many other reasons) you may want to look at getting a wired modem from them and setting up your own wireless router that would allow you to manage it to allow this kind of monitoring.
The other option is to install a bandwith monitor on each computer in the household.
Most routers and hubs are switches which means traffic is directed from one device to another and a bystander cannot see what is going on. For example, your desktop will not be able to see how much traffic is being used by the laptop. So the router is the best choice to monitor, if you can, as it will see traffic from everything.
The report you produced accounts for maybe 7 gig. That leaves 10 gig going up to the internet. By the numbers here I suspect it is the desktop that is consuming bandwidth. There is always traffic both ways as each end of the connection acknowledges reciept of information from the other end of the connection. A large amount of inbound traffic could be a symptom of a larger amount of outbound traffic.
I would reboot the desktop and run a good virus scan on it. Then run an anti-malware on it incase there is something that is hiding from the virus scanner. You may have an anti-malware component to your virus scanner.
That is the simple stuff. Monitoring the outbound traffic would be the best thing to do on your router if you can, then investigate the system with the highest volume.
Ron
If you are adventurous and not afraid to brick your router you can install alternative firmware on it like Gargoyle that will give you the ability to identify the bandwidth hogs and/or enforce restrictions. Your other choice is to install a freeware bandwidth monitor on each device. I don’t know what your current cap is but I switched from Cogeco Cable and their measly 80GB/month to start.ca and get 300GB and a faster connection.
@Shane — I’d definitely put me in the “power user” category so this is helpful. And the only thing Apple’s Cloud is used for is the Calendar. (Movies, music, photos moving through Apple’s Cloud is disabled). Tx.
@OldSchoolTech First thing I asked myself when I saw my daily bandwidth gobble start to rise is what new software/hardware has been added to my home network. Apple’s latest iOS and latest OS make heavy use of Apple’s Cloud service (“It’s a feature not a bug!”) but I quickly figured that out and disabled most Cloud services (see above). Nothing but a $40 wireless printer from HP was connected over the holidays and I don’t think that printer has local storage for even a gigabyte let alone 17. No updates to the OS have been installed. Notifications that software updates get pushed to all devices but I’m the only admin that can actually being the “download and update” process. Tx!
@Joshua Davis : Almost certainly that is the step that’s going to be taken.
@David Magda : You guessed right — I’m with with one of the biggies and, as I tried to explain to their tech support people, I cannot be the only who is trying to get a handle on managing data on my local WLAN (or LAN). Their tools only measure what is coming from their servers to their router (the gateway to my WLAN). Great. Their customers need to know where their router is now sending that data. I realize that, for privacy reasons, ISPs should not be monitoring traffic past that gateway but it seems it would be a real competitive advantage for any ISP that provided its customers easy tools installed at the gateway (i.e. the modem or router at the customers’ premise) that the customer could use, if s/he chooses, to audit and manage WLAN traffic. And you’re bang on that I should not be one jumping through hoops. It’s a good thing I’m the jumpy type 🙂
@Steve: Read the post closely: Some of these IP-enabled devices are DVD players, televisions, etc. — devices that run on operating systems that are well beyond the ability of me and most “prosumers” to modify. It ain’t the computers I’m worried about. Computers I know I know how to manage. It’s all the other IP-enabled gadgets.
@Ron : The “report” I produced is a hypothetical report I wish existed.
@Arch Stanton: Gargoyle, eh? Ok, looking into it.
There are a few things you can try to move farther down the path you’ve begin exploring.
First, on your PC(s), install and run a netwok monitoring tool (e.g. Microsoft’s (free) Network Monitor tool (google for it) or WireShark (used to be called “Ethereal”) from wireshark.org.
That will allow you to break down the packets flowing from your PC. It won’t give you a deeper understanding of traffic from other devices, but it’s a start.
I presume you’ve been reviewing the various logs on your modem/router (Under Administration / Reporting -> View Log, according to the manual)
To monitor other traffic you might need to separate the modem function from the router function (i.e. buy another router and make it the ONLY device connecting to the modem. Connect wireless devices through this new router, too.) Then, you would need to add another device (switch) between the modem and the router and set it into promiscuous mode with a network monitor attached so that you can “tap in” to all of your network-bound traffic.
That isn’t the only (or necessarily the best) way to tap all of your network-bound traffic, but I don’t see anything in your modem configuration to do it with just that device, so it’s going to get complicated no matter what.
All the best!
Do you use Dropbox on various devices? Or similar services from Microsoft, Ubuntu, etc? (You mention that your iCloud is restricted). Multiple devices on Dropbox, especially if some folders are simply re-structured, could create a lot of bandwidth usage.
I myself have had this issue when my son watches a lot of youtube videos and also from some games sites as well. I have two routers. One is from my ISP and that has my main PC and my Netbook connected through it. The other is connected to the first router and provides wireless to my kids PC’s and their game systems. The second router lets me control when they can access the web and also what sites they can go to. When I see my usage going up I restrict access to youtube and netflix or just turn off their router completely. I know this isn’t a way to find the specific culprit but it might help. Also is your router properly secured to disallow anyone not from your house to use it. I restrict mine by MAC address.
@Paul I (and others) (1) There are no operating systems on any device in this household which run a Windows operating system. (2) Allowances were made months ago for DropBox, Google Drive, etc. It ain’t that. Remember: Issue cropped because after, chugging along at 5 GB of bandwith consumption on most days, one day we hit 17 GB. I want to know why.
@Jim Richmond: Tx Jim — Router and all devices are properly tied down. FYI: Been around a few years on this. Was an AOL member, for example, when AOL had just 5,000 members. Yup. 5,000. That’s how far back I go in Internet time.
Also keep in mind that online games, can and do update, and if it’s any type of mmo(World of Warcraft and many many others), the updates are getting larger and larger. Also if you dont have anti-virus, your computers can be used to assist bot wars in IRC, this has been the cause of some of my family members going over their limit. You need to make sure that your antivirus has a boot time scan. I also reccomend spybot search and destroy which will help remove rootkits and other malaware. Also bit torrents when you forget to turn em off if you use them. Either way the limit itself isn’t the problem, it’s the changes to bandwidth that providers have installed so that users pay more for what they consume. That’s their story anyways. It’s simply a cash grab by ISP’s, and their lack of competition which allows them to use such tactics and get away with it.
There are quite a few technical ways to accomplish what you want, but they tend not to be present on “residential” or “consumer” products (at least not with the default software/firmware), and mostly on “business” or “enterprise” class devices.
If you don’t want change ISPs at this time, I’d look at the the idea that “Shane” suggested: changing the software on your gateway/firewall.
Generally, when a household connects to an ISP at least one device is needed: the “modem”. This translates the ISP’s signals (DSL, cable/DOCSIS) into Ethernet, which is what all computers come with today. (WiFi/802.11 is actually just an ‘extension’ of sorts to Ethernet to send the bits that normally go over cables over radio.)
The modem can also act as a gateway/firewall, but this usually isn’t strictly necessary. Instead of using a modem/gateway combination, many people hook up their modem to a router/firewall that’s made by Linksys/DLink/Asus/Netgear/etc.
Given that your “Cisco DPC3825” does not have the functionality you desire, then picking up a new router with the functionality you want would be the next most logical step. Some people have mentioned installing monitoring software on all your machines, but IMHO this is the wrong decision in the time/money/hassle trade off matrix. Given that you probably have a WiFi network, I’m guessing you have a wireless router already that can act as the gateway for your household (if it doesn’t already). If you do have a WiFi device, it may be worth mentioning the exact model in this thread, and also stating how it’s configured: is it just creating a wireless network off to the side, or is it acting as the gateway for the household?
If you don’t such a device, the Asus devices mentioned here are good choices:
http://thewirecutter.com/leaderboard/networking/
If you search for “Toastman TomatoUSB”, “Teaman Tomato”, and “DD-WRT monitor bandwidth per ip” you should be able to find links on getting per machine/IP tracking of usage with different software. Note that because there are a variety of routers, all made with different chips internally, there are a variety of firmware downloads to handle all the various combinations. It can be a bit convoluted.
I would like to re-iterate my thoughts that if you’re going to spend some time tackling this, I think it would be most productive to look at other ISPs–even if you only end up calling your current one to get a better deal.
As an addendum, if you have other technical questions, I’d recommend posting them to Super User:
http://superuser.com/
Even if you post here, and say “I’ve asked a question at SU, please check it out”, you’ll get access to a larger pool of folks. You don’t even have to register to ask (just fill in fake information).
It’s a helpful community, and it’s run by some some good folks. I use one of their IT-related sister sites (Server Fault) regularly. They have a number of high-quality Q&A sites on various topics (cooking, photography, English language usage, etc.):
http://stackexchange.com/sites
Whereas most sites are set up as forums which you have to wade through to find useful information, the Stack Exchange model is that each post is a single, focused question. Responses are answers to that specific question, and the best one is usually voted to the top (though you have to register to vote).
The only mention I made of Windows was that Microsoft offers a free network monitoring tool. I also mentioned a second free tool which runs on other OSs. And I discussed setting up a network bandwidth monitor probe to clearly identify the issue. But if you don’t want help, that’s fine. Your choice.