NetworkManager: traffic statistics?


hmc

Active Member
Joined
Dec 19, 2011
Messages
787
Location
Bavaria, Germany
Hi guys,


it would be nice to monitor 3G modem traffic, as there are "flatrates", which are limited to a specific data volume per month.


Ideally, it should be configurable to automatically reset statistics on a specific day in the month, and to give a warning, if traffic within a specific time period (a month, reaching from day X of a month to day X-y on the following month) reaches a specific limit.


E.g. my data plan's billing is from day 10 of each month to day 9 to the following month, and allows 1GB of data volume.


So I'd like to set a warning at 500MB and one more warning on 750MB maybe, and counting shold be reset automatically on each 10th of the month.


Is there a way to do this currently, in a setup involving NetworkManager?


Any pointers to suitable existing software?


Thanks a lot!


Daniel
 
Well you're not going to like this, but my only suggestion is greping /sbin/ifconfig....

wlan0 Link encap:Ethernet HWaddr 00:21:85:7b:c4:9b


inet addr:192.168.1.102 Bcast:192.168.1.255 Mask:255.255.255.0


inet6 addr: fe80::221:85ff:fe7b:c49b/64 Scope:Link


UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1


RX packets:57796 errors:463 dropped:48102 overruns:0 frame:0


TX packets:48744 errors:3 dropped:0 overruns:0 carrier:0


collisions:0 txqueuelen:1000


RX bytes:64828945 (61.8 MiB) TX bytes:7811897 (7.4 MiB)


Interrupt:17 Memory:f82b0000-f82b0100

Obviously it wouldn't be wlan0, but you can see the RX bytes and TX bytes clearly. So someone could make a script that parses this to give you the data rates, and even dump the results to a file for persistence (both reset every boot, possibly reset when the interface goes up/down). And then make an XFCE applet out of it so you can see the rates and reset if you'd like.


RX byes and TX bytes also suffer from a 32 bit overflow, but I can't imagine that being a huge issue on a 3G network?


So a bit of work, but that'd be how I'd do it....
 
Sounds interesting! Thanks for the idea.


If no more convenient solution is reported, I may take this chalenge and build something usable around this idea :)


Daniel
 
Good, let us know if you whip anything up, I bet other users would find it handy.


Also, you can grab the raw data out of /proc/net/dev, which may be a little more efficient than /sbin/ifconfig.
 
okay, will do.


Added this to my todo list for developing, and also added a link to this thread, so I won't forget to post here if I have something.


But it will take some time. Don't expect anything before maybe February! I have so much to do on the Pandora :)


Daniel
 
Hi guys,


I have found an existing solution now: vnstat.


http://humdi.net/vnstat/


Compiled successfully on my Pandora.


It gathers statistics about data volume transfered via the network interfaces and can be configured to the individual billing period to reset counters on a specific day.


Presentation of the data seems to be done in text files, as well as image files with nicely designed statistics.


The one thing, which seems to be missing, is a warning for the user on a specific configurable data volume. But I guess that can be added relatively easily in different ways. E.g. a cron job, which compares the current status with the configured value and, if reached or exceeded, sets a signal the user can see. In whatever form. Ideas welcome.


Now, how to configure and package vnstat?


- I think it does not make sense to make a PND for that. The vnstat deamon, if used at all, should run all the time.


- A package to be put into some feed (which one?) and to be installed manually via opkg by each interested user?


- Or should this, once preconfigured properly for the Pandora, go into the next Hotfix directly?


If the deamon should run or not could be configured the same way (and using the same dialog, with a new item for vnstat) as start of the deamons for dropbear, samba etc (Setting / Startup / Enable/Disable services on boot)


Opinions?


I think the vnstat deamon is useful for anyone using the Pandora to access the Internet via a USB 3G modem or similar device, which gets a dedicated network interface in the OS, in order to monitor transfered data volume. Most mobile "flatrates", at least in Germany, are limited to a specific volume, such as 500MB or 1GB, and once this volume is reached, providers limit access speed to (nominally) GPRS speed, but my experience is that the access is unusable then, i.e. limited to virtually zero bits per second.


However, mobile Internet via WiFi using a wireless 3G router would not play well with vnstat, because vnstat acts on network device level, and the WiFi device is the same, no matter if you connect via wireless router on the go or via cable/DSL router at home, so vnstat cannot distinguish, if you are online via volume-limited 3G modem or "unlimited" home Internet. But I assume that such wireless routers have an own volume traffic monitor anyway.


What do you think?


Daniel
 
I have found an existing solution now: vnstat.


http://humdi.net/vnstat/

Ah, vnstat! I use that on servers all the time. I don't know why I didn't think about that.


Personal opinion (and let the wiser devs correct me), you might be best off wrapping this up in a PND, actually (and though useful, I don't think it should belong in a HF by default).


I imagine a PND with three applications, a START (or install), CONFIGURE, and STOP application. Start would obviously start the daemon, and stop would stop it.


I imagine CONFIGURE could be used to control if it should automatically turn on or not. There should be a pre-up trigger for the network device, so if the user wants that, the PND could edit /etc/interface/whatever (or something around there) to start up the PND (and fail gracefully if the PND isn't there).


Obviously there'd be problems if the user ejected the SD Card while vnstat was running, which installing on the local system would fix. So that could be an option, too.


Hmm, the more I think, the more I don't know.
 
FWIW, I have a well working setup now on my Pandora:


- vnstat installed the standard way in the system (only a few small binaries, and a configuration file. About 270kB.


- I created a start/stop script /etc/init.d/vnstat, as for all the daemons


- I created an entry in the Settings > Startup > Enable/Disable services at bootup dialog, which lets the user globally switch on or off vnstat (default: off).


Planned:


A menu entry, probably under "Accessories", which displays the current vnstat statistics, easily done with the command



Code:
terminal --command=vnstat --title="VNStat Output" -H


That's all what's needed for basic operation of vnstat. Working very well here :)


I think PND is a great thing for games and applications, but after some consideration I found I would never put a deaemon inside a PND. As you say, user only needs to remove the SD card while the daemon is running, and then what happens....?


Since it takes up only 300kB of disk space,. I think it's not a problem having this on the system partition.


Are there other optional packages, not distributed as PND, but IPK? If so, where are they? (i.e. where should I put vnstat.IPK, once it's finished?)


And second question:


The command for starting the vnstat output using terminal, should that one be wrapped into a PND? Actually it would be enough, if I create a .desktop file launching the terminal command line. But will Minimenu see this, or does Minimenu need a PND file?


Daniel
 
why not package the ipk inside a PND so that the PND starts the ipk installer for people plus it means you can then add this to the repo too.


what I would suggest is making the pnd start script provide an install/unstall option incase the user wants to get rid at a later date, and just mention in the PXML that this writes to the OS which means if running the OS from nand it will write to the nand.
 
Good idea milkshake!


WIll do it that way, if vnstat should not go into the Firmware anyway.


ED, do you read this?


What do you think? PND with IPK installer, or should vnstat go into the next HF?


Anyone here willing to test the IPK or PND once it's finished?


Daniel
 
ok guys,


the vnstat PND is almost ready, but i have one issue. Maybe someone who knows vnstat can help me with that:


If vnstat is installed and the daemon is started for the first time, it creates databases for the currently active interfaces.


Then you start the Internet connection via UMTS 3G, which creates network interface ppp0 (which is the one you actually want to monitor probably).


Problem:


vnstat does not add ppp0 to the monitored interfaces automatically while running.


In order to let vnstat add ppp0 to its monitoring interfaces, you would need to stop and restart vnstat manually.


Is there any clever way around that? I could not find a setting in the conf file to let vnstat add new interfaces on the fly.


Thanks!


Daniel
 
correction:


Even restarting the daemon after establishing the ppp0 connection does not add ppp0 to the monitored interfaces!


ppp0 seems to be active while the vnstatd is started for the first time!


That means, the only solution I currently can think of is asking the user to establish the connection he wants to monitor and only THEN start the installation.


What do you think?
 
I took a look at vnstat and as far as I can see you really have to restart vnstat. The way i would try to solve this would also be to ask the user by using a zenity-script. Such a script should also be launchable after installation in order to change your setup:


First a zenity-info-message to "activate all interfaces that are supposed to be monitored and press 'OK'.", then show a list-window showing all active interfaces and for each the choice whether it shall be monitored or not. The command



Code:
ifconfig | grep "Link encap" | awk '{ print $1 '}
will output them, so you can feed it to the zenity-list. Once you press OK in the list window for each entry either "vnstat -u -i <interface>" or "vnstat -i <interface> --delete" would be called, depending whether the entry was chosen or not.



The man-page of vnstat also mentions the parameter "--enable, --disable" to "Enable or disable updates for selected interface. Useful for interfaces that aren't always available, like ppp0. If the interface goes down it should be disabled in order to avoid errors. Add something like vnstat -r --disable -i ppp0 to the script that's executed when the interface goes down and vnstat --enable -i ppp0 to the up script. These two options aren't needed when the daemon is used."

So something like "vnstat --enable" should somehow be integrated into the network-configuration. For example on my gentoo-pc I use





Code:
postup() {


[ "${IFACE}" == "usb0" ] && /etc/init.d/iptables start

return 0

}


in /etc/conf.d/net to automatically start iptables when i plug in my pandora (with enabled usb-network-mode) via usb and because of that usb0 comes up. The pandora-network-configuration differs though. Maybe you instead have to put a small script containing the vnstat-line into "/etc/network/" or so. Need to stop here for now. Sorry for the not too precise information.
 
Thanks for that...


I thought about a similar way, but vnstat -u does not lead to the new interface to be added, if it was not available on first vnstat start:



Code:
pan1sd:~/devel/vnstat-1.11$ vnstat -u -i ppp0

Error: Unable to read database "/var/lib/vnstat/ppp0".

Error: Unable to write database "/var/lib/vnstat/ppp0".


You write that --enable / --disable is not needed if the daemon is used.


I use the daemon. So this wouldn't help, right?


Daniel
 
Thanks for that...


I thought about a similar way, but vnstat -u does not lead to the new interface to be added, if it was not available on first vnstat start:



Code:
pan1sd:~/devel/vnstat-1.11$ vnstat -u -i ppp0

Error: Unable to read database "/var/lib/vnstat/ppp0".

Error: Unable to write database "/var/lib/vnstat/ppp0".

hm...are you trying this as normal user or root? Does /var/lib/vnstat/ exist after installation? What rights does it have? I just did test in on my gentoo-pc with usb0:


- installed vnstat, ran "vnstat -u -i usb0" while usb0 was active -> database for usb0 was added


- removed the entry by running "vnstat -i usb0 --delete --force" -> was removed


- deactivated usb0 and tried to add it -> didn't work, just as you already said


- enabled usb0 and again ran "vnstat -u -i usb0" -> was added


So somehow it should work after installation on the Pandora too.

You write that --enable / --disable is not needed if the daemon is used.


I use the daemon. So this wouldn't help, right?

To be honest I must say that i saw that after posting it. ;) Yeah, when you are using the daemon, then you do not need it. I by now started to think if there will be any advantage if you would NOT run it as a daemon, but so far I can't think of any.
 
Last edited by a moderator:
PND finished! :)


http://www.hermocom.com/en/downloads/openpandora/vnStat-Installer/


Lomaxx, I used your idea of feeding the available network interfaces into a zenity list and letting the user choose his interface to be monitored.


Of course the interface must exist at install time, so there is a dialog in the installation routine, which asks the user to establish the connection to be monitored.


It would be great if some interested people could test the PND.


Once it works reliably, I will put it into the repo.


Thanks for all your help!


Daniel
 
@hmc


I tested the pnd. Installation worked. Just a few minimal mistakes:


- I'd rather speak of "establishing connections" in the message-window, where the user shall press "ok"


- the option to choose a list-window doesn't really make much sense as any listed device will be added. In my case i had wlan0 and usb0 active. Both were added. So either just show a message saying "these devices will be monitored:" or if you are eager make a proper selection-mechanism where the ones that were not chosen are not being added. However this is more work. You instead could simply also tell in the message-box that the user should deactivate all interfaces that shall NOT be monitored.


-when showing the statistic through "start->network->Network Traffic Statistics ..." I can't exit by pressing q, ESC or ctrl+c, space, enter. A bit annoying, but nothing serious as i still can press fn+alt+f4. You could also use the command "watch -n <digit>" in your desktop file to keep the statistics updated every <digit> seconds. For example the command "watch -n 1 date" shows the actual time every second. Then ctrl+c would also exit if you remove "-H" from the terminal command.


Also IMHO it's better to use just "nvstat" instead of "nvstat -i usb0" when displaying the statistics, since 1.) the interface-name might differ (ah, i just understood that the selection-window of the installation affects this, but still) and 2.) the user might want to monitor more than one interface. Maybe add a quick note to the install-pnd that all this can be changed manually in "/usr/share/applications/vnstat_output.desktop".


- on deinstallation maybe add a "killall vnstat"?


Everything else seems to be fine from what i can tell after a quick test.
 
Last edited by a moderator:
Thanks Lomaxx,

- I'd rather speak of "establishing connections" in the message-window, where the user shall press "ok"

you mean plural instead of singluar? ok, makes sense.

- the option to choose a list-window doesn't really make much sense as any listed device will be added. In my case i had wlan0 and usb0 active. Both were added. So either just show a message saying "these devices will be monitored:" or if you are eager make a proper selection-mechanism where the ones that were not chosen are not being added. However this is more work. You instead could simply also tell in the message-box that the user should deactivate all interfaces that shall NOT be monitored.

The selection mechanism is meant to provide a choice for the "standard interface" (configured in vnstat.conf and used for the vnstat command issued by the Network menu entry to show the current status. )


This assumes that the user wants to monitor only one interface conveniently. He is still able to output statistics for all other monitored interfaces, via the command line.


I just found out that the --list dialog also accepts no selection and then returns nothing. So I could add to --title of that dialog "You may select a standard interface from the list below to monitor. If you choose nothing, you will see all interfaces in the output.!


What do you think about this?


I have done this now. You may test it if you like.

-when showing the statistic through "start->network->Network Traffic Statistics ..." I can't exit by pressing q, ESC or ctrl+c, space, enter. A bit annoying, but nothing serious as i still can press fn+alt+f4. You could also use the command "watch -n <digit>" in your desktop file to keep the statistics updated every <digit> seconds. For example the command "watch -n 1 date" shows the actual time every second. Then ctrl+c would also exit if you remove "-H" from the terminal command.

Good idea. Implemented the watch method.

Also IMHO it's better to use just "nvstat" instead of "nvstat -i usb0" when displaying the statistics, since 1.) the interface-name might differ (ah, i just understood that the selection-window of the installation affects this, but still) and 2.) the user might want to monitor more than one interface. Maybe add a quick note to the install-pnd that all this can be changed manually in "/usr/share/applications/vnstat_output.desktop".

1) is not an issue, because selection on install time sets the selected interface there.


2) could be made dependent on the additional choice at install time (see above)...? I have implemented it this way now. If you choose one at install time, -i variant is used, otherwise no -i.


The -i method makes for a nicer output and is more convenient for that kind of user who wants to monitor only one interface.

- on deinstallation maybe add a "killall vnstat"?

Oops, did I forget that one?? Bummer.


I also added a /etc/init.d/vnstat stop.

Everything else seems to be fine from what i can tell after a quick test.

Fine, thanks a lot for testing!


New version is online now, at the same link.


http://www.hermocom.com/en/downloads/openpandora/vnStat-Installer/


Daniel
 
Last edited by a moderator:
Back
Top