This is the website of Abulsme Noibatno Itramne (also known as Sam Minter). Comments here or emails to me at abulsme@abulsme.com are encouraged... or follow me on Twitter as @abulsme.

Categories

Calendar

June 2010
S M T W T F S
 12345
6789101112
13141516171819
20212223242526
27282930  

Getting Backblaze Statistics from the Command Line on a Mac

I’ve been trying to keep a little dashboard showing how well Backblaze is doing keeping up with backing up my primary system (a Mac).  I’d been doing this by hand every once in awhile by just taking the numbers of files and MB left shown by their icon in the menu bar.  But that was manual and tedious.  I wanted to automate it.  I Googled a number of times things like “Backblaze Command Line” to try to find a way to pull this info, but never found anything useful.  I poked around a bit looking for relevant log files, but didn’t find them until tonight.  I found what I needed, so I thought I’d post so that if anybody else is looking to do the same thing, when they Google, they will find the answer, rather than having to dig and find this from scratch.  Of course this is just for the Macintosh version.  I am sure there is something equivalent in their Windows software, but I don’t have a Windows machine, so I’m not going to be the one to find it there.

Anyway…

It turns out what you need is all in a nice little file:

/Library/Backblaze/bzdata/bzreports/bzstat_remainingbackup.xml

The contents are pretty straight forward:

<?xml version=”1.0″ encoding=”UTF-8″ ?>
<contents>
<remaining remainingnumfilesforbackup=”2414327″ remainingnumbytesforbackup=”459544254971″ />
</contents>

So, for remaining files needing to be backed up:

cat /Library/Backblaze/bzdata/bzreports/bzstat_remainingbackup.xml | grep remaining | cut -d \” -f 2

And for remaining bytes needing to be backed up:

cat /Library/Backblaze/bzdata/bzreports/bzstat_remainingbackup.xml | grep remaining | cut -d \” -f 4

I use these together in a quick little shell script that I run every few hours using cron:

#!/bin/bash
# Updates BackBlaze Stats
FCFILE=/Users/abulsme/Dev/MailCount/bbfiles.txt
MBFILE=/Users/abulsme/Dev/MailCount/bbmeg.txt
date “+%m/%d/%Y %H:%M:%S” | tr -d ‘\n’ >> $FCFILE
echo -ne ‘\t’ >> $FCFILE
cat /Library/Backblaze/bzdata/bzreports/bzstat_remainingbackup.xml | grep remaining | cut -d \” -f 2 >> $FCFILE
date “+%m/%d/%Y %H:%M:%S” | tr -d ‘\n’ >> $MBFILE
echo -ne ‘\t’ >> $MBFILE
cat /Library/Backblaze/bzdata/bzreports/bzstat_remainingbackup.xml | grep remaining | cut -d \” -f 4 >> $MBFILE

My actual script actually does one more thing to convert bytes to megabytes for my charts, but the above gets you the raw data.  What you do with it after that depends on your application and what you are trying to do with this info.

For me, I’m generating dashboard charts:

Everything prior to 2010 Jun 10 was me grabbing data manually for the graph.  June 10th forward is pulled automatically by the above.

Anyway, hope this ends up being helpful for anybody else wanting to programmatically monitor how well Backblaze is or is not keeping up.

Edit 2012 Jul 9 13:02 UTC:  In recent versions of Backblaze, the path that contains the report has changed to /Library/Backblaze.bzpkg/bzdata/bzreports/ .

3 comments to Getting Backblaze Statistics from the Command Line on a Mac

  • If anybody is curious, the “reports” files on Windows are identical in every way to the files on the Macintosh. At Backblaze we develop both the Mac and Windows versions in the same source tree and they share a massive amount of base level library code between them. One Windows, the “bzstat_remainingbackup.xml” file can be found here:

    Windows XP: C:\Documents and Settings\All Users\Application Data\Backblaze\bzdata\bzreports\bzstat_remainingbackup.xml

    Windows Vista and Seven: C:\ProgramData\bzdata\bzreports\bzstat_remainingbackup.xml

    For the programmers poking about in these files, it is ok for your scripts to read them quickly from time to time (this is what they are designed for), but please do not open the files exclusively and hold them open, Backblaze needs to update them. :-)

    — Brian Wilson
    CTO, Backblaze

  • Thanks for the additional information Brian!

  • Hello Brian, don’t know if you set this thread to email you when there were new comments or not, but thought I’d follow up here before trying to ping elsewhere. I’ve now been monitoring the Backblaze stats mentioned above for awhile, and am seeing something I don’t understand:

    Files left: http://www.abulsme.com/dashboard/bigdash.php?df=bbfiles.txt&df2=&w=800&h=450&ss=&mult=&days=365.242

    MB left: http://www.abulsme.com/dashboard/bigdash.php?df=bbmeg.txt&df2=&days=365.242

    Age of last backup: http://www.abulsme.com/dashboard/bigdash.php?df=bbsincelast.txt&df2=&days=365.242

    (That last is in Ms, multiply by 11.6 to get days…)

    Anyway, a couple of weeks ago, the “Date of last backup” started reporting that the backup was up to date, no more than a few days old. But yet the number of files left remains over 2 million, with over 250 GB remaining. I have verified a few recent files, and the ones I spot checked are indeed backed up on backblaze, but if things were truly caught up, shouldn’t files and MB remaining also be bouncing around the bottom of a chart like this, rather than still showing so much left to back up?

    (Note, in March I upgraded to a new computer… restoring from the Time Machine archive of the old computer. Backblaze picked up and started backing up again, but once it got to zero files remaining it didn’t recognize any further changes until I upgraded the backblaze client, at which time it suddenly seemed to think it had to back up EVERYTHING again from scratch… it has been catching up from that ever since.)

    Anyway, let me know your thoughts. How does it saying it is generally caught up and keeping up jive with the huge backlog still reported in the other metrics? I would expect to generally have some non-zero amount of backlog due to daily churn in my files, but while I do probably have a decent amount of churn, it is not on the order of 250 GB!

    Thanks for your help!

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  

  

  

This site uses Akismet to reduce spam. Learn how your comment data is processed.