Some statistics and graphs of your LBP levels...

OK, so lbp.me is cool. It lets you view the plays and ratings of your LBP levels on the web. But it only shows a snapshot of these numbers at the time you view the page, and I was interested in seeing some more detailed stats and their trends over time.

So I wrote a quick program to grab the numbers from lbp.me level pages and log them to a .csv file so I could graph the results. The graphs turned out to be pretty cool, so I figured other people might be interested in using it for their own levels as well. So I cleaned it up and turned it into a little system tray application, and here is the result.

LBP Level Logger: http://code.google.com/p/lbplevellogger/

It's available for Windows and *nix, and may build for other platforms with small changes to the code (if anyone wants to get it working on another platform and submit a patch, drop me a line).

You run it, punch in your level details and it will regularly log the level's stats from lbp.me to a CSV file. You can then hit the Graph button to view a graph of your level's vital statistics. Leave the app logging for some time and you can end up with interesting graphs like these (generated from one of my own levels):

http://s10.postimg.org/s63dis009/sample_graph_2.png

Anyway, that's it. A little more info is on the project page. Any thoughts/suggestions/problems, let me know. :)

Enjoy!

The pic isnt showing up on my screen but this sounds very cool!

this is a very cool idea, and nice to see you take the intiative and build something like this, if my levels ever got played i would definitely try this :P

I really like the look of this, but I can't get it to work. Is there any other program other than LiveGraph I could use, or could you perhaps explain how to use it in more detail? Thanks :)

Looks cool dude :D

Quote:

---

Originally Posted by kirbyman62

I really like the look of this, but I can't get it to work. Is there any other program other than LiveGraph I could use, or could you perhaps explain how to use it in more detail? Thanks :)

---

There are others, but they wouldn't be simpler than this. Here's exactly how to open it in LiveGraph:

1) In LiveGraph's 'Data file settings' window, hit the 'Open...' button
2) Browse to where you saved your .csv file (you'll need to change the 'Files of Type' dropdown to CSV so it's visible) and hit 'Open'
3) In LiveGraph's 'Graph settings' window, change the X axis to data series using timestamp (like this).

That's all there is to it. :) You will need to leave my app logging for some time before you'll get any really interesting graphs of course.

Quote:

---

Originally Posted by Dr C

There are others, but they wouldn't be simpler than this. Here's exactly how to open it in LiveGraph:

1) In LiveGraph's 'Data file settings' window, hit the 'Open...' button
2) Browse to where you saved your .csv file (you'll need to change the 'Files of Type' dropdown to CSV so it's visible) and hit 'Open'
3) In LiveGraph's 'Graph settings' window, change the X axis to data series using timestamp (like this).

That's all there is to it. :) You will need to leave my app logging for some time before you'll get any really interesting graphs of course.

---

Oh, OK. I did all that and a few points were plotted but no lines, so I guess it only records data from when the level is added (makes sense really). Thanks, I'll be sure to use this in future :)

Quote:

---

Originally Posted by Dr C

Any thoughts/suggestions/problems, let me know. :)

---

A couple of suggestions, although you probably won't like them. :)

1. Instead of sourcing the data by repeatedly parsing the level stats on a timer, how about sourcing it by parsing the "Recent Activity" log for the level? That way you could generate stats from historical data.
2. Might be nice to include a graphing capacity into the program directly, rather than having to import the data into a third-party tool.

Thanks for creating such a useful little tool! It's things like this that really add to the LBP community.

Quote:

---

Originally Posted by Aya042

A couple of suggestions, although you probably won't like them. :)

---

I like all suggestions, whether they're doable or not. ;)

Quote:

---

Originally Posted by Aya042

Instead of sourcing the data by repeatedly parsing the level stats on a timer, how about sourcing it by parsing the "Recent Activity" log for the level? That way you could generate stats from historical data.

---

That's an interesting idea, but it's not really practical unfortunately. Aside from the fact that the recent activity timestamps are really vague ("About xx hours/days/months ago"), the recent activity log generally doesn't go all the way back to when the level was published, so you can't generate reliable totals from it.

Quote:

---

Originally Posted by Aya042

Might be nice to include a graphing capacity into the program directly, rather than having to import the data into a third-party tool.

---

I may add a button to open the graphs in LiveGraph directly from the app or something. But for now I've just Kept It Simple Stupid. :)

Quote:

---

Originally Posted by Dr C

Aside from the fact that the recent activity timestamps are really vague ("About xx hours/days/months ago"), the recent activity log generally doesn't go all the way back to when the level was published, so you can't generate reliable totals from it.

---

That's a shame. Ironically, all the historical data will be stored on their DB server - there's just no way to access it. :/

Quote:

---

Originally Posted by Dr C

I may add a button to open the graphs in LiveGraph directly from the app or something. But for now I've just Kept It Simple Stupid. :)

---

Another option might be to generate a URL compatible with the Google Chart API.

This really is awesome, but I'm confused by the data... mostly regarding the chart labels. Can this really be correct, your examples span two months of plays for a level that's been online for 4 years? (day 15191 to day 15243)? And then the Yay/Boo ratio takes some effort to work out as between 12 and 13 Yays for every 1 Boo, and I am too poor at math to make sense of the Heart/Plays ratio (about 1 in 3 players heart the level?)... if there's a clearer way to label these things it would be helpful. Also interesting would be having multiple paths for multiple levels overlaying for an instant visual of which level is doing "better" than the other. Although I suppose that could just be done with Photoshop or the like.

Quote:

---

Originally Posted by arbiekko

Can this really be correct, your examples span two months of plays for a level that's been online for 4 years? (day 15191 to day 15243)?

---

About 2 months of plays, yep. But the number of days indicates 41 years, not 4. That's not the time elapsed since the level was published (obviously), but the time elapsed since Jan 1 1970. It's all due to the app logging the timestamps in unix time and LiveGraph showing them as a number of days rather than a date. It's not ideal obviously. ;)

Quote:

---

Originally Posted by arbiekko

And then the Yay/Boo ratio takes some effort to work out as between 12 and 13 Yays for every 1 Boo

---

Right. :)

Quote:

---

Originally Posted by arbiekko

and I am too poor at math to make sense of the Heart/Plays ratio (about 1 in 3 players heart the level?)

---

Right again. :)

Quote:

---

Originally Posted by arbiekko

Also interesting would be having multiple paths for multiple levels overlaying for an instant visual of which level is doing "better" than the other.

---

That's not directly possible with this setup - though you can overlay various statistics of the same level on a single graph in LiveGraph.

I'll give Google Charts a look like Aya042 suggested, that might give some more flexibility for all this stuff. :)

Does it update the levels everytime I boot the program up? From what it says, it seems like it, but I can't tell. It's not really practical to leave my computer running 24/7 for the sake of LBP. The graph still isn't showing anything, but I'll give it a bit longer.

Quote:

---

Originally Posted by Dr C

That's an interesting idea, but it's not really practical unfortunately. Aside from the fact that the recent activity timestamps are really vague ("About xx hours/days/months ago"), the recent activity log generally doesn't go all the way back to when the level was published, so you can't generate reliable totals from it

---

Your software have that problem either way, what if you turn it on when level is published? :p you wont have data from past, it could be good supportive feature. "ago" time can be easily translated with help of string functions

Quote:

---

Originally Posted by kirbyman62

Does it update the levels everytime I boot the program up? From what it says, it seems like it, but I can't tell. It's not really practical to leave my computer running 24/7 for the sake of LBP. The graph still isn't showing anything, but I'll give it a bit longer.

---

It updates every x hours depending on how many hours you've set it to. If you start the program and it's been longer than that since an update then yes, it'll update immediately.

You don't need to leave your computer up 24/7. Even if the program only runs once every day or two that's plenty of data points for a decent graph. :)

If your graph isn't showing *anything*, have you got it set up right? Drop me a PM if you're not sure. :)

Quote:

---

Originally Posted by Shadowriver

Your software have that problem either way, what if you turn it on when level is published? :p you wont have data from past, it could be good supportive feature. "ago" time can be easily translated with help of string functions

---

I'm not sure what you mean here. By reading current *totals* from the lbp.me page, you know exactly how many plays and ratings you have at that moment in time. You can't glean those totals from the recent activity log if some activity just isn't available. And even if you could, generating timestamps that could be out by up to a month is really inaccurate and would produce crappy graphs.

OK, here are screenshots (using this level as an example) :)

[spoiler]
http://img839.imageshack.us/img839/2613/pic2mz.png

http://img411.imageshack.us/img411/5181/pic1so.png

http://img510.imageshack.us/img510/7459/pic4w.png

http://img443.imageshack.us/img443/3350/pic3pxt.png
My screen is too small ._.
[/spoiler]

EDIT: I fixed it! It turned out there were two files for the level, and the one I was loading up in LiveGraph wasn't the one that the Level Logger was updating.

OK, I've found a better graphing solution and it's now built right in to the application, all you have to do is hit the Graph button, select your settings:

http://i52.tinypic.com/2w74t2r.png

and voila:

http://i55.tinypic.com/be906.png

No more messing around with manually running LiveGraph. I've updated the original post to reflect this. :)

@Aya042: The Google Charts API is nice, except it's limited to 16kB of data, which isn't enough. So I went with CliChart, which works well. :)

Got the update and it's great. Thanks :)

Amazing efforts, just wish it were Mac based also. Not a complaint or a detrimental comment, as this type of data is really fascinating.

Personally I would like to see amount of people online when the data was captured.