Forum Settings
       
Reply To Thread

ZAM Rift Log Parser!!!Follow

#1 Jun 21 2011 at 5:09 PM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
http://rift.zam.com/en/logparser.html
Patch 1.3 is nigh upon us and ZAM Rift is ready! We are pleased to announce we'll be taking advantage of the new discovery logs with our log parser!

Help ZAM with NPC locations, loot tables, and drop rates by uploading your Discovery.log file. To assist even more, create and upload your Log.txt file which will help us with gathering, artifact nodes, quest items, and more! Follow the simple instructions to painlessly upload your logs to ZAM Rift, where they will be parsed and the information instantly added to our databases.

Edited, Jun 21st 2011 9:45pm by Calthine
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#2 Jun 21 2011 at 8:12 PM Rating: Excellent
Avatar
**
373 posts
Calthine wrote:
http://rift.zam.com/en/logparser.html
Patch 1.3 is nigh upon us and ZAM Rift is ready! We are pleased to announce we'll be taking advantage of the new discovery logs with our log parser!

Help ZAM with NPC locations, loot tables, and drop rates by uploading your Discovery.log file. To assist even more, create and upload your Log.txt file which will help us with gathering, artifact nodes, quest items, and more! Follow the simple instructions to painlessly upload your logs to ZAM Rift, where they will be parsed and the information instantly added to our databases.

Edited, Jun 21st 2011 9:45pm by Calthine


I think Cal's excited. :)
#3 Jun 21 2011 at 8:48 PM Rating: Decent
**
526 posts
Why does the link goto save a page that is 0Kb bytes big? It is not working. PLEASE FIXXX DO want to help!
#4 Jun 21 2011 at 8:59 PM Rating: Excellent
Avatar
**
373 posts
Try now, it should be fixed. :)
#5 Jun 21 2011 at 9:02 PM Rating: Decent
**
526 posts
Micajah wrote:
Try now, it should be fixed. :)


It is working now. That is a lot to do. Hopefully Trion makes this process more simple or we get an add on to make this automatic for you guys
#6 Jun 21 2011 at 9:21 PM Rating: Excellent
Empress of News
Avatar
***
2,350 posts
Micajah wrote:
Calthine wrote:
http://rift.zam.com/en/logparser.html
Patch 1.3 is nigh upon us and ZAM Rift is ready! We are pleased to announce we'll be taking advantage of the new discovery logs with our log parser!

Help ZAM with NPC locations, loot tables, and drop rates by uploading your Discovery.log file. To assist even more, create and upload your Log.txt file which will help us with gathering, artifact nodes, quest items, and more! Follow the simple instructions to painlessly upload your logs to ZAM Rift, where they will be parsed and the information instantly added to our databases.

Edited, Jun 21st 2011 9:45pm by Calthine


I think Cal's excited. :)

!!!!!!!!!!!!!!!!!!!!

Just lending her some extra exclamation points. Smiley: grin
____________________________
Cyliena Velnarin | @Cyliena on Twitter

#7 Jun 21 2011 at 10:46 PM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
Micajah wrote:
I think Cal's excited. :)


Who, me!? Be excited at automated data collection?!? ME?!?!?

Nah!!!!


Edited, Jun 22nd 2011 12:46am by Calthine
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#8 Jun 22 2011 at 10:14 AM Rating: Decent
**
526 posts
* /discoverylog toggles the new log type on and off and persists through logouts. Creates a file named discovery.log that is appended to for each play session.

http://forums.riftgame.com/showthread.php?212051-RIFT-1.3-NA-8-00am-PDT-6-22-11-EU-TBA

Calt go update the instructions how to use. The new notes state you do not have to type that every time now
#9 Jun 22 2011 at 10:41 AM Rating: Excellent
Empress of News
Avatar
***
2,350 posts
Thanks, we're on it. Smiley: nod
____________________________
Cyliena Velnarin | @Cyliena on Twitter

#10 Jun 22 2011 at 10:58 AM Rating: Decent
**
526 posts
Cyliena wrote:
Thanks, we're on it. Smiley: nod


That is a pretty sweet deal for Zam finally. No more data mining or bending over backwards to get up to date info. As long as the player base comes here and uploads the file, then you guys have severely current database on basically everything in the game.
#11 Jun 22 2011 at 1:48 PM Rating: Good
Database Master
*
55 posts
umm yay.






I mean
[img=200051]
YEAH!
#12 Jun 22 2011 at 6:45 PM Rating: Decent
**
526 posts
I keep getting this message:

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, webmaster@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.
#13 Jun 22 2011 at 7:20 PM Rating: Good
Database Master
*
55 posts
OK 
 
The server encountered an internal error or misconfiguration and was unable to complete your request. 
 
Please contact the server administrator, webmaster@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error. 
 
More information about this error may be available in the server error log. 
 
Apache/2.2.9 (Debian) mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0 Server at app4 Port 80


:/
#14 Jun 22 2011 at 8:10 PM Rating: Decent
13 posts
I'm getting the server error when I try to upload as well. I'm on Win 7 home premium, 64 bit.

Edit: Tried it again the morning of 23 JUN and it worked! Can't wait for this database to start rolling.

Edited, Jun 23rd 2011 7:50am by Khrush
#15 Jun 23 2011 at 3:21 AM Rating: Good
Database Master
*
55 posts
tried it again right now after tonight's raid and it worked! so lol I dunno what was happening before but yay!
#16 Jun 23 2011 at 5:31 AM Rating: Decent
5 posts
will these data be applied with these weeks or do we have to wait till you guys fully develop the system to analyze it?
#17 Jun 23 2011 at 10:20 AM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
warhowl wrote:
will these data be applied with these weeks or do we have to wait till you guys fully develop the system to analyze it?


It's instantly parsed and applied.
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#18 Jun 23 2011 at 10:38 AM Rating: Decent
**
526 posts
Calthine wrote:
warhowl wrote:
will these data be applied with these weeks or do we have to wait till you guys fully develop the system to analyze it?


It's instantly parsed and applied.


That is pretty sick. I like that a lot. Will definitely upload my discovery log after work today
#19 Jun 23 2011 at 11:19 AM Rating: Decent
5 posts
indeed very nice if it's applied instantly
uploading mine now and will reccoment to friends aswell
#20 Jun 23 2011 at 11:36 AM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
warhowl wrote:
indeed very nice if it's applied instantly
uploading mine now and will reccoment to friends aswell


Awesome, thanks! Yeah, we'll start seeing things like this:

http://rift.zam.com/en/item/A9B5E1B50501010101/Yew-Timber

____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#21 Jun 23 2011 at 11:57 AM Rating: Good
**
526 posts
Calthine wrote:
warhowl wrote:
indeed very nice if it's applied instantly
uploading mine now and will reccoment to friends aswell


Awesome, thanks! Yeah, we'll start seeing things like this:

http://rift.zam.com/en/item/A9B5E1B50501010101/Yew-Timber



I just ******* with excitement. That is just ****. Good job Zam!
#22 Jun 23 2011 at 1:32 PM Rating: Good
***
3,702 posts
Calthine wrote:
warhowl wrote:
indeed very nice if it's applied instantly
uploading mine now and will reccoment to friends aswell


Awesome, thanks! Yeah, we'll start seeing things like this:

http://rift.zam.com/en/item/A9B5E1B50501010101/Yew-Timber



Ok that's awesome.

Just need Telara Street View now with points of interest Smiley: nod
____________________________
svlyons wrote:
If random outcomes aren't acceptable to you, then don't play with random people.
#23 Jun 23 2011 at 3:22 PM Rating: Good
Database Master
*
55 posts
zomg. so cool. mapping ftw.
#24 Jun 23 2011 at 6:09 PM Rating: Good
**
526 posts
I just uploaded a crap ton of data. This thing is very very cool
#25 Jun 24 2011 at 10:48 AM Rating: Decent
13 posts
Should we be deleting our Discovery.log file from our own drives after we've uploaded? The reason I ask is because mine seems to be appending rather than overwriting. I played for a couple of hours and uploaded and it said it recorded 800+ npc locations. I went back and played for a bit longer, nowhere near as long as before, and when I uploaded after that it said 1300 npc locations. Now for locations that isn't a big deal, but if it is recording the same drops every time then the numbers are going to be skewed.
#26 Jun 24 2011 at 12:59 PM Rating: Excellent
Avatar
**
373 posts
Khrush wrote:
Should we be deleting our Discovery.log file from our own drives after we've uploaded? The reason I ask is because mine seems to be appending rather than overwriting. I played for a couple of hours and uploaded and it said it recorded 800+ npc locations. I went back and played for a bit longer, nowhere near as long as before, and when I uploaded after that it said 1300 npc locations. Now for locations that isn't a big deal, but if it is recording the same drops every time then the numbers are going to be skewed.


Getting the 100% accurate answer from our devs right now.
#27 Jun 25 2011 at 5:02 PM Rating: Decent
**
526 posts
The way this database is updating is just ***. Especially in leveling up professions and finding artifacts. Thanks for supporting this new log.
#28 Jun 26 2011 at 10:26 PM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
Puremallace wrote:
The way this database is updating is just ***. Especially in leveling up professions and finding artifacts. Thanks for supporting this new log.


Please tell your friends :)
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#29 Jun 27 2011 at 9:42 AM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
Some stats. Since launched, y'all have uploaded the following:

* 7,811 individual mobs have been logged in > 487,821 in different locations (that's at least one location for 78% of the mobs in our database)
* 9,085 mob-to-item relationships have been captured for items other than cash
* 2,947 mobs have cash drops (or lack thereof) recorded

____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#30 Jun 27 2011 at 1:41 PM Rating: Good
Database Master
*
55 posts
this is just great, I am gunna work on fixing some of the purchase stats on some of the vendor stuff later on tonight, but it is great to see that the discoverylog parser is working like a charm. Great Job!!!
#31 Jun 28 2011 at 10:43 AM Rating: Decent
5 posts
how about recipies and crafted items? are they also in the discovery log? becouse most items crated still got no stats.
#32 Jun 28 2011 at 5:15 PM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
warhowl wrote:
how about recipies and crafted items? are they also in the discovery log? becouse most items crated still got no stats.


Nope, there's no item data in the logs, that's from Trion. What you see is what we get from them.
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#33 Jun 29 2011 at 7:25 AM Rating: Decent
**
526 posts
Is there any reason when I upload my log it is no longer showing me that screen of how much I uploaded, but just shows a blank screen?

#34 Jun 29 2011 at 7:30 AM Rating: Excellent
Avatar
**
373 posts
Puremallace wrote:
Is there any reason when I upload my log it is no longer showing me that screen of how much I uploaded, but just shows a blank screen?



We'll look into it Pure.
#35 Jun 29 2011 at 10:21 AM Rating: Decent
**
526 posts
Micajah wrote:
Puremallace wrote:
Is there any reason when I upload my log it is no longer showing me that screen of how much I uploaded, but just shows a blank screen?



We'll look into it Pure.


I'll take a screenshot after I get off work. It basically looks like the regular screen, but where the info was appearing it is just blank now so I am not sure if the data even uploaded
#36 Jun 29 2011 at 12:53 PM Rating: Good
Database Master
*
55 posts
I have also had the upload page seemingly timeout on me before like he was saying, but only when doing especially big logs, sometimes my log file can get over 1mb and it takes a decent amount of time to parse it, and I think the browser just times out when it happens. I have not had this problem in the last 2 days though, only when you guys first started it.

update
I just did a huge amount of the new invasion rifts / other things and my discovery log has gotten very large, 1.4mb and I am not able to get it to parse, I wait and the page just goes black. I have tried it in both Chrome & IE and no luck. I am going to have to find a way to split these logs lol.


Edited, Jun 29th 2011 5:40pm by pquegg
#37 Jul 13 2011 at 11:22 AM Rating: Decent
5 posts
think i got same timeout problem, my discovery file atm is 9.5 MB and i haven't uploaded data in a week or 2 and now i also get a timeout screen

what is the max size we can upload?
#38 Jul 14 2011 at 9:36 AM Rating: Excellent
Special Snowflake
Avatar
****
6,786 posts
Has anyone tried splitting those huge log files up and then parsing them?
____________________________
[img]http://i12.photobucket.com/albums/a230/calthine/EmptyMind-2.jpg[/img]
Community Manager | QA Lead
ZAM: Support FAQ | Forum FAQ | Forum Rules
Cook Ten Rats
#39 Jul 14 2011 at 3:53 PM Rating: Excellent
$title{308642}
*****
0X317B posts
Calthine wrote:
Has anyone tried splitting those huge log files up and then parsing them?


It's doable, but you'd have to make sure not to split the tags wrong or the page will likely error out due to invalid XML. You'd have to copy your Discovery.log to something else, then paste segments back into the Discovery.log file. The good news, however, is that even if it doesn't tell you it succeeded (or the page doesn't load,) it's probably still parsing all the results in the background.

In other news, I did some reading and found a different toolset to use that helped speed things up quite a lot. I shaved 140 seconds off of a 7.4x MB log file through various methods. One more thing to try to see if I can speed it up more. BTW, ~9.53ish MB is the biggest file the page will accept; that was where it timed out every single times. I still recommend clearing your Discovery.log file after every upload, though, to avoid the issue entirely. Simply open it when Rift isn't running (or /Discoverylog to turn it off first and back on after,) open in notepad, select all, delete, save.
____________________________
Since 1 March 2004

[riftsig]308642[/riftsig]
#40 Jul 18 2011 at 3:28 PM Rating: Good
Database Master
*
55 posts
I guess since the changes you made in the last week or so, I have not had any problems what so ever uploading my logs now. They have been ranging from 1 to 3 mb in size. I also did try to split one that was really big the other week and ran into errors.

Edited, Jul 18th 2011 2:31pm by pquegg
Reply To Thread

Colors Smileys Quote OriginalQuote Checked Help

 

Recent Visitors: 18 All times are in CDT
Anonymous Guests (18)