MovieXPlayer Solution

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

Hello All

I'm participating in the discussion to find a fix for the movieXplayer fiasco that is crippling my ted. See the discussion here:
https://ted.nu/forum/viewtopic.php?t=1989

I've got eclipse installed and downloaded the source. I'm poking around looking at the different classes and getting familiar with the structure.

My Question:
Can you point me to the class or classes that contain the torrent validation code? I'd like to test if my solution below is possible.

I proposed the following solution:
The user has the ability to set their local time zone in the preferences.
I'm making an assumption that the TV show list is coming from mytvrss or something like that. Within the RSS feed is: <pubDate>Wed, 14 Oct 2009 22:00:00 GMT</pubDate>

Since TV for the US is keyed off EST (GMT -5) We know this show airs at 18:00 (6pm) EST. Any torrent posted before this time is flat out rejected and maybe anything posted before 19:00 is also rejected. An unscientific polling of the torrents on eztv shows me they post on average two hours after the airing.

Pseudo code:
Get pubDate from rss feed
Get UserPrefTimeZone from Preferences
Get torrentCreateDateTime from torrent file

If torrentCreateDateTime < (pubDate + UserPrefTimeZone)
REJECT
Else
Perform regular tests
size
not compressed etc...
End If

roel · **Posted:** Sun Oct 18, 2009 2:26 pm

You want to check the TedParser, its a huge class that in fact does the RSS parsing and torrent validation.

You can use the air date information stored in the episode datastructures to find out when the show should have aired. I think you can also get the publish date of a torrent quite easily. The hardest part is to figure out what timezone the airdate is stored in. Beware: we also list shows that air in Great Brittain, so you should take care of that somehow. We might even want to store or retrieve the timezone of the show somewhere?

If you like, I can give you access to our bugtracker where you can take ownership of this task and discuss the implementation details over there.

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

Roel

Sure I will do that. Give me access to bug tracker and I will work on this bug. I will have many questions I'm sure..

Should I post my questions to this thread?

Regards
Russ

roel · **Posted:** Sun Oct 18, 2009 5:16 pm

Hmm good question

If you have implementation questions, like "where can I find ...", please post them here. As soon as you have questions about how it should work (requirements), please post them in the bugtracker.

Please PM me your email address and desired username for the bugtracker so I can add you!

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

Hey
I'm mucking about in the code and I can't find 2 items.

Where specifically is the torrent creation Date and Time?

Where do I find the user's time zone?

Russ

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

I'm adding my functions to TedParser.java I'm not done but this is what I am working on now:
Looking in ted.datastructures.StandardStructure
There is a function:

Code:

                            /**

                            *  The airdate of this structure, with time zone correction applied

                            *  AirDateUnknownException When the airdate is not known

                            */

                            public Date getAirDate() throws AirDateUnknownException

Over in TedParser.Java I'm writing a few functions to use getAirDate() to filter out torrents published before the show was aired by comparing the current date time to the air Date Time. but I'm not sure the right way to get it. I might need to add an import...

Code:

                            //RWW added to get air date time.

                            import ted.datastructures.StandardStructure;

In TedParser.Java I've got a few functions I'm working on. Perhaps my code should just be with the timer. What do you think?

Code:

                               /**

                                *  Returns the current system time as epoch time

                                */

                               private long getCurrentEPOCHDateTime (){

                                  long epoch = System.currentTimeMillis()/1000;

                                  return epoch;

                               }

                               /**

                                * Checks if torrent was created after air dateTime

                                *  torrent - torrentImpl of torrent to check

                                *  serie -TedSerie that torrent belongs to

                                *  Returns if the torrent is past airDateTime

                                */   

                               private boolean isPastAirTime(TorrentImpl torrent, TedSerie serie,   ){

                                  //local variables

                                  long currentEPOCHDateTime = 0;

                                  long airDateTime = 0;

                                  //get the current system Date Time in epoch

                                  currentEPOCHDateTime = getCurrentEPOCHDateTime();

                                  //get the air date time - should be epoch

                                  airDateTime = torrent.getAirDate();

                                  //compare the dates if the current date is less than the

                                  //air date time then the torrent was put up before the

                                  //show aired and is probably a fake.

                                  if (currentEPOCHDateTime <  airDateTime   ){

                                     return false;

                                  }else{

                                                   return true;

                                           }

                               }

I would also like to use this same code for the timer. When the timer goes to check torrent feeds for new episodes use this same function to see if the particular torrent was published before the air date time.

Your comments feedback and suggestions appreciated!
Russ

roel · **Posted:** Sat Oct 24, 2009 4:25 pm

I would imagine that you add your check on the creation date of the torrent to the

Code:

private void ParseItem(SyndEntry item, TedSerie serie, String source)

method. There you have a TorrentImpl instance available (after we check the seeders from the feed entry. This torrentimpl instance holds the torrent creation date.

You have to check if torrentimpl.creationDate() is > airdate.

But, we have to be carefull with timezones here. I don't know what is the timezone of the torrentimpl creation date. The airdate is in the users timezone, but you might want to use the original airdate, without timezone correction. Or apply timezone correction to the creation date of the torrent.

The check you do against the current users time is not neccesary. That is something that is already done in the scheduler (since that is where shows are put on 'check' and 'hold'. See checkAirDate() in SeasonEpisodeScheduler.
We already check the airdate against the users local time. But: the airdate time zone correction is sloppy. We only apply correction of a day if the timezone is right of the median. We should do the correction more detailed (so just add the timezone difference that is the real timezone difference).

: why did we implement this in this way? Was that because we were not sure about the show timezones?

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

Ok I did some double checking.

According to the Bittorrent Protocol Specification v1.0 as found here:
http://wiki.theory.org/BitTorrentSpecification

Quote:

creation date: (optional) the creation time of the torrent, in standard UNIX epoch format (integer, seconds since 1-Jan-1970 00:00:00 UTC)

I think that means the torrent create date is in UTC or GMT - 0 so that they are universal. I'm not sure if that is always true but so far everything points to this being true. I'm trying to verify this fact now.

I looked at a torrent details on mininova and looked at the details:
Added on: Sat, 24 Oct 2009 04:37:09 +0200
The details include the UTC adjustment

So my thinking is to always set every date for comparison to UTC (GMT-0).

Do you agree?

What do you think is the best way to handle this?

Regards
Russ

Jofo · **Posted:** Sat Oct 24, 2009 8:20 pm

roel wrote:

: why did we implement this in this way? Was that because we were not sure about the show timezones?

No, we did it because it was good enough at the time. No real reason to keep it the way it is right now.

Jofo · **Posted:** Sat Oct 24, 2009 8:25 pm

WebRuss wrote:

So my thinking is to always set every date for comparison to UTC (GMT-0).

That would make life simpler

Quote:

What do you think is the best way to handle this?

To handle what exactly? IÂ´m going away in a few minutes so I donÂ´t really have the time to think about it. Will have a look at it tomorrow.

WebRuss · **Joined:** Sun Sep 27, 2009 3:16 pm **Posts:** 12

Question on the TorrentImpl torrent

//get the air date time
airDateTime = torrent.getAirDate();

//create date time
createDateTime = torrent.creationDate();

Are these dates in EPOC format? I'm not sure.

Regards
Russ

rwe · **Joined:** Tue Oct 13, 2009 8:46 pm **Posts:** 9

If you do implement this solution, please implement a way to turn it off.

As I pointed out in another thread, there are times when shows are posted well before their scheduled times.

For one the schedules aren't always right.

But mainly shows sometime show the day before in Canada

I have even found shows a week before they were to air, and they were good.

Let alone when shows will change days or schedules because of network reasons

I think the real solution is to identify where the fakes are coming from and block those sites.

By deleting btjunkie, and moving isohunt last, I have eliminated all the fakes as of now. My currnet filters have the file sizes set to zero and the seeds set to zero, and it is finding all my show with no problems.

eztv.it seems to have the least amount of fakes, I can't remember the last time I got one from them.

At best this solution will make the fake poster wait to post the show at the correct time, it's still not going to stop him from posting fake shows.

Maybe we can find out why eztv has so few fakes and let btjunkie know how they do it. LOL

jaggles · **Joined:** Mon Sep 28, 2009 11:10 pm **Posts:** 26

Same here; after removing btjunkie and isohunt completely, I have not received any fakes at all. (So far...)

Jofo · **Posted:** Mon Oct 26, 2009 12:10 pm

WebRuss wrote:

Are these dates in EPOC format? I'm not sure.

I would not know, the TorrentImpl is an external library. There is documentation for that library but it doesn't say anything about the date format.

bluenote · **Joined:** Tue Nov 20, 2007 10:36 pm **Posts:** 41

Please forgive an intrusion by somebody who isn't familiar with java and hasn't even looked at any of this code. (I hate it when users say, "I'm not an expert but it seems to me ...").

So. "I'm not an expert but..."

Webruss: I note that it sounds to me like the creation date/time of the torrent file is coming from the torrent file itself. And considering that we are talking about fakers, whats to stop them from arbitrarily setting that date/time to a "proper" date time. Having said that, this might be a "good enough" solution. I note that your quote also say it is optional though.

In my mind, a more reliable (and, sigh, more difficult for you) stat would be "uploaded time" or something similar. Which would correspond to, when that torrent was made available to the index site. (or, perhaps, first seen by ted! ... For example, if you know by airtime info that a show cannot possibly be posted, but its showing up, then that hash can be 'black listed'. This actually strikes me as a pretty good and easy test. And it fits really nicely into how ted operate right now which is airTIME-blind as far as checking. (ie: starts checking on 12:01 of the airdate, depending on your timer config).

In response (sorry, I cant remember) users suggesting that the solution is to reduce the # of sources ... in my estimation thats a solution that ends up needing constant tweaking and attention. If you block by tracker, they'll find another tracker, and if you block by index site, well, its a good bet that soon the one or two remaining index sites will soon get fakes too. ...

I have a couple comments on the general method though --
1) It *appears* to me that the fakers may be utilizing two methods to "get the jump" on the real torrents. This would be:
a) posting early so as to gather up all that uncontended-for traffic
b) possibly using trackers that are compromised/vulnerable to seeder faking, so as to get a higher seeder count and therefore rise higher in the rankings more easily.

I know that some of these shows are posted before they could possibly be done in reality. I also know they have really high seeder to leecher ratios (on the order of like 6000:200) which appear to ME to be faked. But thats just my guess, and I have no data to base it on.

If we somehow eliminate both of those issues, it would put regular torrents on a proper footing with the fake torrents, and then, "the system" should be able to take care of them more quickly, naturally and effectively, just like it currently does.

I'd also like to put in a vote for a very simple "check X hours after air time" or "X days after air date" because it seems to me that it doesn't take very log for people to drop off the fakes and the real ones to become more popular, pushing themselves to the top of the list(s).

Anyways, I hope all my blabbering is worth something ...

I love this software and I hope it can continue to beat out the fakes that are trying to ruin this great thing we have going.

thanks,

Cory

MovieXPlayer Solution

Who is online