It is currently Sat Nov 26, 2011 8:04 pm

All times are UTC + 1 hour




Post new topic Reply to topic  [ 36 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject:
PostPosted: Wed Oct 28, 2009 9:48 am 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 4:11 pm
Posts: 645
Location: Netherlands
, can you send me your stuff (in a patch) somewhere before the weekend? I hope to have time saturday or sunday to help you fix this problem. We should really make this fix and send it out for testing.

: Filtering on seeder/leecher ratio could be done, but it goes into the principles of a 'good' torrent, mostly you would want a torrent with a lot of seeders since that will result in a faster download. So I don't think that filter method would be robust.
I first want to see if filtering on torrent uploaded time (I have to investigate if that is the same as 'creation time', I thought it was) is good enough before we try any extra filter methods.

_________________
Roel

i watch: simpsons, south park, lost, top gear, true blood & fringe
download - documentation - bugs/feature requests - sourceforge -


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 28, 2009 11:01 am 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
roel wrote:
: Filtering on seeder/leecher ratio could be done, but it goes into the principles of a 'good' torrent, mostly you would want a torrent with a lot of seeders since that will result in a faster download. So I don't think that filter method would be robust.

Is that true? It is better to have a torrent with a ratio of 500/1000 than one with 1000/100000. The seeders only have a limited bandwidth and if they have to distribute this over more leechers the average speed for a leecher will decrease.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Oct 31, 2009 9:41 am 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
: Any progress on implementing this feature?


Top
 Profile  
 
 Post subject:
PostPosted: Tue Nov 03, 2009 10:20 pm 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 4:11 pm
Posts: 645
Location: Netherlands
Jofo wrote:
roel wrote:
: Filtering on seeder/leecher ratio could be done, but it goes into the principles of a 'good' torrent, mostly you would want a torrent with a lot of seeders since that will result in a faster download. So I don't think that filter method would be robust.

Is that true? It is better to have a torrent with a ratio of 500/1000 than one with 1000/100000. The seeders only have a limited bandwidth and if they have to distribute this over more leechers the average speed for a leecher will decrease.


That is what I meant. You want a torrent with a good ratio. A good ratio is a lot of seeders and a small amount of leechers. And as indicated by bluenote: the fake torrents have a lot of seeders and a small amount of leechers.

So, if you build a filter on ratio to block the fake torrents, you should filter out torrents with an excellent seeder/leecher ratio (a lot of seeders, a small amount of leechers). That is not what you want.

_________________
Roel

i watch: simpsons, south park, lost, top gear, true blood & fringe
download - documentation - bugs/feature requests - sourceforge -


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 04, 2009 11:10 am 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
roel wrote:
That is what I meant. You want a torrent with a good ratio. A good ratio is a lot of seeders and a small amount of leechers. And as indicated by bluenote: the fake torrents have a lot of seeders and a small amount of leechers.

Well, that's not what I meant :)In my opinion a good ratio is a lot of seeders and a lot of leechers. A leecher is (most of the time) someone which doesn't have a complete copy of the file yet, but it is able to upload the parts of the file it already has.
The number of leechers is therefor an indication of how many people think this file is worth downloading.

The seed/leech ratio should therefor be as small as possible with a threshold on the number of seeders and leechers of course.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 04, 2009 11:35 am 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 4:11 pm
Posts: 645
Location: Netherlands
Jofo wrote:
In my opinion a good ratio is a lot of seeders and a lot of leechers. A leecher is (most of the time) someone which doesn't have a complete copy of the file yet, but it is able to upload the parts of the file it already has.
The number of leechers is therefor an indication of how many people think this file is worth downloading.

The seed/leech ratio should therefor be as small as possible with a threshold on the number of seeders and leechers of course.


Ah I see what you mean now.
If I understand you correctly, to combat the fakes, you prefer torrents with a lot of seeders AND a lot of leechers. So then, the ratio would be like 1:1 (almost as much seeders as leechers).

My argument however is that a torrent with lot of seeders and less leechers is better, since that will give you a better download speed (see this table), since leechers also still use the seeders to download from.
That ratio would be something like 4:1 or so.

But this ratio is exactly what the fake files have right now.

I'm afraid that building a filter on seeder/leecher ratio will block torrents that are just normal, good torrents with a lot of speed.

_________________
Roel

i watch: simpsons, south park, lost, top gear, true blood & fringe
download - documentation - bugs/feature requests - sourceforge -


Top
 Profile  
 
 Post subject:
PostPosted: Wed Nov 04, 2009 12:22 pm 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
Maybe this article also helps :)


Top
 Profile  
 
 Post subject:
PostPosted: Sun Nov 08, 2009 1:53 am 
Offline

Joined: Fri Oct 09, 2009 8:11 pm
Posts: 14
First off, I'd like to note that leechers are only called so as they do not have the entire file yet. They can in most cases supply you with pieces you don't have. Thus, a file with lots of seeders and lots of leechers may actually give you a very good speed.

Anyway, I have been fiddling with an idea that I think would be great for TED, but does require integrating it more tightly with the torrent client.

Basically, by using a image lib together with a video lib you can get a few frames off a video that has finished downloading. (You'd have to watch the finished folder of the client to see when the file appears there.)
By having a folder of "banned" images you can do a RMS comparison between the "banned" images and the frames you grab from the start of the finished files. If the RMS value is under a certain treshold you can assume they match and re-download the torrent with a different filesize or other fingerprint after deleting or moving it to a fake-folder.

This method would work for most video torrent as long as you can get screenshots from them via the video/image lib.
It doesn't save you from redownloading but atleast it would allow TED to retry without user interaction. Thus, when you get home to find a new episode it is real...


Top
 Profile  
 
 Post subject:
PostPosted: Mon Nov 09, 2009 4:46 pm 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
Well, that is a creative solution :)

Unfortunately there is no communication from the torrent client back to ted where it has put the video file. And even if this was know if wouldn't be that easy I think.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Nov 09, 2009 7:18 pm 
Offline

Joined: Fri Oct 09, 2009 8:11 pm
Posts: 14
Actually. It would.
We'd need:
1. A text/folder type input telling ted where your "finished" torrents end up.
2. A way to integrate with a video library
3. A way to integrate with a image library

First is very simple, the last two are harder though. But there is Jffmpeg (http://jffmpeg.sourceforge.net/) that may do the trick.

By using a set of frames from randomly samples spots at the start of the file one should be able to easily confirm fakes.
Audio support is maby even harder, so good thing it is not neded.

(This would actually have a side benefit as the step of later adding player in TED becomes very small if one would ever want to.)


Top
 Profile  
 
 Post subject:
PostPosted: Sat Nov 28, 2009 2:44 am 
Offline

Joined: Tue Nov 20, 2007 10:36 pm
Posts: 41
any news on this? with the advent of the fall of mininova I think this will become more important


Top
 Profile  
 
 Post subject:
PostPosted: Sun Nov 29, 2009 1:25 am 
Offline

Joined: Fri Oct 09, 2009 8:11 pm
Posts: 14
Not really.

It's possible with java but requires a lot of additional installs on eny users machine.

Alternatively, why not make a platform dependant addon to TED?
Users who want the functionality would download the tool seperately and TED would call it with a filename as a parameter. If it exits cleanly, the file is ok, if not it's bad.

This would allow any number of third party addons to check files.
Would also make it easier as you can integrate with windows media under windows, and video4linux and such under linux...


Top
 Profile  
 
 Post subject:
PostPosted: Sun Nov 29, 2009 12:38 pm 
Offline
Lead Developer
User avatar

Joined: Mon Jan 16, 2006 5:46 pm
Posts: 904
Location: Netherlands
I made a development build incorporates a seeders/leechers ratio option for a torrent which should filter out the fakes. It is not finished yet (that's why it is a development build :)) but the current functionality should work.

Information about how to install the development build can be found on the wiki.


Top
 Profile  
 
 Post subject:
PostPosted: Sun Nov 29, 2009 2:29 pm 
Offline

Joined: Fri Oct 09, 2009 8:11 pm
Posts: 14
I just had a revelation!

FFMPEG
It can extract stills from any spot in the videos.
It is also avaliable on many platforms and as source.

The command:
ffmpeg -i <videofile> -r 1 -ss 00:00:00 -vframes 1 <imagefile>.png
will capture a frame from the <videofile> at time 00:00:00 (in hours, minutes, seconds) to the <imagefile>.

Now all I need is a way to do a RMS comparison between two png images via commandline.


Top
 Profile  
 
 Post subject: Eureka!!!
PostPosted: Sun Nov 29, 2009 3:41 pm 
Offline

Joined: Fri Oct 09, 2009 8:11 pm
Posts: 14
Another breakthrough!

Firstly, the fake files do not actually play for 20+ minutes.
They usually skip forward quite fast.

Asking ffmpeg to grab an image from around 1 minute into the video will result in 0 images captured as the file play-time ends prematurely.

Thus, if TED tries to extract a single image from a downloaded video at for example 1 min position (00:01:00) then it will only succeed if the files playing-time is longer.

Extracting images far into the files will take a long time so I suggest using an early point for now. When the fakers start adding real data to the beginnings we can randomize it.

If you still want to do comparisons it is easy!
ImageMagic has a compare and a scale command.
By scaling all images to a specific size (width actually) and comparing against a number of templates, it should be easy to find fakes.

I need for someone to post some url's to fakes here so I can download em and grab some template images. Different formats will be needed.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 36 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group