Video scan time vs. Video+Audio scan time

The best solution for finding and removing duplicate files.
User avatar
therube
Posts: 646
Joined: Tue Jun 28, 2011 4:38 pm

Video scan time vs. Video+Audio scan time

Post by therube »

Video scan time vs. Video+Audio scan time.

Video mode.
Hash - Video -vs.- Hash - Video+Audio / Music


Why is Video only so much faster compared to Video+Audio?
We're talking seconds vs. minutes.

(Depending on file sizes... & granted that you're hashing two streams vs. 1... But still V+A takes significantly longer then the V alone.)


i assume you're using ffmpeg

& using a, -f streamhash -hash md5 -map 0:v, for the Video alone?

not sure what you're doing for the V+A?
maybe you're doing a framehash, & then generating a single hash'd output value from that?
-f framehash -hash md5... & then ... ???

but, if so, is there any reason for that, as opposed to using streamhash for both the V & A
(& then generating a hash on that, if you're looking for only a single output result)

streamhash on V+A takes only slightly longer then streamhash on V alone, & both are far quicker then framehash.


(& granted, I don't really understand such things, but I can see the time differences involved.)
User avatar
DigitalVolcano
Site Admin
Posts: 1921
Joined: Thu Jun 09, 2011 10:04 am

Re: Video scan time vs. Video+Audio scan time

Post by DigitalVolcano »

I've checked and v+a is using:
-map 0 -f md5

Which looks to be way to much work - it's re-encoding everything. I'll see if I can update it to a faster method (ie hashing the two streamhashes together).
Version 5.27 is ready to go but I'll see if I can sneak this in:)
User avatar
therube
Posts: 646
Joined: Tue Jun 28, 2011 4:38 pm

Re: Video scan time vs. Video+Audio scan time

Post by therube »

it's re-encoding everything
Then you must have left off the, -c copy, no?

Code: Select all

timethis ffmpeg -i %x% -c copy -map 0 -f md5 -
User avatar
therube
Posts: 646
Joined: Tue Jun 28, 2011 4:38 pm

Re: Video scan time vs. Video+Audio scan time

Post by therube »

So the outputs (the resultant hash) are different, depending on whether you use '-c copy' or not.
But are both methods just as valid? (I'll suppose they are.)

Code: Select all

MD5=38db798448c6ab3040735a22a8994e68
[out#0/md5 @ 00000000004d7300] video:2916675KiB audio:31010KiB subtitle:0KiB oth
er streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 8642 fps=1791 q=-0.0 Lsize=       0KiB time=00:06:00.02 bitrate=   0.0kbi
ts/s speed=74.6x elapsed=0:00:04.82

TimeThis :  Command Line :  ffmpeg -i xxx.mp4 -map 0 -f md5 -
TimeThis :    Start Time :  Thu May 07 12:30:37 2026
TimeThis :      End Time :  Thu May 07 12:30:42 2026
TimeThis :  Elapsed Time :  00:00:04.940

Code: Select all

MD5=7b10e8330c156e685a47265dd6163237
[out#0/md5 @ 0000000002753600] video:21754KiB audio:2840KiB subtitle:0KiB other
streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 8642 fps=0.0 q=-1.0 Lsize=       0KiB time=00:06:00.00 bitrate=   0.0kbit
s/s speed=3.8e+03x elapsed=0:00:00.09

TimeThis :  Command Line :  ffmpeg -i xxx.mp4 -map 0 -f md5 -c copy -
TimeThis :    Start Time :  Thu May 07 12:30:53 2026
TimeThis :      End Time :  Thu May 07 12:30:53 2026
TimeThis :  Elapsed Time :  00:00:00.210
Post Reply