When the Same Video Arrives Twice, Which Copy Should You Keep?

If you archive more than a handful of creators, you already have duplicates. The same clip is posted to TikTok, reposted by a fan account, mirrored to a forum thread, and cross-posted to Reddit. Every archiver has to decide what to do about that, and almost all of them make the same quiet mistake, keeping whichever copy happened to arrive first. This post is about why that is the wrong rule and what a better one looks like.

Why you end up with duplicates

Duplicates are not a sign you have set something up wrong. They are the natural result of archiving more than one account.

A creator posts a clip to TikTok. They cross-post it to Instagram Reels a day later. A fan account reposts it and picks up more views than the original. Somebody mirrors it into a forum thread, and somebody else cross-posts it to a relevant subreddit. That is one piece of content and five plausible routes into your library.

If you follow the creator on two platforms, or you follow both the creator and a community that reposts them, you will get the same file more than once. On a library of any size this is not marginal. It is one of the main reasons archives grow faster than the amount of actual content in them.

The problem with "first one wins"

Every deduplicating tool needs a rule for which copy to keep. The overwhelmingly common one is: whichever we saw first stays, later matches are skipped.

It is easy to implement and it feels neutral. It is not neutral at all.

Arrival order is uncorrelated with quality. Which copy reaches you first depends on which feed you happened to add first, how often each is checked, and when each account posted. None of those things has any relationship to which file is better.

And the copies genuinely differ, usually in the same direction:

• Reposts get cut short. A reposter trims the intro, or clips the funny twenty seconds out of a two-minute video. The result is not a worse-looking copy of the same thing, it is a copy with content missing.
• Reposts lose audio. Some are re-uploaded silent, deliberately or through a bad export.
• Reposts get re-encoded. Downloading and re-uploading through a third-party tool almost always costs resolution and bitrate.

So the systematic effect of first-one-wins is that a fan account's cropped, quieter, more compressed copy can permanently displace the creator's own upload, purely because you added the fan account first. The better file arrives, gets recognised as a duplicate, and is discarded without ever being looked at.

Worse, it is invisible. Nothing errors. Your library looks clean and deduplicated. You simply have the wrong copy, and no way of knowing.

A better rule, in order

StreamStash compares the two copies and keeps the better one. The order the comparison runs in matters more than the individual checks, so it is worth being explicit about it.

1. Length. A copy that is missing content never wins, however good it looks. If one file runs two minutes and the other runs twenty seconds, the twenty-second one is a clip of the other, not a competing version of it. Putting length first is what stops a sharp, well-encoded fragment displacing the whole video.

2. Audio. If one copy has an audio track and the other does not, the one with sound wins. Trading the audio away for a slightly sharper picture is not a trade anyone would make deliberately.

3. Resolution. Only once both copies are the same length and both have sound does picture quality decide.

4. File size. The tie-breaker when two copies match on everything above, since at equal length and resolution the larger file is generally the less compressed one.

The practical effect is that a creator's own upload usually beats someone else's repost of it, which is the outcome you would have chosen by hand every time. Your library quietly improves as duplicates arrive, instead of ossifying around whatever landed first.

It happens inside one account too

Cross-account duplicates are the obvious case, but a single creator generates plenty on their own:

• A story that later gets posted properly to the grid
• A story and the highlight it was added to
• The same attachment posted twice in a long forum thread
• A clip reposted to the creator's own feed months later

These get the same treatment, on every platform. Previously that comparison only ran on forum threads and Snapchat, and only on photos; it now applies to everything StreamStash saves and to video as well as images.

Nothing is deleted outright

Any automatic system that picks between two files will occasionally pick wrong, and a rule that silently deletes the loser is asking to be trusted more than it deserves.

So the displaced copy is not deleted. It goes into a Replaced Copies list in Settings, which shows what is being held, how much space it is using, and how confident the match was. Every entry has a Restore button that returns the file and its details to the feed it came from, and there is a Restore All to undo a whole batch.

Each row also records why the other copy won, captured at the moment of the swap: longer, has an audio track, higher resolution, or simply the larger file. That one line does most of the work, because it means most rows can be approved at a glance and only the genuinely ambiguous ones are worth opening.

Actually seeing the difference

For the ambiguous ones, side-by-side thumbnails are close to useless. Two copies of the same clip at different bitrates look identical at thumbnail size, and if they are not showing the same frame you are not comparing anything meaningful.

So every held copy has a Compare button that opens both versions in one frame with a slider you drag across to reveal one over the other. Same moment, same size, same place on screen, which is the only arrangement where a quality difference is actually visible rather than inferred from numbers.

Videos play together in step on one scrubber, and you choose which one you hear, so a copy that lost its audio becomes obvious immediately rather than being something you have to go looking for. Photos work the same way. Where the two are shaped differently, or one of them will not play, it falls back to showing them side by side and says why rather than pretending the slider is meaningful.

Restore and Delete sit in that same window, and the arrow keys move through the rest of that creator's held copies, so reviewing a batch does not mean opening and closing a dialog thirty times.

You choose how long copies are held before they are cleared: one day, three, seven (the default), fourteen, thirty, or not at all. Choosing not to keep them frees the space the moment the better copy lands but leaves no way to undo a wrong match, and the setting says so before you save it.

One detail worth knowing: for posts with several photos, only the specific image that matched is replaced. The rest of the post is left exactly as it is.

Reviewing by hand

Automatic replacement handles duplicates as they arrive. For a library that already has years of them, there are two review tools, and they behave differently on purpose.

Dedupe This Feed works on one creator and shows matches side by side. Each pair shows how long each copy runs, marks one that has no sound, and puts a short line on the kept copy saying why it won, whether that was length, audio, resolution or size. A choice that turned on length rather than resolution should not look arbitrary.

Dedupe Now sweeps the whole library. It scans first and reports what it found before deleting anything: how many files would go, how much space that frees, and which creators they sit under, sorted by space reclaimed. Nothing is removed until you press the button on that summary. It deliberately does not offer pair-by-pair review, because a library-wide list runs to thousands of rows and nobody reads it; for individual matches, use Dedupe This Feed.

Deletions from those review tools are permanent and are not held in Replaced Copies, which is a real difference from automatic replacement. Both tools say so before you confirm.

When to leave it switched off

Deduplication is a Power feature and follows a switch in Settings, and there are legitimate reasons to leave it off.

If you are archiving for provenance rather than for viewing, the duplicates are the data. Which account posted what, when, and in what state is exactly the thing you are recording, and collapsing five copies into the best one destroys that. Researchers, moderators and anyone building a case for how something spread should keep every copy.

For everyone else, whose library exists to be watched and searched, duplicates are noise and storage cost. There is also a middle setting: leave deduplication on but turn off Replace lower-quality copies, and StreamStash goes back to keeping whichever arrived first while still recognising duplicates. Anything already held stays restorable either way.

If your library is growing faster than you expected, duplicates are usually part of the reason, and planning storage for a social media archive covers the rest of it.

FAQ

Why do I get duplicates in a media archive?

Because creators cross-post and other people repost. The same clip can reach you from the creator's own TikTok, a fan account that reposted it, a forum thread that mirrored it, and a Reddit cross-post. If you follow more than one of those, you get more than one copy of the same content.

What is wrong with keeping whichever copy arrived first?

Arrival order tells you nothing about quality. A repost frequently arrives before you get round to adding the original creator, and reposts are often shorter, quieter or more heavily compressed than the source. Keeping the first arrival means a better version can turn up later and be discarded unopened.

How should two copies of the same video be compared?

Length first, because a repost is often cut short and a copy missing content is never worth keeping over the full-length one however sharp it looks. Then whether audio is present, since some reposts arrive silent and losing the sound is not a fair trade for a slightly better picture. Then resolution, and finally file size to settle two copies that match on everything else.

Is a deduplicated file gone for good?

Not immediately. In StreamStash a replaced copy is held in a Replaced Copies list showing how confident the match was, how much space is being held, and why the other copy won. Every entry has a Restore button, and you choose how long copies are kept before they are cleared, from one day up to thirty, or not at all.

How can I tell which copy is actually better?

Each held copy has a Compare button that opens both versions in one frame with a slider you drag across to reveal one over the other, at the same moment and the same size. Videos play together on a single scrubber and you choose which one you hear, so a copy missing its audio is immediately obvious. Where the two are shaped differently or one will not play, it shows them side by side and explains why. Restore and Delete are in the same window, and arrow keys move through the rest of that creator's held copies.

Does deduplication work across different platforms?

Yes. Matching is on the content itself rather than the filename or URL, so the same clip is recognised whether it came from TikTok, Instagram, Reddit, a forum thread or elsewhere. It also applies to duplicates within a single account, such as a story that later becomes a post.

Can I turn deduplication off?

Yes. It is a Power feature and follows a switch in Settings. There is a second switch for replacing lower-quality copies specifically, so you can keep deduplication on while reverting to first-arrival-wins if you prefer. Anything already held stays restorable either way.