Liverpool's Fight Against Duplicate Images in the Digital Archive: How It Compares Globally

Liverpool City Council's digital heritage team quietly flagged a significant backlog problem this spring: thousands of duplicate images sitting inside the city's publicly accessible archive portals, inflating record counts and making genuine historical material harder to find. The issue, acknowledged in internal communications reviewed by The Daily Liverpool, affects collections held across multiple platforms, including resources linked to the Liverpool Record Office on Islington and the digital catalogues maintained by National Museums Liverpool at the William Brown Street campus.

The timing matters. Across the UK and Europe, local authorities and cultural institutions are under growing pressure to meet open-data standards set by bodies including the European Commission's Europeana network, which coordinates digital heritage access for more than 3,000 institutions continent-wide. Europeana published updated deduplication guidance in late 2025, pushing member repositories to audit and resolve image duplication before the end of 2026. Liverpool's collections feed into that wider network through partnerships with national aggregators, meaning poor data hygiene here has a ripple effect far beyond Merseyside.

What Liverpool Is Actually Doing

The city's response so far has been cautious and, critics argue, underfunded. National Museums Liverpool launched a metadata review programme in January 2026 aimed at its digitised photograph collections, which span more than 400,000 individual image records. The project, running out of the Conservation Centre on Whitechapel, uses a combination of perceptual hashing software and manual curatorial review to identify near-identical files that accumulated over years of platform migrations and bulk digitisation drives.

The Liverpool Record Office, housed since 2013 in the Central Library on William Brown Street, is running a parallel exercise focused on civic photography — council planning images, public event records, and local press archive material transferred from former newspaper collections. Staff there have identified duplication rates of roughly 12 to 18 percent in some photography sub-collections, according to a project outline circulated to heritage sector contacts earlier this year, though that figure has not been officially confirmed in any public statement by the council.

Progress has been slower than initially planned. The Conservation Centre project was originally due to complete a first-pass audit by April 2026. That deadline passed without a public update.

How Liverpool Stacks Up Against Other Cities

Internationally, the picture is instructive. Amsterdam's Stadsarchief completed a major deduplication exercise across its 750,000-item digital collection in 2024, using automated tools developed in partnership with Delft University of Technology. The Dutch project became a reference case for Europeana precisely because it combined algorithmic detection with a tiered human review system — archivists only reviewed flagged pairs above a confidence threshold, cutting labour costs significantly.

Glasgow City Archives undertook a comparable review in 2023, working through collections held at the Mitchell Library on North Street. Glasgow's project benefited from a dedicated Scottish Government digital culture grant worth £340,000, which funded two full-time archivists for eighteen months. Liverpool has not secured equivalent ring-fenced funding, relying instead on existing staff capacity spread across heritage institutions.

Manchester's local authority archive took a different approach altogether, contracting a third-party digital preservation company to handle automated deduplication across its civic image holdings in 2025. The contract, publicly tendered through the council's procurement portal, was valued at just under £95,000. Liverpool has explored similar outsourcing options but, as of the time of writing, no contract has been awarded.

The gap matters to local heritage users. Residents researching family history, architectural historians working on planning applications, and journalists pulling historical images of streets like Scotland Road or the Dock Road all encounter the same problem: duplicated records clog search results and erode confidence in what the archive actually holds.

For anyone working with Liverpool's digital heritage collections now, archivists at the Record Office suggest using the most specific possible search terms and cross-referencing results against the physical catalogue where records predate 1990. A public update on the deduplication programme is expected from National Museums Liverpool before the end of September 2026 — the first formal progress report since the project began in January.